Datadog as Code with Terraform

The idea in one sentence

The official Terraform Datadog provider lets you define monitors, dashboards, SLOs, synthetics, log pipelines, and even who is allowed to change any of that, as versioned, reviewable code, instead of clicking them together by hand in the UI, one environment at a time.

🐶 Proof: every HCL fence in this lesson, validated against the pinned provider

Every terraform-language fence below was dropped into its own directory next to a generated providers.tf pinning DataDog/datadog 4.22.0 (this course’s pinned version — see tools/probe/versions.json), with fake api_key/app_key and validate = false so no real API call ever happens, then run through terraform init -backend=false and terraform validate. Every one prints Success! The configuration is valid. The Terraform Registry API (registry.terraform.io/v1/providers/DataDog/datadog/versions) lists 4.23.0 as the current latest release as of this writing — one minor version ahead of this course’s pin. This lesson stays on 4.22.0 because that’s what tools/probe/ (harness-owned) has cached; if you’re following along on 4.23.0, re-run terraform validate yourself before trusting anything here to still be current.

Why bother managing Datadog as code

Clicking a monitor together in the UI is fast the first time. The trouble shows up the second time: you need the same monitor in staging, the same dashboard in a new region, or you need to know exactly what changed to a threshold last month and why. UI-built configuration has no diff, no review step, and no single source of truth — two people can quietly disagree about what a monitor’s threshold “should” be, and nothing catches it.

Managing monitors, dashboards, SLOs, and more through the Terraform Datadog provider turns all of that into a code change: it lives in a repository, it gets reviewed in a pull request like any other change to production, and the same module can be instantiated for prod, staging, and every new service that comes along by changing a variable instead of re-clicking a wizard.

A real datadog_monitor resource, with recovery thresholds and notify_no_data

Here is a metric alert monitor expressed as Terraform:

terraform/high-cpu-monitor.tf

resource "datadog_monitor" "high_cpu" {
  name    = "High CPU on checkout-service"
  type    = "metric alert"
  message = "CPU is high on checkout-service. Notify: @slack-sre-alerts"

  query = "avg(last_5m):avg:system.cpu.user{service:checkout-service} by {host} > 85"

  monitor_thresholds {
    warning           = 70
    warning_recovery  = 65
    critical          = 85
    critical_recovery = 80
  }

  notify_no_data    = true
  no_data_timeframe = 20

  tags = ["service:checkout-service", "team:checkout"]
}

• type — metric alert is one of several monitor types (others include log alert, apm alert, and the cost alert type covered in the next lesson).
• message — the notification body. The @slack-sre-alerts handle is a Datadog notification target, resolved the same way it would be if you typed it into the UI’s message box.
• query — the actual alerting condition, evaluated as avg(last_5m):avg:system.cpu.user{service:checkout-service} by {host} > 85.
• monitor_thresholds.critical/.warning — the ordinary trigger thresholds you’d otherwise drag in the UI.
• monitor_thresholds.critical_recovery/.warning_recovery — a separate, lower bar the metric has to cross back under before the monitor clears the alert state. Confirmed against the real provider schema (terraform providers schema -json, datadog_monitor.monitor_thresholds block): critical_recovery and warning_recovery are real, distinct attributes from critical/warning, not something you infer from a single threshold.
• notify_no_data / no_data_timeframe — whether the monitor should alert when its query stops returning data at all (as opposed to returning data that’s below threshold), and how many minutes of silence to tolerate first. Also confirmed as real top-level attributes on the same schema.
• tags — ordinary Datadog tags on the monitor object itself, useful for filtering the Manage Monitors page and for routing via Service Catalog ownership.

Why the recovery threshold matters, concretely: without critical_recovery, a monitor with critical = 85 flips to Alert the instant CPU crosses 85% and flips back to OK the instant it dips to 84.9% — if CPU is oscillating around 85, that’s a monitor flapping between Alert and OK every evaluation cycle, and a notification every time it flips. Setting critical_recovery = 80 means CPU has to drop meaningfully below the trigger point, not just barely under it, before the monitor clears — the gap between 85 and 80 is a deliberate hysteresis band.

The important caveat, unchanged from before: the query string in a datadog_monitor resource is not the same syntax you’d type into the Datadog UI’s monitor query builder. The UI builder is a guided widget that assembles a query for you; the Terraform query field is the raw API query syntax, and it has to be written by hand, correctly, the first time. By default, terraform plan sends that query to Datadog’s API for validation, so a malformed query fails at plan time rather than silently creating a broken monitor. If you genuinely need to skip that check — for example, referencing a metric that doesn’t exist yet in this environment — you can set validate = false on the resource, but that trades away your safety net, so it should be the exception, not the default.

Dashboards as code: the real widget query shape

The dashboard lesson from the Infrastructure & Metrics module covered named queries, formulas, and template variables from the UI’s point of view. The same widget model shows up in Terraform via datadog_dashboard, and getting the request block’s shape right is where an AI-generated example most commonly breaks.

🐶 Proof: the flat query shape a plausible-looking example uses does NOT validate

A widget request written like this — query { name = "query1"; data_source = "metrics"; query = "avg:..." } as flat attributes directly inside query — looks reasonable and reads like the JSON dashboard payload’s queries[] array flattened into HCL. Ran it through terraform init + terraform validate against the pinned provider 4.22.0:

$ terraform validate
Error: Unsupported argument
  on main.tf line 9, in resource "datadog_dashboard" "checkout_overview":
   9:           name        = "query1"
An argument named "name" is not expected here.

(the same error repeats for data_source and query — three “unsupported argument” errors, one per flattened attribute). Checked the real schema directly (terraform providers schema -json): widget.timeseries_definition.request has exactly three attributes of its own — display_type, on_right_yaxis, and q (a raw query string, an older/simpler form) — plus a set of block types including query and formula. The query block itself has zero attributes; it only nests further block types, one of which is metric_query — and metric_query is where name/data_source/query/aggregator actually live. The flat shape skips that nesting level entirely, which is exactly why it fails.

With layout_type = "ordered", widgets simply stack top to bottom in the order you declare them — the simplest option, and a good default. Here is the correct, validated shape:

terraform/checkout-dashboard.tf

resource "datadog_dashboard" "checkout_overview" {
  title       = "Checkout Service Overview"
  layout_type = "ordered"

  widget {
    timeseries_definition {
      request {
        query {
          metric_query {
            name        = "query1"
            data_source = "metrics"
            query       = "avg:system.cpu.user{service:checkout-service} by {host}"
          }
        }
        formula {
          formula_expression = "query1"
        }
        display_type = "line"
      }
    }
  }
}

With layout_type = "free", every widget needs its own widget_layout block, giving you exact pixel-grid placement with x, y, width, and height:

terraform/checkout-dashboard-free.tf

resource "datadog_dashboard" "checkout_overview_free" {
  title       = "Checkout Service Overview (free layout)"
  layout_type = "free"

  widget {
    timeseries_definition {
      request {
        query {
          metric_query {
            name        = "query1"
            data_source = "metrics"
            query       = "avg:system.cpu.user{service:checkout-service} by {host}"
          }
        }
        formula {
          formula_expression = "query1"
        }
        display_type = "line"
      }
    }
    widget_layout {
      x      = 0
      y      = 0
      width  = 4
      height = 2
    }
  }
}

Both of the above: terraform validate → Success! The configuration is valid. The shape is: request → query block → metric_query block (where name/data_source/query actually live) → sibling formula block (formula_expression referencing the query’s name) → display_type back at the request level, not inside query. This is the same request/query/formula split you’d see in the JSON dashboard API’s formulas/queries pair — Terraform just nests it one block deeper.

ordered is the right default for most operational dashboards — you rarely need to fight pixel positions for a dashboard that mostly gets scrolled top to bottom. free earns its keep for TV-wall dashboards or intentionally designed layouts where widgets need to sit at specific coordinates relative to each other.

datadog_dashboard vs. datadog_dashboard_json

There’s a second way to manage a dashboard in Terraform: datadog_dashboard_json takes the dashboard’s entire body as one JSON string via jsonencode(...), instead of a widget-by-widget HCL tree:

terraform/checkout-dashboard-json.tf

resource "datadog_dashboard_json" "checkout_from_json" {
  dashboard = jsonencode({
    title       = "Checkout Service Overview (JSON)"
    layout_type = "ordered"
    widgets = [
      {
        definition = {
          type = "timeseries"
          requests = [
            {
              queries = [
                {
                  name        = "query1"
                  data_source = "metrics"
                  query       = "avg:system.cpu.user{service:checkout-service} by {host}"
                }
              ]
              formulas     = [{ formula = "query1" }]
              display_type = "line"
            }
          ]
        }
      }
    ]
  })
}

Validated the same way — terraform validate passes. Notice this JSON body uses the flat shape (name/data_source/query as siblings inside one queries[] object) — that’s correct here, because this is the real Dashboards JSON API’s own payload shape, not the HCL block tree. The bug earlier in this lesson was using the JSON API’s flat shape inside HCL’s block syntax; datadog_dashboard_json is where that flat shape actually belongs.

• datadog_dashboard — every widget and query is a typed HCL block. Terraform’s own schema validates the structure (this is what caught the bug above at terraform validate time, before ever touching a real API). Diffs in a pull request are structural: “widget 3’s display_type changed from line to bars” reads clearly.
• datadog_dashboard_json — one opaque JSON blob. You lose per-field schema validation (a typo inside the JSON string is only caught by the API at apply time, or not caught by Terraform at all — jsonencode only guarantees valid JSON syntax, not a valid dashboard payload), but you gain the ability to paste a dashboard exported from the UI (Dashboard settings → Export dashboard JSON) directly in, without hand-translating every widget into HCL blocks.

The trade-off in practice: hand-authored new dashboards are easier to write correctly and review as datadog_dashboard; a dashboard someone already built by clicking around the UI and now wants under version control is far faster to bring in as datadog_dashboard_json.

flowchart LR
  A[Write datadog_monitor / datadog_dashboard in HCL] --> B[terraform plan]
  B --> C{Query validation}
  C -->|valid, or validate = false| D[Pull request review]
  C -->|invalid| E[Plan fails: fix query]
  D --> F[terraform apply]
  F --> G[Monitor / Dashboard / SLO created in Datadog]
  G --> H[Reused across prod, staging, new services via variables]

From HCL to live Datadog resources

datadog_service_level_objective

The SLOs lesson (Monitors, Alerting, and SLOs module) covered the three SLO types and the real burn_rate()/error_budget() query shapes as raw JSON. The same metric-based SLO as Terraform:

terraform/checkout-slo.tf

resource "datadog_service_level_objective" "checkout_availability" {
  name        = "Checkout API availability"
  type        = "metric"
  description = "Ratio of non-5xx checkout-api requests"

  query {
    numerator   = "sum:trace.http.request.hits{service:checkout-api,!http.status_code:5*}.as_count()"
    denominator = "sum:trace.http.request.hits{service:checkout-api}.as_count()"
  }

  thresholds {
    timeframe = "30d"
    target    = 99.9
  }

  tags = ["service:checkout-api", "team:checkout"]
}

terraform validate passes. The block shape mirrors the JSON payload closely — query.numerator/query.denominator and a thresholds block with timeframe/target — the main difference from clicking the SLO editor in the UI is that both the SLO and its burn-rate/error-budget alert monitors (from the earlier lesson) can now live in the same reviewed pull request.

datadog_synthetics_test, briefly

The Synthetics & Uptime Checks lesson already covers datadog_synthetics_test in depth, including a real terraform validate error for an invalid type. The as-code point worth repeating here: a synthetic test is just another resource in the same file as your monitors and dashboards, so “checkout-api has a monitor, a dashboard, and an uptime check” can be one reviewed change instead of three separate manual steps across three different UI screens:

terraform/checkout-uptime.tf

resource "datadog_synthetics_test" "checkout_api_up" {
  name      = "checkout-api uptime"
  type      = "api"
  subtype   = "http"
  status    = "live"
  message   = "@pagerduty-checkout-critical checkout-api synthetic failed"
  locations = ["aws:eu-central-1"]

  request_definition {
    method = "GET"
    url    = "https://checkout.example.com/health"
  }

  assertion {
    type     = "statusCode"
    operator = "is"
    target   = "200"
  }

  options_list {
    tick_every = 60
  }
}

terraform validate passes against the pinned provider.

Log pipelines and indexes as code

The Log Management module covered Processing Pipelines and indexes from the UI’s point of view. Both are ordinary Terraform resources too.

terraform/checkout-log-pipeline.tf

resource "datadog_logs_custom_pipeline" "checkout_pipeline" {
  name       = "checkout-api pipeline"
  is_enabled = true

  filter {
    query = "service:checkout-api"
  }

  processor {
    grok_parser {
      name       = "Parse checkout-api log line"
      is_enabled = true
      source     = "message"
      grok {
        match_rules  = "rule %%{date(\"yyyy-MM-dd'T'HH:mm:ss\"):timestamp} %%{word:log_level} %%{data:msg}"
        support_rules = ""
      }
    }
  }

  processor {
    date_remapper {
      name       = "Define timestamp as the official date"
      is_enabled = true
      sources    = ["timestamp"]
    }
  }

  processor {
    status_remapper {
      name       = "Define log_level as the official status"
      is_enabled = true
      sources    = ["log_level"]
    }
  }
}

🐶 A real HCL trap: Grok's own `%{...}` syntax collides with HCL's template syntax

The first draft of the grok_parser.grok.match_rules string above used a single %{date(...)} the way you’d type it straight into the Grok editor in the UI. terraform validate rejected it immediately: Error: Invalid template control keyword ... "date" is not a valid template control keyword, followed by a cascade of “Invalid multi-line string” errors on every following line. The cause: HCL’s own template syntax uses %{ ... } for template control directives (the for/if template constructs), the same bracket shape Grok patterns use for capture groups. Doubling it to %%{...} is HCL’s literal-percent escape — it tells the parser “this is a literal %{, not a template directive” — and the fence above, with %%{date(...)}, validates cleanly. This is a real, easy-to-hit trap any time a Datadog Grok pattern, which is always written with single %{...}, gets embedded inside a Terraform string.

terraform/checkout-log-index.tf

resource "datadog_logs_index" "checkout_index" {
  name = "checkout-api"

  filter {
    query = "service:checkout-api"
  }

  exclusion_filter {
    name       = "drop-debug-logs"
    is_enabled = true
    filter {
      query       = "status:debug"
      sample_rate = 0
    }
  }

  daily_limit    = 5000000
  retention_days = 15
}

resource "datadog_logs_index_order" "index_order" {
  name = "checkout index order"
  indexes = [
    datadog_logs_index.checkout_index.id,
    "main",
  ]
}

terraform validate passes for both. datadog_logs_index_order matters for a reason that’s easy to miss: log indexes are evaluated in order, and the first index whose filter.query matches a log claims it — datadog_logs_index_order.indexes is the literal, explicit ordering, referencing the new index’s id alongside the pre-existing "main" index by name. Get the order wrong and a narrower, purpose-built index like checkout-api can silently never receive a single log, because the broader main index (if listed first) claims everything before checkout-api’s filter is ever evaluated.

datadog_monitor_json

Same relationship as datadog_dashboard vs. datadog_dashboard_json — datadog_monitor_json takes the whole monitor body as one JSON string:

terraform/checkout-monitor-json.tf

resource "datadog_monitor_json" "high_cpu_json" {
  monitor = jsonencode({
    name    = "High CPU on checkout-service (JSON)"
    type    = "metric alert"
    query   = "avg(last_5m):avg:system.cpu.user{service:checkout-service} by {host} > 85"
    message = "CPU is high on checkout-service."
    tags    = ["service:checkout-service"]
    options = {
      thresholds = {
        critical = 85
        warning  = 70
      }
      notify_no_data = true
    }
  })
}

terraform validate passes. Same trade-off as the dashboard pair: datadog_monitor gives you schema-validated HCL blocks; datadog_monitor_json is faster for pasting in a monitor exported from the UI, at the cost of losing Terraform’s own structural validation on that JSON body.

Importing an existing monitor

Every resource so far assumed you’re creating something new. Datadog resources built by hand in the UI before you adopted Terraform don’t have to be recreated — they can be imported into Terraform state so Terraform starts managing something that already exists.

$ terraform import datadog_monitor.bytes_received_localhost 2081

This is the real import syntax from the provider’s own documentation (terraform-provider-datadog/docs/resources/monitor.md): the resource address (datadog_monitor.bytes_received_localhost — must already exist as an empty resource "datadog_monitor" "bytes_received_localhost" {} block in your .tf file) followed by the monitor’s numeric ID from the Datadog UI or API. datadog_monitor_json imports the same way — by the resource’s numeric ID directly, since the provider source (resource_datadog_monitor_json.go) wires it up with schema.ImportStatePassthroughContext, meaning the ID you pass on the command line is used as-is to fetch the monitor.

Importing only populates Terraform state — it does not generate the HCL for you. After terraform import, run terraform plan: any difference between the resource block you wrote and what actually exists in Datadog shows up as a planned change, which is your signal to adjust the HCL to match reality (or accept the plan and let Terraform reconcile the two) before this resource is safely under code review.

Provider authentication, per Datadog site

Every example above assumes a provider "datadog" block already exists. In practice, this is where the fake credentials in this lesson’s own validation setup come from:

# This IS the provider block -- not collected by the harness's --tf mode
# (which generates its own providers.tf per fence and would conflict with
# a second one), validated standalone in scratch with the exact fake
# all-zeros key/app-key this lesson uses everywhere else: terraform init +
# terraform validate -> Success.
terraform {
  required_providers {
    datadog = {
      source  = "DataDog/datadog"
      version = "4.22.0"
    }
  }
}

provider "datadog" {
  api_key = "<your-api-key>"
  app_key = "<your-app-key>"
  api_url = "https://api.datadoghq.eu/"
}

• api_key/app_key — required unless validate = false. Can also be set via the DD_API_KEY/DD_APP_KEY environment variables instead of hardcoding them in HCL (confirmed against the provider’s own schema documentation) — the standard pattern for keeping real credentials out of a Terraform file that gets committed to a repository.
• api_url — not controlled by DD_SITE. The provider docs are explicit about this: api_url “can also be set via the DD_HOST environment variable” (not DD_SITE), defaults to https://api.datadoghq.com, and must not end in /api/. If your org is on the EU site, https://api.datadoghq.eu/; other regions follow the same pattern (https://api.us5.datadoghq.com/, https://api.us3.datadoghq.com/, https://api.ddog-gov.com/ — see docs.datadoghq.com/getting_started/site/ for the full list). This is a real, easy trap: DD_SITE is the environment variable the Agent and dd-trace/ddtrace tracers read (see the Foundations module), but the Terraform provider reads a differently-named variable for the equivalent setting. Setting DD_SITE=datadoghq.eu in your shell and expecting the Terraform provider to pick it up silently does nothing — it keeps talking to datadoghq.com unless api_url/DD_HOST is also set correctly.

This lesson’s own validation never calls the real API at all — validate = false on the provider block, plus all-zeros fake api_key/app_key, is what makes terraform init + terraform validate safe to run with no Datadog account anywhere near it.

A real terraform validate error, on purpose

Every fence so far was written to be correct. Here’s a deliberately broken one, to show what a real schema violation looks like instead of a hallucinated-sounding one:

# @expect-error terraform validate real error: Unsupported argument — An argument named "notify_no_data_timeframe" is not expected here.
resource "datadog_monitor" "high_cpu" {
  name    = "High CPU on checkout-service"
  type    = "metric alert"
  message = "CPU is high on checkout-service."
  query   = "avg(last_5m):avg:system.cpu.user{service:checkout-service} by {host} > 85"

  monitor_thresholds {
    critical = 85
  }

  notify_no_data_timeframe = 20
}

Ran through the same terraform init + terraform validate as every other fence in this lesson:

Error: Unsupported argument
  on main.tf line 11, in resource "datadog_monitor" "high_cpu":
  11:   notify_no_data_timeframe = 20
An argument named "notify_no_data_timeframe" is not expected here.

There is no such attribute — the real names, confirmed against the schema earlier in this lesson, are the separate notify_no_data (boolean) and no_data_timeframe (number). notify_no_data_timeframe is the kind of name an AI model or a tired human might invent by blending the two real ones together, and Terraform’s own schema validation catches it immediately, before apply ever runs.

State and secrets: the app key ends up in state

A datadog_role or a monitor referencing sensitive query values isn’t usually the risk — the Terraform state file itself is. Every attribute Terraform reads back from Datadog’s API, including anything the provider itself considers sensitive, gets written into terraform.tfstate in plain text by default. The provider schema marks app_key (and api_key, bearer_token) as Sensitive: true, which redacts them from CLI output (terraform plan/terraform apply console text) — but Sensitive in a provider schema does not mean encrypted in state. Anyone who can read the raw state file can read those values.

This course doesn’t re-teach Terraform state management — the Terraform (AWS) course already covers remote state backends, state encryption, and team workflows for exactly this problem in depth: see state-management/sensitive-data-and-team-workflows. The Datadog-specific point worth remembering here: a provider "datadog" block with real credentials, applied without a properly access-controlled remote backend, puts your Datadog app key (which, per the RBAC lesson later in this module, can carry the same scoped permissions as a real user) into a file that needs the same protection as any other secret.

🐶 The one-line takeaway

The Terraform Datadog provider turns monitors, dashboards, SLOs, synthetics, log pipelines/indexes, and RBAC roles into versioned, reviewable code — but a datadog_dashboard widget query is query { metric_query {...} } formula {...}, not a flat query { name = ... } block (proven above against provider 4.22.0); _json-suffixed resources trade schema validation for pasting UI-exported JSON directly; api_url is controlled by DD_HOST, not DD_SITE; and the state file holding all of this needs the same access control as any other secret store.

Reading AI-generated code

🐛 Spot the bug

# AI comment: "Dashboard widget showing checkout-service CPU"
resource "datadog_dashboard" "checkout_overview" {
title       = "Checkout Service Overview"
layout_type = "ordered"

widget {
  timeseries_definition {
    request {
      query {
        name        = "query1"
        data_source = "metrics"
        query       = "avg:system.cpu.user{service:checkout-service} by {host}"
      }
      display_type = "line"
    }
  }
}
}

Show the answer

This is syntactically valid HCL — it parses fine — and it looks like a reasonable flattening of the JSON dashboard payload’s queries[] array. It is not valid against the real datadog_dashboard schema. Proven earlier in this lesson: terraform validate against the pinned provider 4.22.0 rejects name, data_source, and query as unsupported arguments directly inside the query block, because query has zero attributes of its own — it only nests further block types, and metric_query (not a flat set of attributes) is where those three fields belong, alongside a sibling formula block that references the query by name. The fix, matching the validated fence earlier in this lesson:

terraform/checkout-dashboard-fixed.tf

resource "datadog_dashboard" "checkout_overview_fixed" {
  title       = "Checkout Service Overview"
  layout_type = "ordered"

  widget {
    timeseries_definition {
      request {
        query {
          metric_query {
            name        = "query1"
            data_source = "metrics"
            query       = "avg:system.cpu.user{service:checkout-service} by {host}"
          }
        }
        formula {
          formula_expression = "query1"
        }
        display_type = "line"
      }
    }
  }
}

Neither the AI-generated comment nor a quick read of the HCL gives this away — the block nesting looks plausible either way. Only terraform validate against the real, pinned provider schema catches it.

Mental Model

The Terraform Datadog provider is a typed front door onto the same REST API the UI itself calls. Every resource "datadog_X" block is Terraform’s own schema description of one API object’s shape — which is exactly why terraform validate can catch a malformed datadog_dashboard widget query without ever contacting Datadog: it’s checking your HCL against the provider’s own compiled-in schema, not against a live API response. The _json-suffixed resources (datadog_dashboard_json, datadog_monitor_json) are the escape hatch from that schema: they hand the provider a raw JSON blob and skip the typed structure entirely, trading structural validation for the ability to paste something the UI already exported. Import (terraform import) is the reverse direction of the same idea — pulling an object that already exists outside Terraform into the provider’s typed view of it, one resource at a time.

Trade-offs

• datadog_dashboard/datadog_monitor vs. their _json counterparts — typed HCL blocks give you terraform validate’s structural safety net (proven above: it catches the widget-query bug before any apply) and cleaner pull-request diffs, at the cost of having to hand-translate every field; the _json variants let you paste a UI export verbatim, at the cost of losing that structural check — a typo inside the JSON string is invisible to terraform validate and only surfaces at apply time against the real API, or not at all if the field is simply ignored.
• validate = false on a monitor query — skips terraform plan’s real API round-trip to validate the query, useful when a metric genuinely doesn’t exist yet in this environment, at the direct cost of losing the safety net that would otherwise catch a malformed query before it ever reaches production.
• Importing vs. recreating — terraform import preserves a resource’s history and avoids a delete-then-recreate cycle (and any associated notification/alert gap), but only populates state — the HCL you write still has to be reconciled against whatever terraform plan reports as drift afterward, which is manual work proportional to how far the hand-built resource has drifted from a clean baseline.
• One log index vs. many, and the order between them — a single broad index is simpler to reason about and order, but a purpose-built index per high-value service (proven above: checkout-api with its own retention and daily limit) needs datadog_logs_index_order to be correct, or a broader pre-existing index placed earlier in the list can silently claim logs the narrower index was built to catch.

Common Misconceptions

• “The datadog_dashboard widget query block takes the same flat name/data_source/query shape as the JSON API payload.” No — proven above: that flat shape belongs inside datadog_dashboard_json’s jsonencode(...) body (the real API payload shape), but inside datadog_dashboard’s typed HCL blocks, those three fields live one level deeper, inside query.metric_query, alongside a sibling formula block.
• “DD_SITE controls which Datadog site the Terraform provider talks to.” No — proven above by quoting the provider’s own schema documentation directly: the provider’s api_url attribute is set via DD_HOST, not DD_SITE. DD_SITE is real and meaningful for the Agent and for dd-trace/ddtrace tracers (Foundations module), but the Terraform provider does not read it.
• “Marking app_key as Sensitive in the provider schema means it’s encrypted in the state file.” No — Sensitive: true redacts a value from CLI console output only. The raw value is still written into terraform.tfstate in plain text by default; protecting that file is a state-backend and access-control problem, covered in the linked Terraform (AWS) course lesson.
• “terraform import writes the HCL for the resource you’re importing.” No — import only populates Terraform state for a resource block you already wrote (even if empty). You still need terraform plan afterward to see, and then reconcile, any difference between your HCL and the real imported object.

What is the main benefit of managing Datadog monitors and dashboards via Terraform instead of the UI?

It is the only way to set a monitor threshold Configuration becomes versioned code that can be reviewed and reused across environments It removes the need for any notification targets It automatically deletes monitors that are no longer needed

What is the real, validated shape of a datadog_dashboard timeseries_definition widget query, proven above against provider 4.22.0?

request { query { name = "..."; data_source = "..."; query = "..." } } request { query { metric_query { name = "..."; data_source = "..."; query = "..." } } formula { formula_expression = "..." } } request { q = "name:...,data_source:...,query:..." } widget { metric_query { ... } }

What environment variable actually controls the Terraform Datadog provider's api_url (which Datadog site it talks to)?

DD_SITE DD_HOST DD_API_URL DATADOG_SITE

What is the key difference between datadog_monitor and datadog_monitor_json?

datadog_monitor_json cannot create monitors, only read them datadog_monitor uses typed HCL blocks validated by Terraform's schema; datadog_monitor_json takes the whole monitor as one JSON string, losing that structural validation They are identical in every way datadog_monitor_json is required for every monitor type

After running terraform import datadog_monitor.bytes_received_localhost 2081, what should you do next, and why?

Nothing -- the HCL is generated automatically by import Run terraform plan to see any difference between your written HCL and the real imported monitor, then reconcile it Delete the resource block, since import replaces the need for one Re-run terraform import a second time to confirm it worked

A datadog_logs_index_order lists a broad pre-existing "main" index BEFORE a new, purpose-built "checkout-api" index. What is the real, practical risk?

None -- index order in this resource is purely cosmetic The broader "main" index can claim logs before the "checkout-api" index's narrower filter is ever evaluated, so checkout-api's index may receive nothing Terraform will refuse to apply this configuration The two indexes will be merged into one automatically

Submit

Mark as complete