Skip to content
Datadog — From Zero to Hero

Review checklist

Run this list, by eye, on any diff — yours or an AI’s — after the tools in Verification tools have run and come back clean. It fits on one printed page on purpose. Every line links the catalog entry that explains why (the AI bug catalog numbers its entries 1–27), so this page stays a list rather than a re-run of the module. The groups follow the reading order in the module overview.

flowchart LR
  A["Agent config<br/>(conf.d, datadog.yaml)"] --> A1["configcheck: shape only<br/>agent status: loads and types"]
  T["Tracer options"] --> T1["tsc / .d.ts / harness --options:<br/>names. Config dump: values. Payload: order"]
  M["Custom metrics"] --> M1["Captured flush + dogstatsd-stats:<br/>type and cardinality. Meaning: review"]
  TF["Terraform"] --> TF1["terraform validate: names and types.<br/>Semantics: API only"]
  J["JSON monitors / SLOs / catalog"] --> J1["Model: types and required.<br/>_unparsed: enums. Grammar: API only"]
  K["Kubernetes wiring"] --> K1["Cluster node Agent configcheck.<br/>Not kubectl"]
  V["Versions and pins"] --> V1["docker image inspect, config dump,<br/>package versions"]
What actually catches each group, if anything

Agent configuration

Section titled “Agent configuration”
  • Does conf.d/<name>.d/ use the check’s name (redisdb, not redis), and does agent status — not just agent configcheck — list the check without a Loading Error? See catalog entry 3, proven with Check redis not found in Catalog.
  • Is instances: a list (- host: …), not a mapping? Entry 1, proven with cannot unmarshal !!map into []integration.RawMap.
  • Are tags: (and every other list-typed key) YAML lists, not env:prod scalars? Entry 2 — configcheck prints it happily; agent status refuses it.
  • Are numeric and boolean values really numbers and booleans (timeout: 5, not "five")? Entry 4, visible only as [ERROR] in agent status.
  • Is DD_SITE a domain (datadoghq.com), never a URL? Entry 6, proven with https://process.https://api.datadoghq.com..
  • Before filing “missing init_config” — did you run the tool? Entry 5: the file loaded. A finding that the tool does not reproduce is a note, not a bug.

Images and Kubernetes wiring

Section titled “Images and Kubernetes wiring”
  • Is the Agent image pinned to an exact tag or a digest rather than latest / a floating major, so a postmortem can name the Agent that ran? Entry 7 — the tags coincide today, which is why nobody notices.
  • Are Autodiscovery annotations on the Pod template (spec.template.metadata), keyed by the container name, in the .checks (v2, Agent 7.36+) form? Entry 8 — docs-only; nothing offline proves it. Verify on a real cluster with the node Agent’s agent configcheck.
  • Are DD_ENV, DD_SERVICE, DD_VERSION derived from one lower-case source (pod labels) on every workload? Entry 12.

Tracer options (Node.js and Python)

Section titled “Tracer options (Node.js and Python)”
  • Does every key passed to tracer.init({...}) exist in TracerOptions, spelled camelCase (logInjection, not log_injection)? Entry 9 and entry 11. TypeScript: tsc --noEmit; JavaScript: the harness --options or the startup config dump.
  • Is dd-trace initialised (or preloaded with --import dd-trace/initialize.mjs) before the libraries it must instrument? Entry 10 — proven with a captured payload that lacked express.request.
  • Is a “deprecated” claim about a setting backed by the docs page, not by memory? Entry 13 — DD_TRACE_SAMPLE_RATE is not deprecated and dd-trace 6.17.0 (Node) honours it, but ddtrace 4.15.2 (Python) silently ignores it; use DD_TRACE_SAMPLING_RULES for cross-language behaviour.
  • In Python: is every ddtrace.config.<x> a real attribute (a read, not a write that silently creates one)? Entry 14.
  • Is tracer.configure(...) called only with keyword arguments that exist in inspect.signature(tracer.configure)? Agent address belongs in DD_TRACE_AGENT_URL. Entry 15.
  • Does browser RUM init carry allowedTracingUrls (and not allowedTracingOrigins)? Entry 16 — tsc catches the name, only a trace in the UI catches the omission.

Custom metrics

Section titled “Custom metrics”
  • Where the requirement says “global percentile”, is the type a distribution, not a histogram? Entry 17 — the captured flush went to /api/beta/sketches versus /api/v1/series.
  • Is every tag value drawn from a small, bounded set (no user_id, request id, or raw URL)? Entry 18 — 50 values, 50 series.

Terraform

Section titled “Terraform”
  • Does every resource block validate with the pinned provider (terraform init && terraform validate), not the one the AI remembers? Entries 19–20.
  • In datadog_dashboard, is each request query query { metric_query { … } } plus a formula block? Entry 19.
  • Are recovery thresholds inside monitor_thresholds { … }, tags a list, and block names singular (template_variable)? Entry 20.
  • For sparse or low-traffic metrics, were notify_no_data, no_data_timeframe, and require_full_window set on purpose? Entry 21 — validate is green either way, and the provider default for notify_no_data is false.
  • Does datadog_role reference permissions by ID, and is logs_modify_indexes really meant for this role? Entry 22.
  • Are exclusion filters as narrow as the intent, and in the order you mean (first active match wins)? Entry 23.

Monitor, SLO, and catalog JSON

Section titled “Monitor, SLO, and catalog JSON”
  • Is a burn-rate query burn_rate("<slo_id>").over("<7d|30d|90d>").long_window("…").short_window("…") > N? Entry 24 — model validation is green for a wrong one; only the API and the docs grammar object.
  • Do composite monitor queries reference real monitor IDs? Entry 25.
  • Was the JSON checked with something stricter than the constructor (an _unparsed check on the deserialized object) so enum typos surface? Entry 26.
  • Is a Service Catalog entry apiVersion: v3, and did a human check its spec keys (the schema does not reject unknown ones)? Entry 27.

Tooling

Section titled “Tooling”
  • Was agent status read after agent configcheck, and was a check given at least one run? See Verification tools.
  • Does CI run npm run verify (options, Terraform, JSON models) on every fence, and node tools/verify-snippets.mjs --agent where Agent YAML is involved?
  • For every claim that no tool proved, is there a docs URL beside it — and a date or version?

Spot the bug

Section titled “Spot the bug”

🐛 Spot the bug

resource "datadog_dashboard" "checkout" {
title       = "Checkout overview"
layout_type = "ordered"


widget {
  timeseries_definition {
    request {
      q            = "avg:system.cpu.user{*}"
      display_type = "line"
    }
    title_size = "large"
  }
}


template_variables {
  name = "env"
}
}
Show the answer

One line, and terraform validate (provider 4.22.0) prints the fix for you: Unsupported block type ... Blocks of type "template_variables" are not expected here. Did you mean "template_variable"? The block is singular. The corrected dashboard validates:

terraform/review-checklist/template-variable.tf
resource "datadog_dashboard" "checkout" {
  title       = "Checkout overview"
  layout_type = "ordered"


  widget {
    timeseries_definition {
      request {
        q            = "avg:system.cpu.user{*}"
        display_type = "line"
      }
      title_size = "large"
    }
  }


  template_variable {
    name = "env"
  }
}

Mental Model

Section titled “Mental Model”
  • Order by cost of being wrong. Names and shapes (cheap, tool-checkable) first, then values and order (one layer deeper), then meaning (review or API).
  • Every line is a yes/no question with an owner. “Which command answers this?” is part of the line. A line without a command is a docs line and carries a URL.
  • Green means “not this class of error.” configcheck green rules out shape; it says nothing about the check loading.

Trade-offs

Section titled “Trade-offs”
  • Seven groups is a lot for a small diff. Skip a group when the diff does not touch it; do not skip the Tooling group.
  • The list can freeze folklore. Entries 5 and 13 are on this page because a plausible rule turned out to be false or only half true. Re-prove a line when the Agent, tracer, or provider version changes.
  • Docs-only lines age. Autodiscovery placement and the burn-rate grammar cite docs URLs; re-read them on upgrade.

Common Misconceptions

Section titled “Common Misconceptions”
  • “If the harness passes, the checklist is done.” The harness proves names, types, and shapes; the Monitor, SLO, and permission lines above are outside its reach.
  • “Checklist items are obvious.” Each of them was, at least once in this module, produced by an assistant that sounded confident.
  • “terraform validate checks against Datadog.” It checks against the provider schema; no request is made (validate = false on the provider block in the harness).
Why does the checklist tell you to read agent status after agent configcheck?
Which checklist group covers this, and what catches it?
import tracer from 'dd-trace';
tracer.init({ service: 'checkout', agentUrl: 'http://dd-agent:8126' });
Which line of the checklist cannot be answered by any offline tool in this module?
The checklist asks whether a 'deprecated' claim is backed by a docs page. What real case in this module motivates that line?
Why is the burn-rate line grouped with 'only the API and the docs grammar object' rather than with the tool-checked lines?
You review a Terraform change that sets critical_recovery = 1.5 on a datadog_monitor. What does the checklist expect?