The Service Catalog and Ownership

The idea in one sentence

The Service Catalog answers a question none of metrics, logs, or traces can answer alone: who owns this service, what does it depend on, and where does its code actually live.

A definition, not a query

Metrics, logs, and traces are all things a service emits. A Service Catalog entry is different — it is a definition you declare about the service, adapted here for a billing-service:

{
  "apiVersion": "v3",
  "kind": "service",
  "metadata": {
    "name": "billing-service",
    "owner": "billing-team",
    "contacts": [
      { "name": "Billing oncall", "type": "slack", "contact": "https://my-org.slack.com/archives/billing" }
    ],
    "links": [
      { "name": "Runbook", "type": "runbook", "url": "https://runbook.example.com/billing-service" }
    ]
  },
  "spec": {
    "dependsOn": ["service:checkout-service"]
  },
  "datadog": {
    "codeLocations": [
      { "repositoryURL": "https://github.com/org/billing-service.git", "paths": ["**"] }
    ]
  }
}

metadata.owner is who to page. A metrics dashboard does not say who to notify at 3 a.m. when it fires. A log line does not say who wrote the service that produced it. A trace does not carry a Slack channel. metadata.owner is literally that missing field — a durable pointer to a team, independent of whether that team’s alert routing even happens to be wired correctly into a monitor today. The real schema also allows metadata.additionalOwners (an array of {name, type}) for services with more than one responsible team, and metadata.contacts/metadata.links (shown above) for exactly the kind of “who do I actually ping, where’s the runbook” detail that never has a home in a metric or a log line.

spec.dependsOn: an edge you declare, not one traffic infers

Compare spec.dependsOn with the Service Map from the APM module. The Service Map is auto-generated by observing live trace traffic between services — it shows what actually called what over the network, right now. spec.dependsOn is different: it is a manually declared edge in the Catalog, which means it can capture relationships that never generate a single span. billing-service might depend on a shared message queue that checkout-service publishes to asynchronously — no direct call, no trace connects them — or on a nightly batch job that reads a database table checkout-service owns. The Service Map shows you what talked to what; dependsOn lets you also declare what depends on what, including relationships that tracing was never going to see in the first place.

datadog.codeLocations: mapping a service to its source

repositoryURL and paths tie the abstract billing-service identity to an actual git repository, so tooling built on top of the Catalog — code security scanning, ownership-based review requirements — knows exactly where the code that produces this service’s telemetry lives, rather than someone maintaining that mapping in a spreadsheet or a wiki page that goes stale the moment it is written.

v2 vs. v3: the schema really did change shape, not just its version number

apiVersion: v3 above is not a cosmetic bump from an older v2-style document — the two are structurally incompatible, and the real schema enforces that. Built a document in the old shape (schema-version/dd-service/team at the top level, the format the Terraform provider’s own official examples still show first) and ran it through the exact same real entity.schema.json:

// @expect-error real jsonschema.Draft7Validator error, v2.1-shaped document under v3 rules
{
  "schema-version": "v2.1",
  "dd-service": "billing-service",
  "team": "billing-team",
  "contacts": [
    { "name": "Billing oncall", "type": "slack", "contact": "https://my-org.slack.com/archives/billing" }
  ]
}

$ python validate.py v2-doc.json
{"ok": false, "error": "[]: 'apiVersion' is a required property; []: 'kind' is a required property;
  []: 'metadata' is a required property; []: Additional properties are not allowed
  ('contacts', 'dd-service', 'schema-version', 'team' were unexpected)"}

Real, specific errors from the real schema: entity.schema.json declares additionalProperties: false at the top level, so a v2.1 document’s own top-level dd-service/team/schema-version/contacts fields are rejected outright, on top of the three now-missing required fields (apiVersion, kind, metadata). The v3 schema didn’t just rename schema-version to apiVersion — it moved ownership, contacts, and links underneath a nested metadata object entirely, which is exactly why a straight find-and-replace from an old document does not produce a valid new one.

The Terraform gotcha: the provider itself still expects the old shape

datadog_service_definition_yaml is the Terraform resource for managing Catalog entries as code — its only argument is service_definition, a raw YAML/JSON string (registry.terraform.io/providers/DataDog/datadog/latest/docs/resources/service_definition_yaml). Ran terraform init + terraform validate (pinned provider 4.22.0, no credentials, validate = false on the provider block so nothing reaches Datadog’s API) against both shapes side by side:

resource "datadog_service_definition_yaml" "billing_v2" {
  service_definition = <<EOF3
schema-version: v2.1
dd-service: billing-service
team: billing-team
EOF3
}

# @expect-error Must be a supported service schema: service_definition (terraform validate, provider 4.22.0, client-side)
resource "datadog_service_definition_yaml" "billing_v3" {
  service_definition = <<EOF2
apiVersion: v3
kind: service
metadata:
  name: billing-service
  owner: billing-team
spec:
  dependsOn:
    - service:checkout-service
EOF2
}

$ terraform validate
Error: Must be a supported service schema: service_definition

  with datadog_service_definition_yaml.billing_v3,
  on main.tf line 17, in resource "datadog_service_definition_yaml" "billing_v3":
  17:   service_definition = <<EOF2
  ...

The v2.1 resource validates cleanly (confirmed separately: Success! The configuration is valid.); the v3 resource — the exact same document that just validated successfully against the real, current entity.schema.json above — is rejected by terraform validate itself, entirely client-side, before any API call. The pinned datadog provider 4.22.0 bundles its own local schema-version allowlist for this resource, and as of this version, it does not yet recognize apiVersion: v3. This is a real, provable version-skew gap between two different parts of Datadog’s own tooling: the JSON Schema repo and the API have moved to v3, but this provider version’s local pre-validation for datadog_service_definition_yaml has not caught up — the officially documented Terraform examples for this exact resource show v2/v2.1/v2.2 only, never v3. If a service definition needs to be managed via this Terraform resource today, it needs the v2.x shape regardless of which shape is otherwise the current standard everywhere else.

flowchart LR
  S["billing-service\n(Service Catalog entry)"] --> O["owner: billing-team"]
  S --> D["dependsOn: checkout-service"]
  S --> C["codeLocations: github.com/org/billing-service"]
  D -.not necessarily seen in.-> Tr[APM traces]

A Catalog entry, independent of live telemetry

Reading AI-generated code

🐛 Spot the bug

# AI comment: "Define billing-service in the catalog using the current v3 schema"
resource "datadog_service_definition_yaml" "billing_service" {
service_definition = <<-EOT
  apiVersion: v3
  kind: service
  metadata:
    name: billing-service
    owner: billing-team
  spec:
    dependsOn:
      - service:checkout-service
EOT
}

Show the answer

The YAML embedded here is textbook-correct v3 — it would pass the exact same real entity.schema.json/service.schema.json validation proven earlier in this lesson, and an AI model trained on current Datadog documentation would have every reason to reach for apiVersion: v3 as “the current schema.” The bug, proven above with a real terraform validate run against the pinned provider 4.22.0: this specific Terraform resource’s own client-side validation does not accept apiVersion: v3 yet — terraform validate fails with Error: Must be a supported service schema: service_definition, entirely locally, before any API call. Being correct against the JSON Schema and the API is not the same as being correct against this one provider version’s terraform validate. The fix, until the provider is updated: write this resource’s service_definition using the v2.x shape (schema-version: v2.1 or v2.2, dd-service, team at the top level) — matching the provider’s own official examples, all of which show v2.x, none v3 — even though a raw JSON/YAML upload through the API or UI would accept the v3 document proven earlier in this lesson.

Mental Model

Think of the Service Catalog’s schema the way you’d think of a company’s own internal ID card format changing — the building’s security desk (Datadog’s API and the JSON Schema that describes what it accepts) updated to the new card format months ago, but one particular vendor’s badge-printing machine (this pinned Terraform provider version) is still stamping out cards in the old format, and it refuses to print a new-format card at all — not because the new format is wrong, but because nobody updated that one machine yet. Two systems can each be internally consistent and correct on their own terms, and still disagree with each other. The fix is never “which one is actually right” (the JSON Schema is unambiguously current) — it’s “which one am I talking to right now,” because that’s the one whose rules apply to this specific interaction.

Trade-offs

Managing Catalog entries via the API/UI directly vs. via datadog_service_definition_yaml — the API accepts the current v3 shape (proven above via real schema validation) and gives immediate feedback; Terraform gives you versioned, reviewable, reproducible infrastructure-as-code for the same entries, at the cost of inheriting whatever schema-version lag this specific provider release has (proven above: v2.x only, as of 4.22.0).
v3’s nested metadata/spec/datadog structure vs. v2.x’s flat top-level fields — v3’s nesting groups related fields together (all ownership/contact info under metadata, all Datadog-product integration under datadog) and separates “what is true about this service” from “what Datadog does with it”; v2.x’s flat structure is simpler to hand-write for a small service but has no such separation, and every field lives at the same top level regardless of what kind of information it represents.
spec.dependsOn (declared) vs. the Service Map (observed) — a declared edge captures relationships tracing structurally cannot see (a queue, a batch job) but goes stale the moment the real dependency changes and nobody updates the Catalog entry; the Service Map never goes stale in that way (it reflects whatever is actually happening right now) but is blind to any relationship that never produces a trace.

Common Misconceptions

“A JSON document that validates against the current v3 JSON Schema will also validate in terraform validate for datadog_service_definition_yaml.” No — proven above: the exact same v3 document that passed real jsonschema validation against entity.schema.json was rejected by terraform validate against the pinned provider 4.22.0, entirely client-side, with a real Must be a supported service schema error.
“v2 and v3 are just a version number bump — old documents mostly still work.” No — proven above: a real v2.1-shaped document run through the real v3 JSON Schema fails on four separate points at once (three missing required fields, four now-disallowed top-level fields) because v3 moved fields under a nested metadata object rather than renaming them in place.
“spec.dependsOn is redundant with the Service Map, since both describe dependencies.” No — spec.dependsOn is a manually declared edge that can capture relationships (a shared queue, a batch job) that generate zero trace traffic and therefore never appear on the Service Map at all; they answer overlapping but different questions.
“The Service Catalog only matters for services that are already emitting telemetry.” No — a Catalog entry (owner, dependencies, code location) is useful even for a service with zero current traffic; none of metrics, logs, or traces exist to answer “who owns this” for a quiet or newly-created service, which is exactly the gap this whole lesson describes.