Skip to content
Datadog — From Zero to Hero

Metrics without Limits and Tag Cardinality

The idea in one sentence

Section titled “The idea in one sentence”

The real cost driver for custom metrics is tag cardinality — how many distinct tag-value combinations get indexed and made queryable — not how many raw data points you ingest, and the previous lesson’s own real 30-distinct-time-series proof is exactly what this lesson’s Metrics without Limits™ feature exists to let you recover from without re-instrumenting.

Cardinality is the cost, not volume

Section titled “Cardinality is the cost, not volume”

It’s tempting to think of metric cost the way you’d think of log or trace cost — more data points, more money. Custom metrics don’t quite work that way. Per docs.datadoghq.com/account_management/billing/custom_metrics/: “A custom metric is uniquely identified by a combination of a metric name and tag values (including the host tag)” — exactly the unit the previous lesson measured exploding to 30 distinct rows from one metric name and a single unique-per-call tag. Billing itself is computed the same way: “Your monthly billable custom metrics usage is calculated by taking the total of all distinct custom metrics (also known as timeseries) for each hour in a given month, and dividing it by the number of hours in the month” — and, critically, “Your billable usage is not impacted by data point submission frequency or the number of queries you run on your metrics.” Submission frequency is free; the number of distinct series is not.

What Metrics without Limits™ actually does

Section titled “What Metrics without Limits™ actually does”

Metrics without Limits™ (a registered trademark of the product name, per docs.datadoghq.com/metrics/metrics-without-limits/) decouples “what tags did I submit with this metric” from “what tags are actually indexed and queryable for this metric in Datadog.” Quoted directly: “Decouple custom metric ingestion and indexing to control volumes and costs by selecting which tags remain queryable.” The same docs page names the two volumes this splits billing into: Ingested Custom Metrics (“the original volume of custom metrics based on all ingested tags”) and Indexed Custom Metrics (“the volume of custom metrics that remains queryable in the Datadog platform”). In practice: you can keep submitting a metric with a high-cardinality tag like pod_name from your application or DogStatsD client — nothing about your instrumentation has to change, and the previous lesson’s real request_id example is exactly the kind of tag this feature is built to tame after the fact — and then configure that specific metric to only index a chosen subset of tags, say service and env. Datadog still ingests the raw submissions, but the queryable time series for that metric collapse down to the low-cardinality combination you configured, and that’s what you’re billed against for the indexed side.

The important part: this configuration is per custom metric, and it is non-destructive and reversible. If a few weeks later you’re debugging a problem and need to slice that same metric by pod_name for deeper investigation, you can turn that tag back on for querying — you are not lost forever just because it wasn’t indexed yesterday, and you never had to touch application code or redeploy anything to make that change.

Metric: myservice.request.duration
Submitted tags:   env, service, pod_name, availability_zone
Indexed tags (configured today):  env, service
  -> queryable combinations: small, low cost
Indexed tags (turned on later, for an investigation): env, service, pod_name
  -> queryable combinations: large, higher cost, but only while you need it

The metric_tag_configuration JSON, validated against the real model

Section titled “The metric_tag_configuration JSON, validated against the real model”

The API/Terraform-managed shape for a Metrics without Limits configuration is MetricTagConfigurationCreateRequest, confirmed to exist exactly under that name in the installed datadog_api_client.v2 module (not v1 — this is a v2-only resource). Real, validated example:

// myservice-request-duration-tags.json -- metric_tag_configurations is not
// one of this harness's collected --json directories (monitors/dashboards/
// slos/service-catalog/logs-pipelines only); validated by hand below against
// datadog_api_client.v2.model.metric_tag_configuration_create_request.
{
  "data": {
    "type": "metric_tag_configurations",
    "id": "myservice.request.duration",
    "attributes": {
      "metric_type": "distribution",
      "tags": ["env", "service"],
      "include_percentiles": true,
      "exclude_tags_mode": false
    }
  }
}
$ python3 -c "
from datadog_api_client.v2.model.metric_tag_configuration_create_request import MetricTagConfigurationCreateRequest
MetricTagConfigurationCreateRequest(**data, _check_type=True)
print('OK: valid')
"
OK: valid

Two real, caught errors worth knowing before they cost debugging time — both genuinely plausible AI-generated-code mistakes given this is a JSON:API-style envelope:

# Missing "id" -- a very natural mistake, since the metric name reads like
# it should be an "attributes.name" field, not the JSON:API "id"
$ python3 -c "
from datadog_api_client.v2.model.metric_tag_configuration_create_request import MetricTagConfigurationCreateRequest
MetricTagConfigurationCreateRequest(data={'type': 'metric_tag_configurations', 'attributes': {...}}, _check_type=True)
"
TypeError: MetricTagConfigurationCreateData.__init__() missing 1 required positional argument: 'id'
# "tags" as a comma-separated string instead of a JSON array -- plausible
# confusion with Datadog's own comma-separated tag-filter query syntax
$ python3 -c "
...
MetricTagConfigurationCreateRequest(data={..., 'attributes': {'tags': 'env,service', ...}}, _check_type=True)
"
ApiTypeError: Invalid type for variable 'tags'. Required value type is list and passed type was str at ['data']['attributes']['tags']

Percentile aggregations are opt-in, and only unlock a wider set on request

Section titled “Percentile aggregations are opt-in, and only unlock a wider set on request”

The include_percentiles field in the JSON above is a real, live field on the model, and it controls something documented explicitly for DISTRIBUTION metrics at docs.datadoghq.com/metrics/distributions/: by default, a distribution metric only offers count, min, max, sum, and avg as query-time aggregations. Opting in (per docs, done today via the Metrics Summary page in the UI, or the include_percentiles field shown here via API/Terraform) unlocks “p50, p75, p90, p95, and p99 or any user-defined percentile of your choosing (with up to two decimal points such as 99.99).” This is the same “choose the aggregation at query time, not submission time” idea the DogStatsD lesson introduced for DISTRIBUTION — percentile availability itself is a further, separate opt-in on top of that, not automatic just because a metric is a DISTRIBUTION.

Percentile aggregation at query time

Section titled “Percentile aggregation at query time”

A related idea that trips people up alongside cardinality: for a DISTRIBUTION metric, you don’t lock in “avg” or “p95” at submission time — you choose the aggregation when you actually query or graph the metric (once percentiles are unlocked per the section above). The same underlying distribution data can be viewed as avg, max, p50, p95, or p99 depending on what you select in the query editor or dashboard widget, without resubmitting anything. This is a different axis from tag cardinality — cardinality controls which tag combinations are queryable at all, aggregation controls how the numeric values within a chosen combination get summarized when you look at them.

flowchart LR
  A[App / DogStatsD submits high-cardinality tags: pod_name, az, env, service] --> B[Raw ingestion: cheap, all tags accepted]
  B --> C{Metrics without Limits config}
  C -->|indexed today: env, service| D[Queryable time series: low cardinality, lower cost]
  C -->|can enable later: + pod_name| E[Queryable time series: higher cardinality, for deep investigation]
Raw ingestion vs. indexed, queryable tags

Reading AI-generated code

Section titled “Reading AI-generated code”

🐛 Spot the bug

// AI comment: "Configure Metrics without Limits for myservice.request.duration"
{
"data": {
  "type": "metric_tag_configurations",
  "attributes": {
    "metric_type": "distribution",
    "tags": "env,service",
    "include_percentiles": true
  }
}
}
Show the answer

This looks reasonable at a glance — the right top-level envelope shape, a real metric type, a plausible-looking tags value. Two real bugs, both proven above against the real model: first, data.id — the metric name itself — is missing entirely; in this JSON:API-style payload the metric name is the resource id, not an attribute, and the real client raises TypeError: ... missing 1 required positional argument: 'id' immediately. Second, "tags": "env,service" is a comma-separated string, not a JSON array — a very natural mistake given Datadog’s own tag-filter query syntax elsewhere uses comma-separated strings, but this field requires a real JSON array (["env", "service"]), and the real client raises a real ApiTypeError naming the exact field and path. Both are real, caught, immediate failures — not silent misconfiguration — which is the one piece of good news here.

Mental Model

Section titled “Mental Model”

Think of a custom metric’s raw submissions as freight arriving at a port, and Metrics without Limits as customs deciding what gets warehoused versus what gets logged and waved through. Everything that arrives is inspected and counted (ingestion — free of charge, per the billing quote above, regardless of how often freight arrives), but only what customs decides to warehouse (index) takes up paid storage space and becomes something you can actually pull back out and query later. Changing what customs warehouses tomorrow doesn’t require the shipper to change anything about what they send — it’s a decision made entirely at the border, after the fact, and it’s reversible in either direction.

Trade-offs

Section titled “Trade-offs”
  • Configuring Metrics without Limits proactively vs. reactively — configuring low cardinality from day one keeps indexed cost predictable from the start, but you have to correctly guess which tags you’ll never need to query by; leaving everything indexed and narrowing later is safer for not-yet-understood systems but risks running up real indexed cost before anyone notices.
  • include_percentiles: true vs. the default aggregation set — the default count/min/max/sum/avg set (proven above via docs) covers most dashboards cheaply; opting into percentiles unlocks the accurate global p95/p99 queries that are often the actual point of choosing DISTRIBUTION over HISTOGRAM in the first place, at whatever additional cost Datadog’s current pricing associates with it (a claim this lesson does not make without a URL, per this course’s own rule).
  • This client’s strict-type-but-loose-enum validation (proven above) vs. a real API call — the local, offline validation this lesson (and the harness) relies on catches structural mistakes (missing required fields, wrong JSON types) immediately and for free; it does not catch every semantically-invalid value (proven above with "histogram" sailing through), so a real API call or --tf Terraform validate remains the last line of defense for that class of bug.

Common Misconceptions

Section titled “Common Misconceptions”
  • “Custom metric cost scales with how often I submit data points.” No — quoted directly from Datadog’s own billing docs above: “Your billable usage is not impacted by data point submission frequency.” Cost scales with the number of distinct metric-name-plus-tag-value series, proven in the previous lesson.
  • “Metrics without Limits changes what my application submits.” No — proven above by the model’s own shape: the configuration lives entirely in a separate metric_tag_configuration resource keyed by metric name; the application’s DogStatsD/check submissions are completely untouched.
  • “A metric_tag_configuration JSON that validates cleanly against the real model is guaranteed correct.” Not quite — proven above: this client validates required fields and types strictly, but a metric_type value like "histogram" that isn’t in the real allowed_values set still passes _check_type=True without error, a real, documented gap this course’s harness README calls out for the exact same reason.
  • “Percentile aggregations are automatically available on any DISTRIBUTION metric.” No — per docs quoted above: the default aggregation set for a distribution is count/min/max/sum/avg; percentiles (p50…p99, or custom) are a separate, explicit opt-in.
What is the primary driver of custom metric cost in Datadog, per Datadog's own billing docs?
What does Metrics without Limits™ let you do, quoted directly from its own docs?
Proven above with a real MetricTagConfigurationCreateRequest, what happens when data.id (the metric name) is omitted from the JSON:API-style payload?
Proven above by introspecting the real, installed model's allowed_values, which metric_type is notably absent from the valid values {gauge, rate, count, distribution} -- and why does that make sense given earlier lessons?
Proven above with a real ApiTypeError, what is wrong with submitting "tags": "env,service" (a comma-separated string) in a metric_tag_configuration payload?
Per docs.datadoghq.com/metrics/distributions/, what aggregations are available on a DISTRIBUTION metric by default, before opting into percentiles?