Security and Compliance Basics

The idea in one sentence

Beyond dashboards and alerts, Datadog also covers cloud security posture and runtime threats, sensitive data detection, and fine-grained access control — and that last one directly protects the cost and reliability decisions taught earlier in this course, provided you read what each permission really grants.

Cloud Security Management: what it is called now

The product family this lesson calls Cloud Security Management (CSM) has been renamed and re-split more than once, and older tutorials (and older AI output) still say CSPM (Cloud Security Posture Management), which was only the posture half. What the current docs actually say:

The docs page at https://docs.datadoghq.com/security/cloud_security_management/ is titled “Cloud Security” and opens with: “Datadog Cloud Security delivers deep visibility, continuous configuration audits, identity risk assessments, vulnerability detection, and real-time threat detection across your entire cloud infrastructure”. The URL path still says cloud_security_management; the page text does not use the name “Cloud Security Management” at all.
Its listed features are Misconfigurations (“Tracks the security hygiene and compliance posture of your production environment”), Identity Risks and Vulnerabilities (same URL).
Runtime detection has its own docs section, Workload Protection (https://docs.datadoghq.com/security/workload_protection/): “Datadog Workload Protection provides real-time visibility and defense for your infrastructure by continuously monitoring file, network, and process activity across your environment”.
The old term survives only in the docs navigation tree, which still labels one entry “Cloud Security Posture Management”.

So the safe vocabulary is: Cloud Security (CSM is the umbrella name you will still see in the wild) for Misconfigurations / Identity Risks / Vulnerabilities, and Workload Protection for runtime threats. If a lesson, PR or AI answer says “enable CSPM”, treat it as the posture half only and check the current product name in the docs before you write it into a runbook.

The two problems the family solves, conceptually:

Posture management (Misconfigurations) — continuously checking your cloud accounts and resources for misconfigurations, like an S3 bucket left publicly readable, an overly permissive IAM policy, or a security group open to the world. These are findings about how things are configured, independent of whether anything bad has happened yet.
Runtime threats (Workload Protection) — detection rules that run against live signals from hosts, containers, and Kubernetes (process activity, file access, network connections) to catch things actually happening right now, like a container spawning an unexpected shell.

The point for this course: it is not a separate product you bolt on with new instrumentation — docs say it “leverages both the Datadog Agent and Agentless”, so much of it reuses the Agent and cloud integrations you configured in the Foundations module, pointed at a different rule engine. Exact pricing and which features are on for your plan are account-specific; this lesson does not quote numbers.

Sensitive Data Scanner

Sensitive Data Scanner (SDS) looks for patterns that match sensitive data — credit card numbers, API keys, IP addresses, PII, or a custom regex — inside telemetry as it flows through Datadog. From the docs (https://docs.datadoghq.com/security/sensitive_data_scanner/): it covers “logs, APM spans, RUM events, and events”, and the UI offers four actions: Redact (replace with a chosen token), Partially redact, Hash (a “non-reversible unique identifier”) and Mask, where Mask is available for logs, APM spans and RUM events only. It does this without requiring you to change what your application actually logs, which is your safety net for the sensitive data that inevitably ends up in log lines.

Scanning configuration can live in Terraform too, and the provider’s vocabulary is not the UI’s vocabulary. This validates against provider 4.22.0:

resource "datadog_sensitive_data_scanner_group" "checkout_logs" {
  name         = "checkout-api logs"
  description  = "Scan checkout-api logs for card numbers"
  is_enabled   = true
  product_list = ["logs"]

  filter {
    query = "service:checkout-api"
  }
}

resource "datadog_sensitive_data_scanner_rule" "card_number" {
  name       = "credit card number"
  group_id   = datadog_sensitive_data_scanner_group.checkout_logs.id
  is_enabled = true
  pattern    = "\\d{16}"

  text_replacement {
    type = "hash"
  }
}

text_replacement.type accepts none, hash, replacement_string, partial_replacement_from_beginning or partial_replacement_from_end (provider docs for datadog_sensitive_data_scanner_rule, tag v4.22.0) — there is no redact or mask value. product_list is checked at validate time; a plausible-looking ["log"] (singular) fails locally:

Error: Invalid enum value
  on main.tf line 27, in resource "datadog_sensitive_data_scanner_group" "logs_group":
  27:   product_list = ["log"]
`invalid value 'log' for SensitiveDataScannerProduct: valid values are [logs
rum events apm]`

Also note the docs list separate permissions for the scanner: data_scanner_read, data_scanner_write and data_scanner_unmask (“Unmask (view) sensitive data that was previously masked”), all defaulting to the Datadog Admin Role (https://docs.datadoghq.com/account_management/rbac/permissions/) — who can un-mask matters as much as the rule itself.

The concrete example: granular RBAC for Log Management

Security posture and data scanning are about telemetry content. RBAC (role-based access control) is about who can do what inside Datadog itself, and Log Management is one of the best places to see how granular that gets — decisions about what gets indexed, excluded, or archived are direct cost decisions.

What the docs say about the log permissions (https://docs.datadoghq.com/account_management/rbac/permissions/, default role in brackets):

logs_modify_indexes [Datadog Standard Role] — “Read and modify all indexes in your account. This includes the ability to grant the Logs Read Index Data and Logs Write Exclusion Filters permission to other roles, for some or all indexes.”
logs_write_exclusion_filters [Standard] — “Add and change exclusion filters for all or some log indexes. Can be granted in a limited capacity per index to specific roles via the Logs interface or API. If granted from the Roles interface or API, the permission has global scope.”
logs_write_pipelines [Standard] — “Add and change log pipeline configurations, including the ability to grant the Logs Write Processors permission to other roles”.
logs_write_archives [Admin] — “Add and edit Log Archives.”
logs_read_archives [Read Only] — “Read Log Archives location and use it for rehydration.”
logs_generate_metrics [Standard] — “Create custom metrics from logs.”
logs_write_facets [Standard] — “Create or edit Log Facets.”
logs_read_data [Read Only] — “Read log data. In order to read log data, a user must have both this permission and Logs Read Index Data.”

The longer Logs RBAC guide (https://docs.datadoghq.com/logs/guide/logs-rbac-permissions/) adds what logs_modify_indexes covers in practice: setting the index filters that route logs into an index, setting retention for an index, and “Granting another role the Logs Read Index Data and Logs Write Exclusion Filters permissions” — and “This permission is global and enables both the creation of new indexes, and the edition of existing ones.”

The trap: three ways a log role goes wrong

logs_modify_indexes is not just a cost knob, it is a delegation right, and it is global. Whoever holds it can create indexes, change routing filters and retention, and hand logs_read_index_data / logs_write_exclusion_filters to other roles. There is no “modify only my team’s indexes” version of it: to scope exclusion-filter rights to a subset of indexes the guide says to remove the global permission and grant it per index in the UI, and “This configuration is only supported through the UI” — so a Terraform-only role cannot express the per-index form.
The default-role column may surprise you. The permissions table lists Datadog Standard Role as the default holder of logs_modify_indexes, logs_write_exclusion_filters and logs_write_pipelines. If you assumed only a platform team could change indexes, you have to check that your users are not still on the managed Standard role. Docs: users “can clone them to create custom roles with specific permissions”.
Reading logs needs two permissions. logs_read_data alone does not let a user read logs — the docs say the user “must have both this permission and Logs Read Index Data”. The guide calls logs_read_index_data the legacy per-index approach but adds “it is still required to have this permission enabled to access indexed data”. A custom “log viewer” role built from only logs_read_data looks right in review and returns nothing to the user.

There is a Terraform-side blind spot as well. The datadog_permissions data source exposes permissions as a map keyed by permission name, resolved against the API at plan time. terraform validate cannot know the keys, so a misspelled key passes. I ran this with the pinned provider:

resource "datadog_role" "typo" {
  name = "Typo"
  permission { id = data.datadog_permissions.dd_perms.permissions.logs_modify_indexez }
}
$ terraform validate
Success! The configuration is valid.

Contrast this with the earlier lesson, where a misspelled attribute (notify_no_data_timeframe) failed at validate. A misspelled permission name is only caught later (plan/apply, which needs a live API — not run here, so what plan prints is unproven). Review permission names by reading, not by trusting a green validate.

A least-privilege investigator role that does read logs, can build log-based metrics, and cannot touch indexes:

data "datadog_permissions" "dd_perms" {}

resource "datadog_role" "log_investigator" {
  name = "Log Investigator"

  permission {
    id = data.datadog_permissions.dd_perms.permissions.logs_read_data
  }
  permission {
    id = data.datadog_permissions.dd_perms.permissions.logs_read_index_data
  }
  permission {
    id = data.datadog_permissions.dd_perms.permissions.logs_generate_metrics
  }
}

The provider docs also describe default_permissions_opt_out on datadog_role: “If set to true, the role does not have default (restricted) permissions unless they are explicitly set”, and that the include_restricted attribute of the data source must be set to manage default permissions in Terraform. I did not run a plan to observe which defaults a new role receives, so audit the resulting role in the UI after the first apply rather than assuming it holds only what you listed.

flowchart LR
  A["Investigator role"] -->|"logs_read_data + logs_read_index_data, logs_generate_metrics"| B["Search logs, build log-based metrics"]
  C["Platform / SRE custom role"] -->|"logs_modify_indexes, logs_write_exclusion_filters, logs_write_pipelines"| D["Change what is indexed, excluded, parsed"]
  D -->|"logs_modify_indexes can also grant read + exclusion rights"| A
  E["Auditor role"] -->|"logs_read_archives"| F["Archive location and rehydration"]
  D -->|"directly affects"| G["Log cost from indexed volume and retention"]

Splitting read access from index/exclusion/archive control

Reading AI-generated code

🐛 Spot the bug

# AI comment: "Read-only role so engineers can search logs and see index config"
data "datadog_permissions" "dd_perms" {}

resource "datadog_role" "log_viewer" {
name = "Log Viewer"

permission {
  id = data.datadog_permissions.dd_perms.permissions.logs_read_data
}
permission {
  id = data.datadog_permissions.dd_perms.permissions.logs_modify_indexes
}
}

Show the answer

This validates (terraform validate prints Success!), which is exactly why it is dangerous. Two problems that no validator sees:

The comment says “read-only” but logs_modify_indexes is not a read permission. Docs: “Read and modify all indexes in your account”, global, and it includes the right to grant other roles Logs Read Index Data and Logs Write Exclusion Filters. Every engineer in this role can change retention and routing for every index — and hand out more rights.
The role still cannot read logs: docs require both logs_read_data and logs_read_index_data. The intended read path is missing.

The fix swaps the dangerous permission for the required one (validated against provider 4.22.0):

data "datadog_permissions" "dd_perms" {}

resource "datadog_role" "log_viewer" {
  name = "Log Viewer"

  permission {
    id = data.datadog_permissions.dd_perms.permissions.logs_read_data
  }
  permission {
    id = data.datadog_permissions.dd_perms.permissions.logs_read_index_data
  }
}

Mental Model

Split permissions into three bands: read (logs_read_data + logs_read_index_data, logs_read_archives), shape (logs_write_pipelines, logs_write_facets, logs_generate_metrics — change how logs are understood), and spend (logs_modify_indexes, logs_write_exclusion_filters, logs_write_archives — change what is kept and for how long, i.e. the bill). logs_modify_indexes also sits in the “delegate” band because it can grant the other two spend/read rights. Security products follow the same rule of thumb: posture and runtime products detect, Sensitive Data Scanner transforms telemetry, and RBAC decides who can turn any of it off.

Trade-offs

Custom roles vs managed roles — managed roles are maintained by Datadog and “may be automatically updated by Datadog as new features are added”, so their content drifts under you; a custom role (cloned or written in Terraform) is stable and reviewable but you own keeping it complete, and logs_read_data without logs_read_index_data shows how easily a hand-built role ends up subtly broken.
Global vs per-index exclusion-filter rights — per-index scope limits blast radius but, per the docs, is configured through the UI only, so it cannot live in your Terraform code.
Hash vs redact vs mask in SDS — a hash is non-reversible, redact/partial redact keep a token or a portion for debugging, and mask has the narrowest product coverage (logs, APM spans, RUM events). Pick per data class; the docs list unmask as its own permission.
Detect posture vs detect runtime — Misconfigurations tells you a door is unlocked, Workload Protection tells you someone walked through it; you generally want both, at the cost of more Agent-side collection.

Common Misconceptions

“CSPM is the current name of Datadog’s cloud security product.” The current docs title is “Cloud Security” with Misconfigurations, Identity Risks and Vulnerabilities; runtime is Workload Protection. CSPM is the older, posture-only name that survives in the docs nav tree.
“logs_modify_indexes is a platform-team-only cost permission.” By default it is held by the managed Standard role and it is global; it also lets the holder grant Logs Read Index Data and Logs Write Exclusion Filters to other roles.
“logs_read_data is enough to read logs.” The docs require it together with Logs Read Index Data.
“A green terraform validate means my datadog_role grants what I intended.” It only checks HCL shape; permission names are map keys resolved at plan time, so a typo passes validate (proven above).
“SDS has the same action names in Terraform as in the UI.” The provider’s text_replacement.type values are none, hash, replacement_string and two partial_replacement_* values.