Skip to content
Terraform & Terragrunt for AWS — From Zero to Hero

Sensitive Data and Team Workflows

The idea in one sentence

Section titled “The idea in one sentence”

Marking a value sensitive = true hides it from your terminal, but the value is still sitting in plaintext in the state file, so the real fix is protecting the state backend itself and keeping long-lived secrets out of Terraform entirely.

What sensitive = true actually does

Section titled “What sensitive = true actually does”

Both variable and output blocks accept a sensitive argument:

variable "database_password" {
  type      = string
  sensitive = true
}


output "db_connect_string" {
  value     = "Server=${aws_db_instance.main.address};Pwd=${var.database_password}"
  sensitive = true
}

With sensitive = true set, Terraform redacts the value everywhere it would otherwise show up on your terminal or in a CI log: it prints (sensitive value) in terraform plan and terraform apply output instead of the real string, and it refuses to print the raw value from terraform output unless you explicitly ask with terraform output db_connect_string. That is a genuinely useful guardrail — it stops a password from scrolling past in a shared CI log or a screen-shared terminal by accident.

Here is the misconception worth being explicit about: sensitive = true is a display feature, not an encryption feature. The value is computed, stored, and diffed by Terraform exactly the same way whether or not you mark it sensitive — it still gets written into terraform.tfstate in full plaintext, tags and all. Anyone who can read the raw state file, whether from an S3 bucket, a local .tfstate, or a terraform state pull, can see that password sitting right there in JSON, sensitive flag or no sensitive flag.

Terminal window
terraform state pull | grep -A3 db_connect_string
# "db_connect_string": {
#   "value": "Server=mydb.abc123.us-east-1.rds.amazonaws.com;Pwd=S3cretPassword!",
#   "type": "string"

Protecting state at rest

Section titled “Protecting state at rest”

Since marking a value sensitive does not remove it from state, the actual protection has to happen at the storage layer. Treat terraform.tfstate itself as a secret, the same way you would treat a .env file full of production credentials:

  • Encrypt the backend at rest. For the S3 backend covered in the previous lesson, that means enabling S3 server-side encryption (SSE) on the state bucket, so every object — including every version of your state file — is encrypted on disk regardless of who accesses the bucket API.
  • Restrict who can read the bucket. An IAM policy that scopes access to the state bucket down to the specific roles and users that legitimately need to run Terraform is what actually stops an unauthorized reader from ever reaching the plaintext, encryption at rest notwithstanding — encryption at rest protects against someone getting at the raw disk, not against someone with valid AWS credentials calling s3:GetObject.
resource "aws_s3_bucket" "state" {
  bucket = "my-company-terraform-state"
}


resource "aws_s3_bucket_server_side_encryption_configuration" "state" {
  bucket = aws_s3_bucket.state.id


  rule {
    apply_server_side_encryption_by_default {
      sse_algorithm = "AES256"
    }
  }
}

Where possible, the better move is avoiding the problem entirely: do not put long-lived secrets into Terraform variables in the first place if you can pull them at apply time instead. A data source that reads from a secrets manager fetches the current value when Terraform runs, without you ever typing the secret into a .tf or .tfvars file:

data "aws_secretsmanager_secret_version" "db_password" {
  secret_id = "prod/app-db/password"
}


resource "aws_db_instance" "main" {
  identifier     = "app-db"
  engine         = "postgres"
  instance_class = "db.t3.micro"
  username       = "app_admin"
  password       = data.aws_secretsmanager_secret_version.db_password.secret_string
}

Be clear-eyed about what this does and does not solve: the resolved secret value still ends up in aws_db_instance.main’s state entry, because Terraform has to record the attribute it set, exactly like the plaintext risk from the first lesson in this module. What it does solve is the secret’s lifecycle everywhere else — it is never typed into a variable, never sits in a .tfvars file that might get committed by accident, and rotating it in Secrets Manager does not require touching any Terraform configuration. Protecting the state backend itself is still the non-negotiable part.

nonsensitive(): deliberately un-redacting a value

Section titled “nonsensitive(): deliberately un-redacting a value”

Sometimes sensitive = true propagates further than you want — any expression that touches a sensitive value becomes sensitive too, even if the result genuinely is not secret (a boolean derived from a password’s length, say). The nonsensitive() function strips that marking from a single value, deliberately, for exactly that case. Real proof with terraform console, first showing the redaction, then removing it:

$ echo 'var.db_password' | terraform console
(sensitive value)
$ echo 'nonsensitive(var.db_password)' | terraform console
"S3cretPassword!"

That is the entire mechanism: nonsensitive() does not add any protection and does not remove the value from state either way — it only changes whether Terraform’s redaction applies to this particular expression’s result going forward. Reach for it only when you are certain the value is not actually secret in this context; calling it on an actual secret just to quiet a warning defeats the one guardrail sensitive = true provides.

Ephemeral values: excluded from state and plan by construction

Section titled “Ephemeral values: excluded from state and plan by construction”

Terraform 1.10 added ephemeral resources, variables, and outputs — values that exist only for the duration of a single Terraform operation and are never written to state or a saved plan file, not “written but redacted” the way sensitive = true values are. Mark a variable ephemeral like this:

variable "session_token" {
  type      = string
  ephemeral = true
}

An ephemeral block works the same way for a resource that only ever exists transiently — the built-in pattern is a short-lived credential or generated secret fetched fresh on every run, never persisted anywhere:

ephemeral "random_password" "db" {
  length = 16
}

Because an ephemeral value is never persisted, Terraform enforces where it is allowed to flow, at plan time, with a real error — not just a convention. Proof: passing ephemeral.random_password.db.result into an ordinary (non-write-only) resource argument:

$ terraform plan
Error: Invalid use of ephemeral value


  with terraform_data.marker,
  on main.tf line 15, in resource "terraform_data" "marker":
  15:   input = ephemeral.random_password.db.result


Ephemeral values are not valid for "input", because it is not a write-only
attribute and must be persisted to state.

That error names the actual rule directly: an ephemeral value may only flow into an attribute the provider has explicitly declared write-only, because a write-only attribute is the one kind of argument Terraform guarantees it will never persist. A second real constraint, proven the same way: an ephemeral output is not allowed at all in the root module —

$ terraform plan
Error: Ephemeral outputs are not allowed in context of a root module

— ephemeral outputs exist so a child module can hand an ephemeral value back up to whatever called it (to be consumed by that caller’s own write-only argument or further ephemeral chain), not so the root module can print one.

Write-only arguments (_wo): the other half of the mechanism

Section titled “Write-only arguments (_wo): the other half of the mechanism”

Terraform 1.11 added write-only arguments — resource attributes, conventionally suffixed _wo and paired with a companion _wo_version integer, that a provider promises never to write into state or a saved plan at all. Per the official example:

resource "aws_db_instance" "main" {
  instance_class       = "db.t3.micro"
  # ...
  password_wo          = ephemeral.random_password.db_password.result
  password_wo_version  = 1
}

password_wo_version is what tells Terraform the value changed and needs re-applying — since the value itself is never stored, Terraform has nothing to diff against on the next plan, so bumping the version number is the only signal that a new write-only value should be sent. This is real, documented mechanism (quoted from developer.hashicorp.com/terraform/language/manage-sensitive-data): “Providers define which managed resources support these arguments, and Terraform discards the values immediately after the operation is complete.”

Write-only arguments are opt-in per resource, per provider — a provider author has to explicitly declare write_only: true on a specific attribute in that resource’s schema. That is not a guess: terraform providers schema -json dumps the exact schema the pinned hashicorp/aws 6.66.0 provider ships, and aws_db_instance really does declare the pair:

$ terraform providers schema -json | python3 -c "
import json,sys
d = json.load(sys.stdin)
attrs = d['provider_schemas']['registry.terraform.io/hashicorp/aws']['resource_schemas']['aws_db_instance']['block']['attributes']
for k, v in attrs.items():
    if v.get('write_only') or k.endswith('_wo_version'):
        print(k, v)
"
password_wo {'type': 'string', 'description_kind': 'plain', 'optional': True, 'sensitive': True, 'write_only': True}
password_wo_version {'type': 'number', 'description_kind': 'plain', 'optional': True}

That schema dump needs no AWS credentials at all — terraform providers schema -json only reads what init already downloaded. What genuinely does need real credentials is applying it: aws_db_instance provisions a real RDS instance, which this course cannot do. Here is the honest gap instead: is this support universal across every resource, or only some? Checking hashicorp/random’s random_password — a resource this course can fully apply offline — with the identical query:

$ terraform providers schema -json | python3 -c "
import json,sys
d = json.load(sys.stdin)
attrs = d['provider_schemas']['registry.terraform.io/hashicorp/random']['resource_schemas']['random_password']['block']['attributes']
print([k for k,v in attrs.items() if v.get('write_only')])
"
[]

Zero. random_password has no write-only attribute at all, and neither does the built-in terraform_data or hashicorp/local’s resources. The real picture, proven both ways above: write-only support is not a Terraform-wide guarantee any resource gets for free at 1.11 — it is something each provider author adds one resource, one attribute at a time, and as of these pinned versions aws_db_instance.password_wo has it while none of this course’s fully-offline-appliable resources do. What is fully provable offline end-to-end, proven above, is the other half of the mechanism that makes write-only arguments useful in the first place: the ephemeral-value enforcement that decides what is even allowed to flow into a _wo argument once a resource has one.

Where shared state stops being enough

Section titled “Where shared state stops being enough”

The previous lesson set up an S3 backend with locking, which solves the “everyone reads and writes the same state safely” problem. That is necessary, but it is not sufficient once a team grows past a couple of engineers. With nothing else in place, “everyone runs terraform apply from their own laptop whenever they feel like it” still has real gaps:

  • Concurrent apply runs do not corrupt anything now that locking is in place, but they do queue up behind each other — a second engineer’s run just waits, with no visibility into what the first run is doing or how long it will take.
  • There is no natural point where a second person reviews a change before it goes live. Locking prevents two applies from racing; it does not stop one engineer from applying a change nobody else has looked at.
  • Each engineer needs valid AWS credentials with enough permission to run apply locally, which is a wider blast radius than most teams want to hand out to every laptop.

None of that means the shared backend from the previous lesson was wrong — it is a prerequisite, not a mistake. It just is not the whole story. Closing this gap is exactly what CI/CD for infrastructure is for: a pipeline runs plan on every proposed change for review before anyone applies it, and apply runs from a controlled pipeline identity instead of individual laptops. That is a topic for later in this course, not something to solve here — the point for now is recognizing the shape of the gap so it does not come as a surprise.

flowchart LR
  sv["variable with sensitive = true"] -->|redacted| cli["CLI output: (sensitive value)"]
  sv -->|full value written| state["terraform.tfstate: plaintext"]
  sm["Secrets Manager"] -->|data source at apply time| res["Resource attribute"]
  res -->|attribute recorded| state
A sensitive variable is redacted from CLI output but still lands in state as plaintext, versus a secret pulled from Secrets Manager at apply time

Mental Model

Section titled “Mental Model”

Three mechanisms in this lesson look similar from a distance — sensitive, ephemeral, and write-only arguments — but they solve genuinely different problems, and the difference is exactly what gets written to state. sensitive = true writes the value to state in full, and only redacts your terminal — proven above, the raw value is one state pull | grep away. ephemeral values are never written to state at all, and Terraform enforces that at plan time with a real error the moment you try to route one into an ordinary attribute — proven above. Write-only arguments are the provider-side half of that same guarantee: a _wo attribute is the one place an ephemeral value is allowed to land, specifically because the provider has promised never to persist it. Read in that order — display-only, structurally-excluded, provider-promised-never-persisted — the three stop looking like three flavors of “secret” and start looking like three different answers to “does this reach disk.”

Trade-offs

Section titled “Trade-offs”
ChoiceBenefitCost
sensitive = trueStops a value scrolling past in a shared terminal or CI log (proven above: (sensitive value))Zero effect on state — proven above, the raw value is still one state pull away
ephemeral variable/resourceStructurally cannot reach state or a saved plan — enforced by a real error, not a convention (proven above)Only usable in a narrow set of contexts (write-only arguments, provider config, other ephemeral values); a root-module ephemeral output is rejected outright (proven above)
Write-only (_wo) argumentsThe value genuinely never touches state, even transiently (per the provider’s own documented promise; proven above aws_db_instance really declares password_wo)Opt-in per resource per provider, not a blanket 1.11 guarantee — proven above, random_password has none at all; real use of aws_db_instance.password_wo needs a real RDS instance and AWS credentials
nonsensitive()Lets a value that is not actually secret escape unnecessary redaction (proven above via console)Removes the one guardrail sensitive = true provides if misapplied to something that actually is secret

Common Misconceptions

Section titled “Common Misconceptions”
  • “sensitive = true keeps a value out of the state file.” Proven above twice: the original state pull | grep example, and the baseline sensitivity claim — it only changes CLI display, never storage.
  • “ephemeral = true is just a stricter version of sensitive = true.” Proven above: they are different mechanisms entirely. sensitive redacts display but still writes to state; ephemeral is structurally barred from state and plan, enforced by a real “Ephemeral values are not valid for …” error the moment you try to persist one.
  • “Any resource can accept an ephemeral value if you just reference it directly.” Proven above: terraform plan refuses with a specific, named reason — the target attribute is “not a write-only attribute and must be persisted to state.” Only a _wo attribute a provider has explicitly built qualifies.
  • “Write-only arguments are a general Terraform 1.11 feature you can use with any resource.” Proven above by dumping two real provider schemas: aws_db_instance genuinely declares password_wo/password_wo_version, while random_password in the exact same pinned toolchain declares zero write-only attributes — it is opt-in, per resource, per provider, not a capability every resource gets for free at 1.11.

Reading AI-generated code

Section titled “Reading AI-generated code”

When reviewing generated code that touches secrets or ephemeral data, check for:

  • sensitive = true presented as “the fix” for a credential, with no mention of the backend still holding it in plaintext — proven above, that is exactly the gap this lesson closes.
  • An ephemeral value referenced directly in a plain (non-_wo) resource attribute, assumed to “just work” because it compiles — proven above, terraform plan rejects this with a specific error; it never gets to apply.
  • A root-module output marked ephemeral = true, intended to hand a generated secret to something outside Terraform — proven above, this is rejected outright; ephemeral outputs only exist for a child module handing a value back up to its caller.
  • nonsensitive() wrapped around a value that genuinely is secret, just to silence a “sensitive value in output” error, instead of marking the output sensitive = true — this defeats the one protection sensitive provides for no real gain.
  • A password_wo/secret_string_wo-style argument used without its paired _wo_version bump, then a report that “the new password never gets applied” — proven above, the version number is the only signal Terraform has that a new write-only value needs sending, since it never stores the value itself to diff against.

🐛 Spot the bug

ephemeral "random_password" "db" {
length = 16
}


resource "aws_db_instance" "main" {
identifier     = "app-db"
engine         = "postgres"
instance_class = "db.t3.micro"
username       = "app_admin"
password       = ephemeral.random_password.db.result
}
Show the answer

Proven above with the real error for the same mistake on terraform_data.marker: password on aws_db_instance is an ordinary, persisted attribute, not a write-only one — Terraform rejects this plan with “Ephemeral values are not valid for …, because it is not a write-only attribute and must be persisted to state.” The fix is not to give up on generating the password ephemerally; it is to route it through the resource’s actual write-only pair instead: password_wo = ephemeral.random_password.db.result plus a password_wo_version that increments whenever the value should change. Writing to the plain password argument would not even get as far as reaching AWS — the plan itself fails first, which is the safety net working as designed, not a bug to route around with sensitive = true (that would only redact display; the ephemeral value still could not be persisted to a non-write-only attribute).

What does setting sensitive = true on a Terraform variable actually protect against
A database password is marked sensitive = true on its variable block. Is it still readable in plaintext anywhere
What does this real error prove about where an ephemeral value is allowed to go?
$ terraform plan
Error: Invalid use of ephemeral value

  with terraform_data.marker,
  ...
  15:   input = ephemeral.random_password.db.result

Ephemeral values are not valid for "input", because it is not a write-only
attribute and must be persisted to state.
The pinned aws_db_instance schema really declares password_wo, but the pinned random_password schema declares zero write-only attributes. What does this actually prove about write-only arguments?
$ terraform providers schema -json | ... aws_db_instance ...
password_wo {'write_only': True} password_wo_version {...}
$ terraform providers schema -json | ... random_password ...
[]
Why does a root-module output marked ephemeral = true fail with "Ephemeral outputs are not allowed in context of a root module"
What is the real way to protect sensitive data stored in a Terraform state file