Skip to content
Terraform & Terragrunt for AWS — From Zero to Hero

The State File

The idea in one sentence

Section titled “The idea in one sentence”

Terraform state is a JSON file that maps every resource in your configuration to the real-world object it created, and it is the only thing that lets Terraform know what it manages.

Why Terraform needs state at all

Section titled “Why Terraform needs state at all”

When you run terraform apply, Terraform creates resources through a cloud provider’s API — an EC2 instance, an S3 bucket, an IAM role. The provider API has no concept of “Terraform.” If you have 500 EC2 instances in an AWS account, the AWS API cannot tell you which of those instances Terraform created for a given resource "aws_instance" "web" block, which ones a teammate created by hand in the console, and which ones belong to a completely different Terraform configuration.

Terraform solves this by keeping its own record. Every time you apply, Terraform writes the mapping between your configuration’s resource addresses and the real infrastructure IDs into a state file, by default named terraform.tfstate:

resource "aws_instance" "web" {
  ami           = "ami-0abcdef1234567890"
  instance_type = "t3.micro"
}

After apply, the state file records something like aws_instance.web -> i-0123456789abcdef0, plus every attribute AWS returned for that instance — its private IP, its ARN, its security group associations, and so on. That cached data means Terraform does not have to call the AWS API and re-describe every resource just to compute a plan; it can compare your configuration against the last known state first, which is both how it detects what changed and a real performance win on large configurations. State is also where Terraform stores dependency metadata, so it knows the order in which resources must be created, updated, or destroyed.

The plaintext risk hiding in your state file

Section titled “The plaintext risk hiding in your state file”

State files are not sanitized. Any attribute value Terraform receives from a provider — or that you set directly in your configuration — is written into terraform.tfstate as plaintext, including values that are secrets. A common example:

resource "aws_db_instance" "main" {
  identifier     = "app-db"
  engine         = "postgres"
  instance_class = "db.t3.micro"
  username       = "app_admin"
  password       = "S3cretPassword!"
}

You never ran terraform output on that password and you never printed it in a log line. It still lands in terraform.tfstate in plaintext, because Terraform has to remember every attribute it set in order to detect drift on the next plan. Anyone who can read that file can read the password. (sensitive-data-and-team-workflows.mdx covers sensitive = true, ephemeral, and write-only arguments — none of which make this particular example safe, and the proof there shows exactly why.)

This is why terraform.tfstate must never be committed to version control. Add it to .gitignore from the very first commit of a new project:

.gitignore
terraform.tfstate
terraform.tfstate.backup
*.tfstate
*.tfstate.*

The next lesson covers moving state off your laptop entirely and into a properly access-controlled remote backend, which is the real fix for this risk — a .gitignore entry only stops the most obvious accident.

Anatomy of a real state file

Section titled “Anatomy of a real state file”

The examples below are not reconstructed from memory — they come from a real terraform apply against two resources that need no cloud credentials at all: terraform_data (the built-in provider’s replacement for the deprecated null_resource) and local_file (hashicorp/local). The config:

main.tf
terraform {
  required_providers {
    local = {
      source  = "hashicorp/local"
      version = "~> 2.5"
    }
  }
}


resource "terraform_data" "trigger" {
  input = "v1"
}


resource "local_file" "config" {
  filename = "${path.module}/generated/app.conf"
  content  = "listen=8080\ntrigger_id=${terraform_data.trigger.output}\n"
}

After terraform init -backend=false && terraform apply -auto-approve, the real terraform.tfstate on disk (machine paths stripped):

{
  "version": 4,
  "terraform_version": "1.16.4",
  "serial": 6,
  "lineage": "91789eb2-a206-8e38-2571-62323b581164",
  "outputs": {},
  "resources": [
    {
      "mode": "managed",
      "type": "local_file",
      "name": "config",
      "provider": "provider[\"registry.terraform.io/hashicorp/local\"]",
      "instances": [
        {
          "schema_version": 0,
          "attributes": {
            "content": "listen=8080\ntrigger_id=v1\n",
            "content_sha1": "7551ae2164513c74b5c0665a6b28b6a40399686f",
            "filename": "./generated/app.conf",
            "id": "7551ae2164513c74b5c0665a6b28b6a40399686f"
          },
          "dependencies": ["terraform_data.trigger"],
          "sensitive_attributes": [[{ "type": "get_attr", "value": "sensitive_content" }]]
        }
      ]
    },
    {
      "mode": "managed",
      "type": "terraform_data",
      "name": "trigger",
      "provider": "provider[\"terraform.io/builtin/terraform\"]",
      "instances": [
        {
          "schema_version": 0,
          "attributes": {
            "id": "7e48645a-6b7e-14c9-d8b3-99d042610145",
            "input": { "value": "v1", "type": "string" },
            "output": { "value": "v1", "type": "string" }
          },
          "sensitive_attributes": []
        }
      ]
    }
  ],
  "check_results": null
}

Every top-level field means something specific:

  • version — the state file format version (4 since Terraform 0.12; unrelated to the Terraform binary version).
  • terraform_version — the actual CLI version that last wrote this file.
  • serial — a counter incremented on every write. Two state files claiming the same lineage are compared by serial to decide which is newer.
  • lineage — a UUID generated once, the first time state is created for a working directory. It never changes for the life of that state, and it is how Terraform tells “an older copy of the same state” apart from “an unrelated state file that happens to look similar.”
  • resources[] — one entry per resource block (not per instance — count/for_each share one entry with multiple instances[]), each with mode, type, name, provider, and instances.
  • instances[].attributes — every attribute the provider returned after create/read, the same data terraform state show prints.
  • instances[].dependencies — the resource addresses this instance’s config referenced (proven above: local_file.config depends on terraform_data.trigger because its content interpolates terraform_data.trigger.output). This is exactly the ordering metadata mentioned above — it is why a destroy tears things down in the right order without recomputing dependencies from scratch.

terraform show -json: the same data, provider-agnostic and stable

Section titled “terraform show -json: the same data, provider-agnostic and stable”

terraform state show above is a human-readable format meant to be read by a person. terraform show -json is the machine-readable contract other tools (Sentinel, OPA/conftest, custom CI checks) are meant to parse — it is documented, versioned (format_version), and does not change shape between provider versions the way raw state internals sometimes do. Real output (trimmed) from the same apply:

{
  "format_version": "1.0",
  "terraform_version": "1.16.4",
  "values": {
    "root_module": {
      "resources": [
        {
          "address": "local_file.config",
          "mode": "managed",
          "type": "local_file",
          "name": "config",
          "provider_name": "registry.terraform.io/hashicorp/local",
          "values": {
            "content": "listen=8080\ntrigger_id=v1\n",
            "id": "7551ae2164513c74b5c0665a6b28b6a40399686f"
          },
          "sensitive_values": { "sensitive_content": true },
          "depends_on": ["terraform_data.trigger"]
        }
      ]
    }
  }
}

drift-detection-and-import.mdx and sensitive-data-and-team-workflows.mdx both come back to show -json, because -detailed-exitcode proofs and the sensitive_values map both live in this exact structure.

Inspecting state

Section titled “Inspecting state”

Terraform ships commands for looking inside state without opening the JSON file by hand:

Terminal window
# Human-readable dump of the entire current state
terraform show


# List every resource address currently tracked in state
terraform state list


# Show full attribute detail for one specific resource
terraform state show local_file.config


# Print the current state as JSON, from whatever backend is configured
terraform state pull > current.tfstate

terraform state list is the fast way to answer “what does this configuration actually manage right now” — it prints addresses with no other detail, real output:

$ terraform state list
local_file.config
terraform_data.trigger

terraform state show <address> drills into one address, real output:

$ terraform state show local_file.config
# local_file.config:
resource "local_file" "config" {
    content              = <<-EOT
        listen=8080
        trigger_id=v1
    EOT
    content_sha1         = "7551ae2164513c74b5c0665a6b28b6a40399686f"
    filename             = "./generated/app.conf"
    id                   = "7551ae2164513c74b5c0665a6b28b6a40399686f"
}

terraform state pull is what a remote backend actually serves state through — even with a local backend it always returns the current state as JSON on stdout, which is why scripting against state (a CI check, a custom drift report) pulls rather than reads the file path directly: the same command works unmodified no matter which backend is configured.

Why hand-editing state breaks it: lineage and serial are enforced, not decorative

Section titled “Why hand-editing state breaks it: lineage and serial are enforced, not decorative”

terraform state push <file> uploads a local state file to replace whatever the backend currently holds — and it enforces lineage and serial before it will do that, precisely to stop you from clobbering a newer state with a stale or unrelated one. Proof: take the real state above, flip its lineage to a random UUID, and push it back:

$ terraform state push tampered.tfstate
Failed to write state: cannot import state with lineage "00000000-0000-0000-0000-000000000000" over unrelated state with lineage "91789eb2-a206-8e38-2571-62323b581164"

Same state, serial set backwards instead:

$ terraform state push tampered2.tfstate
Failed to write state: cannot import state with serial 1 over newer state with serial 6

Both are real refusals from Terraform 1.16.4, not simulated errors. This is exactly why hand-editing terraform.tfstate in a text editor is dangerous even when you only mean to fix one field: unless you also correctly bump serial (and never touch lineage), Terraform’s own consistency check rejects the write the moment you try to push it back — and if you bypass that check by overwriting the backend object directly (skipping state push entirely, e.g. hand-copying a file into an S3 object), nothing stops you, which is the real failure mode: silent corruption with no error at all, only discovered on the next plan when Terraform’s idea of reality no longer matches what is actually deployed. terraform state mv, import, and rm all exist specifically so you never need to open the JSON by hand — they update serial correctly as part of the same operation.

What the cloud backend actually stores

Section titled “What the cloud backend actually stores”

This course uses a local backend for every proof above (no credentials needed), but production AWS state lives in an S3 backend. Per the S3 backend’s own documentation:

“The S3 backend stores state data in an S3 object at the path set by the key parameter in the S3 bucket indicated by the bucket parameter.” — developer.hashicorp.com/terraform/language/backend/s3

So the state JSON shown above is exactly what lands in that S3 object — same version/serial/lineage/resources shape, nothing AWS-specific added. Locking is a separate, additional object: S3 native locking (use_lockfile = true, added in Terraform 1.10.0 — “The s3 backend now supports S3 native state locking,” per the Terraform 1.10.0 CHANGELOG) writes a <key>.tflock object alongside the state object itself. remote-backends-and-locking.mdx covers the full backend block, migration, and force-unlock.

flowchart LR
  cfg["Terraform config (.tf files)"] --> apply["terraform apply"]
  apply --> aws["Real AWS resources"]
  aws -->|attributes written back| state["terraform.tfstate"]
  cfg -->|next run| plan["terraform plan"]
  state -->|read for comparison| plan
Config and real AWS resources both feed the state file that the next plan reads

Mental Model

Section titled “Mental Model”

State is Terraform’s only memory of reality — not a cache you can safely discard and not a log you can safely append to by hand. Everything in this lesson follows from one fact: the provider API has no idea Terraform exists, so every bit of “what did I create, and what did it look like” has to live somewhere Terraform fully controls. lineage and serial are not metadata for humans to read — they are the mechanism that makes state push a safe, checked operation instead of a blind overwrite. Treat the state file the way you’d treat a database’s transaction log: you read it with the tools built for reading it (state show, show -json, state pull), and you mutate it with the commands built for mutating it (state mv, import, rm) — never a text editor.

Trade-offs

Section titled “Trade-offs”
ChoiceBenefitCost
Caching every attribute in stateplan compares against cached data instead of re-describing every resource from the provider API — real performance win at scaleEvery attribute, including secrets, sits in plaintext on disk or in the backend object (proven above)
lineage/serial consistency checks on state pushA stale or unrelated state file cannot silently clobber current state (proven above, twice)Only protects the state push path — a direct backend-object overwrite bypasses the check entirely, with no error
terraform show -json as a stable machine formatTooling (policy checks, CI, custom scripts) can rely on a documented, versioned schema instead of parsing human-readable outputA second representation of the same data to keep in mind; raw terraform.tfstate internals and show -json are not the same shape

Common Misconceptions

Section titled “Common Misconceptions”
  • “State only stores what I put in output blocks.” Proven above: local_file.config’s full attribute set — including derived values it never explicitly output — is written into resources[].instances[].attributes regardless of any output block.
  • “I can safely hand-edit terraform.tfstate as long as I’m careful.” Proven above: state push rejects a lineage change and a serial regression with a real error. The danger is exactly the case that error doesn’t cover — overwriting the backend object directly, which produces no error and no warning, only a plan that no longer matches reality.
  • “serial is the resource count, or a version number I control.” It is an internal write counter Terraform increments itself. You never set it, and setting it to anything other than “higher than the current value” is what state push refuses in the proof above.
  • “terraform show -json and the raw .tfstate file are the same format.” Proven above: show -json’s values.root_module.resources[].values map and the raw state’s resources[].instances[].attributes map carry the same underlying data but are documented as separate, independently versioned schemas (format_version vs. state version).

Reading AI-generated code

Section titled “Reading AI-generated code”

When reviewing generated Terraform that touches state directly, check for:

  • A script that opens terraform.tfstate with sed/jq -i/a text editor to “fix” one value in place, then relies on the next apply to pick it up — proven above, this either gets rejected by state push’s lineage/serial check, or worse, silently corrupts the backend object if the check is bypassed.
  • .gitignore missing *.tfstate* entirely, or only listing terraform.tfstate and missing terraform.tfstate.backup and the *.tfstate.* (workspace) variants.
  • Code or a runbook that assumes serial is something safe to set to a specific number (“set serial to 1 to reset”) — it is a monotonic write counter enforced by the backend, not a field to hand-pick.
  • A custom drift-checking script that reads terraform.tfstate directly with a hardcoded local path instead of terraform state pull, which breaks the moment the project switches to a remote backend.
  • Confusing terraform show -json’s schema with the raw state file’s schema when writing a parser — they document different fields (values.root_module.resources vs resources[].instances[].attributes) even though they describe the same resources.

🐛 Spot the bug

#!/usr/bin/env bash
# "quick fix" for a wrong instance_type recorded in state
sed -i '' 's/"t3.micro"/"t3.small"/' terraform.tfstate
terraform state push terraform.tfstate
Show the answer

Two real problems, proven above. First, state push enforces lineage/serial — but sed -i here doesn’t change either field, so the push might actually succeed, which is the trap: Terraform now believes instance_type is t3.small while AWS still has a t3.micro instance running. Nothing calls the AWS API to verify. The next plan will show no change (state now falsely agrees with the edited value), so drift between the edited state and real infrastructure goes completely undetected until something else surfaces it. The correct fix for “state disagrees with reality” is never a hand edit — it’s terraform apply (if the config should win), terraform plan -refresh-only (to see what changed), or terraform import/state rm (covered in drift-detection-and-import.mdx), all of which go through Terraform’s own consistency machinery instead of bypassing it.

Why does Terraform need a state file instead of just querying the cloud provider API each time
A database password was set as a resource attribute but was never printed with terraform output. Can it still end up readable in plain text
A hand-edited state file like this is pushed with terraform state push tampered.tfstate against a backend currently at serial 6, lineage 91789eb2-.... What happens?
{
  "serial": 1,
  "lineage": "00000000-0000-0000-0000-000000000000"
}
What does the dependencies field on a state resource instance actually record, proven in the local_file.config / terraform_data.trigger example above?
What is the difference between terraform state list and terraform state show
Why does a custom drift-checking script that parses terraform.tfstate directly (by local file path) break when a project switches to a remote backend?