Skip to content
Terraform & Terragrunt for GCP — From Zero to Hero

The State File

The idea in one sentence

Section titled “The idea in one sentence”

Terraform state is a JSON file that maps every resource in your configuration to the real-world object it created, and it is the only thing that lets Terraform know what it manages.

Why Terraform needs state at all

Section titled “Why Terraform needs state at all”

When you run terraform apply, Terraform creates resources through Google Cloud’s API — a Compute Engine instance, a Cloud Storage bucket, an IAM binding. The GCP API has no concept of “Terraform.” If a project has 500 Compute Engine instances, the Compute Engine API cannot tell you which of those instances Terraform created for a given resource "google_compute_instance" "web" block, which ones a teammate created by hand in the Console, and which ones belong to a completely different Terraform configuration.

Terraform solves this by keeping its own record. Every time you apply, Terraform writes the mapping between your configuration’s resource addresses and the real infrastructure IDs into a state file, by default named terraform.tfstate:

resource "google_compute_instance" "web" {
  name         = "web-vm"
  machine_type = "e2-medium"
  zone         = "us-central1-a"


  boot_disk {
    initialize_params {
      image = "debian-cloud/debian-12"
    }
  }


  network_interface {
    network = "default"
  }
}

After apply, the state file records something like google_compute_instance.web -> projects/my-project-id/zones/us-central1-a/instances/web-vm, plus every attribute the Compute Engine API returned for that instance — its internal IP, its self link, its network interfaces, and so on. That cached data means Terraform does not have to call the GCP API and re-describe every resource just to compute a plan; it can compare your configuration against the last known state first, which is both how it detects what changed and a real performance win on large configurations. State is also where Terraform stores dependency metadata, so it knows the order in which resources must be created, updated, or destroyed.

The plaintext risk hiding in your state file

Section titled “The plaintext risk hiding in your state file”

State files are not sanitized. Any attribute value Terraform receives from a provider — or that you set directly in your configuration — is written into terraform.tfstate as plaintext, including values that are secrets. A common example:

resource "google_sql_user" "app" {
  name     = "app_admin"
  instance = google_sql_database_instance.main.name
  password = "S3cretPassword!"
}

You never ran terraform output on that password and you never printed it in a log line. It still lands in terraform.tfstate in plaintext, because Terraform has to remember every attribute it set in order to detect drift on the next plan. Anyone who can read that file can read the password.

This is why terraform.tfstate must never be committed to version control. Add it to .gitignore from the very first commit of a new project:

.gitignore
terraform.tfstate
terraform.tfstate.backup
*.tfstate
*.tfstate.*

The next lesson covers moving state off your laptop entirely and into a properly access-controlled remote backend, which is the real fix for this risk — a .gitignore entry only stops the most obvious accident.

Inspecting state

Section titled “Inspecting state”

Terraform ships three commands for looking inside state without opening the JSON file by hand:

Terminal window
# Human-readable dump of the entire current state
terraform show


# List every resource address currently tracked in state
terraform state list


# Show full attribute detail for one specific resource
terraform state show google_compute_instance.web

terraform state list is the fast way to answer “what does this configuration actually manage right now” — it prints addresses like google_compute_instance.web or module.network.google_compute_network.main with no other detail. terraform state show <address> drills into one of those addresses and prints every attribute Terraform has recorded for it, which is often faster than going to the Google Cloud Console when you need to check a specific value Terraform is tracking.

Anatomy of a real state file

Section titled “Anatomy of a real state file”

The examples below are not reconstructed from memory — they come from a real terraform apply against two resources that need no cloud credentials at all: terraform_data (the built-in provider’s replacement for the deprecated null_resource) and local_file (hashicorp/local). This is the same underlying JSON structure GCP state uses — the format is provider-independent; only the provider string and the resource type/attributes would differ for a real google_* resource. The config:

main.tf
terraform {
  required_providers {
    local = {
      source  = "hashicorp/local"
      version = "~> 2.5"
    }
  }
}


resource "terraform_data" "trigger" {
  input = "v1"
}


resource "local_file" "config" {
  filename = "${path.module}/generated/app.conf"
  content  = "listen=8080\ntrigger_id=${terraform_data.trigger.output}\n"
}

After terraform init -backend=false && terraform apply -auto-approve, the real terraform.tfstate on disk (machine paths stripped):

{
  "version": 4,
  "terraform_version": "1.16.4",
  "serial": 6,
  "lineage": "91789eb2-a206-8e38-2571-62323b581164",
  "outputs": {},
  "resources": [
    {
      "mode": "managed",
      "type": "local_file",
      "name": "config",
      "provider": "provider[\"registry.terraform.io/hashicorp/local\"]",
      "instances": [
        {
          "schema_version": 0,
          "attributes": {
            "content": "listen=8080\ntrigger_id=v1\n",
            "content_sha1": "7551ae2164513c74b5c0665a6b28b6a40399686f",
            "filename": "./generated/app.conf",
            "id": "7551ae2164513c74b5c0665a6b28b6a40399686f"
          },
          "dependencies": ["terraform_data.trigger"],
          "sensitive_attributes": [[{ "type": "get_attr", "value": "sensitive_content" }]]
        }
      ]
    },
    {
      "mode": "managed",
      "type": "terraform_data",
      "name": "trigger",
      "provider": "provider[\"terraform.io/builtin/terraform\"]",
      "instances": [
        {
          "schema_version": 0,
          "attributes": {
            "id": "7e48645a-6b7e-14c9-d8b3-99d042610145",
            "input": { "value": "v1", "type": "string" },
            "output": { "value": "v1", "type": "string" }
          },
          "sensitive_attributes": []
        }
      ]
    }
  ],
  "check_results": null
}

Every top-level field means something specific:

  • version — the state file format version (4 since Terraform 0.12; unrelated to the Terraform binary version).
  • terraform_version — the actual CLI version that last wrote this file.
  • serial — a counter incremented on every write. Two state files claiming the same lineage are compared by serial to decide which is newer.
  • lineage — a UUID generated once, the first time state is created for a working directory. It never changes for the life of that state, and it is how Terraform tells “an older copy of the same state” apart from “an unrelated state file that happens to look similar.”
  • resources[] — one entry per resource block (not per instance — count/for_each share one entry with multiple instances[]), each with mode, type, name, provider, and instances. For a google_compute_instance, type would be "google_compute_instance" and provider would read provider["registry.terraform.io/hashicorp/google"].
  • instances[].attributes — every attribute the provider returned after create/read, the same data terraform state show prints.
  • instances[].dependencies — the resource addresses this instance’s config referenced (proven above: local_file.config depends on terraform_data.trigger because its content interpolates terraform_data.trigger.output). This is exactly the ordering metadata mentioned above — it is why a destroy tears things down in the right order without recomputing dependencies from scratch.

terraform show -json: the same data, provider-agnostic and stable

Section titled “terraform show -json: the same data, provider-agnostic and stable”

terraform state show above is a human-readable format meant to be read by a person. terraform show -json is the machine-readable contract other tools (Sentinel, OPA/conftest, custom CI checks) are meant to parse — it is documented, versioned (format_version), and does not change shape between provider versions the way raw state internals sometimes do. Real output (trimmed) from the same apply:

{
  "format_version": "1.0",
  "terraform_version": "1.16.4",
  "values": {
    "root_module": {
      "resources": [
        {
          "address": "local_file.config",
          "mode": "managed",
          "type": "local_file",
          "name": "config",
          "provider_name": "registry.terraform.io/hashicorp/local",
          "values": {
            "content": "listen=8080\ntrigger_id=v1\n",
            "id": "7551ae2164513c74b5c0665a6b28b6a40399686f"
          },
          "sensitive_values": { "sensitive_content": true },
          "depends_on": ["terraform_data.trigger"]
        }
      ]
    }
  }
}

drift-detection-and-import.mdx and sensitive-data-and-team-workflows.mdx both come back to show -json, because -detailed-exitcode proofs and the sensitive_values map both live in this exact structure.

Why hand-editing state breaks it: lineage and serial are enforced, not decorative

Section titled “Why hand-editing state breaks it: lineage and serial are enforced, not decorative”

terraform state push <file> uploads a local state file to replace whatever the backend currently holds — and it enforces lineage and serial before it will do that, precisely to stop you from clobbering a newer state with a stale or unrelated one. Proof: take the real state above, flip its lineage to a random UUID, and push it back:

$ terraform state push tampered.tfstate
Failed to write state: cannot import state with lineage "00000000-0000-0000-0000-000000000000" over unrelated state with lineage "91789eb2-a206-8e38-2571-62323b581164"

Same state, serial set backwards instead:

$ terraform state push tampered2.tfstate
Failed to write state: cannot import state with serial 1 over newer state with serial 6

Both are real refusals from Terraform 1.16.4, not simulated errors. This is exactly why hand-editing terraform.tfstate in a text editor is dangerous even when you only mean to fix one field: unless you also correctly bump serial (and never touch lineage), Terraform’s own consistency check rejects the write the moment you try to push it back — and if you bypass that check by overwriting the backend object directly (skipping state push entirely, e.g. hand-copying a file into a GCS object), nothing stops you, which is the real failure mode: silent corruption with no error at all, only discovered on the next plan when Terraform’s idea of reality no longer matches the GCP resources actually deployed. terraform state mv, import, and rm all exist specifically so you never need to open the JSON by hand — they update serial correctly as part of the same operation.

What the GCS backend actually stores

Section titled “What the GCS backend actually stores”

This course uses a local backend for every proof above (no credentials needed), but production GCP state lives in a gcs backend. Per the GCS backend’s own documentation:

“Stores the state as an object in a configurable prefix in a pre-existing bucket on Google Cloud Storage (GCS).” — developer.hashicorp.com/terraform/language/backend/gcs

So the state JSON shown above is exactly what lands in that GCS object — same version/serial/lineage/resources shape, nothing GCP-specific added. The prefix argument controls where under the bucket that object (and any named workspace’s object) lives — per the docs, “named states for workspaces are stored in an object called <prefix>/<name>.tfstate.” The gcs backend documentation confirms locking is supported but, unlike the S3 backend page (which spells out the <key>.tflock object) or the azurerm page (which names the blob-lease mechanism), it does not document the exact locking mechanism in the page text itself — worth stating plainly rather than guessing at an implementation detail no fetched HashiCorp page confirms. What the docs do confirm: no separate lock resource (table, extra bucket) needs to be provisioned — locking is a property of the backend against the same bucket. remote-backends-and-locking.mdx covers the full backend block, migration, and force-unlock.

flowchart LR
  cfg["Terraform config (.tf files)"] --> apply["terraform apply"]
  apply --> gcp["Real GCP resources"]
  gcp -->|attributes written back| state["terraform.tfstate"]
  cfg -->|next run| plan["terraform plan"]
  state -->|read for comparison| plan
Config and real GCP resources both feed the state file that the next plan reads

Mental Model

Section titled “Mental Model”

State is Terraform’s only memory of reality — not a cache you can safely discard and not a log you can safely append to by hand. Everything in this lesson follows from one fact: the GCP API has no idea Terraform exists, so every bit of “what did I create, and what did it look like” has to live somewhere Terraform fully controls. lineage and serial are not metadata for humans to read — they are the mechanism that makes state push a safe, checked operation instead of a blind overwrite. Treat the state file the way you’d treat a database’s transaction log: you read it with the tools built for reading it (state show, show -json, state pull), and you mutate it with the commands built for mutating it (state mv, import, rm) — never a text editor.

Trade-offs

Section titled “Trade-offs”
ChoiceBenefitCost
Caching every attribute in stateplan compares against cached data instead of re-describing every resource from the GCP API — real performance win at scaleEvery attribute, including secrets, sits in plaintext on disk or in the GCS object (proven above)
lineage/serial consistency checks on state pushA stale or unrelated state file cannot silently clobber current state (proven above, twice)Only protects the state push path — a direct GCS-object overwrite bypasses the check entirely, with no error
terraform show -json as a stable machine formatTooling (policy checks, CI, custom scripts) can rely on a documented, versioned schema instead of parsing human-readable outputA second representation of the same data to keep in mind; raw terraform.tfstate internals and show -json are not the same shape

Common Misconceptions

Section titled “Common Misconceptions”
  • “State only stores what I put in output blocks.” Proven above: local_file.config’s full attribute set — including derived values it never explicitly output — is written into resources[].instances[].attributes regardless of any output block.
  • “I can safely hand-edit terraform.tfstate as long as I’m careful.” Proven above: state push rejects a lineage change and a serial regression with a real error. The danger is exactly the case that error doesn’t cover — overwriting the GCS object directly, which produces no error and no warning, only a plan that no longer matches reality.
  • “serial is the resource count, or a version number I control.” It is an internal write counter Terraform increments itself. You never set it, and setting it to anything other than “higher than the current value” is what state push refuses in the proof above.
  • “The GCS backend’s prefix argument is used to auto-create the bucket, like project/location are.” remote-backends-and-locking.mdx proves with real terragrunt render output that project and location are consumed to auto-create the bucket while prefix only ever affects the object path inside it — the docs quote above (“named states for workspaces are stored in an object called <prefix>/<name>.tfstate”) confirms prefix is purely a path, not a bucket-creation input.

Reading AI-generated code

Section titled “Reading AI-generated code”

When reviewing generated Terraform that touches state directly, check for:

  • A script that opens terraform.tfstate with sed/jq -i/a text editor to “fix” one value in place, then relies on the next apply to pick it up — proven above, this either gets rejected by state push’s lineage/serial check, or worse, silently corrupts the backend object if the check is bypassed.
  • .gitignore missing *.tfstate* entirely, or only listing terraform.tfstate and missing terraform.tfstate.backup and the *.tfstate.* (workspace) variants.
  • Code or a runbook that assumes serial is something safe to set to a specific number (“set serial to 1 to reset”) — it is a monotonic write counter enforced by the backend, not a field to hand-pick.
  • A custom drift-checking script that reads terraform.tfstate directly with a hardcoded local path instead of terraform state pull, which breaks the moment the project switches to a remote gcs backend.
  • A claim, stated as fact, about exactly how GCS implements the backend’s locking internally — proven above, the fetched HashiCorp docs confirm locking is supported without documenting the mechanism; state that gap rather than inventing a plausible-sounding internal.

🐛 Spot the bug

#!/usr/bin/env bash
# "quick fix" for a wrong machine_type recorded in state
sed -i '' 's/"e2-medium"/"e2-standard-4"/' terraform.tfstate
terraform state push terraform.tfstate
Show the answer

Two real problems, proven above. First, state push enforces lineage/serial — but sed -i here doesn’t change either field, so the push might actually succeed, which is the trap: Terraform now believes the instance’s machine_type is e2-standard-4 while GCP still has an e2-medium instance running. Nothing calls the Compute Engine API to verify. The next plan will show no change (state now falsely agrees with the edited value), so drift between the edited state and the real instance goes completely undetected until something else surfaces it. The correct fix for “state disagrees with reality” is never a hand edit — it’s terraform apply (if the config should win), terraform plan -refresh-only (to see what changed), or terraform import/state rm (covered in drift-detection-and-import.mdx), all of which go through Terraform’s own consistency machinery instead of bypassing it.

Why does Terraform need a state file instead of just querying the cloud provider API each time
A database password was set as a resource attribute but was never printed with terraform output. Can it still end up readable in plain text
A hand-edited state file like this is pushed with terraform state push tampered.tfstate against a backend currently at serial 6, lineage 91789eb2-.... What happens?
{
  "serial": 1,
  "lineage": "00000000-0000-0000-0000-000000000000"
}
What does the dependencies field on a state resource instance actually record, proven in the local_file.config / terraform_data.trigger example above?
What is the difference between terraform state list and terraform state show
According to the gcs backend documentation quoted above, what does the prefix argument actually control?