Remote Backends and Locking

The idea in one sentence

A remote backend stores state in a shared location like S3 instead of on one person’s laptop, and locking stops two people from writing to that shared state at the same time.

Why local state does not work for a team

By default, terraform.tfstate lives on the machine where you ran apply. That is fine for a solo experiment, but it breaks down the moment a second person joins:

Teammate A runs apply and creates a VPC. Their laptop’s local state file now knows about it.
Teammate B, on a different laptop, has never seen that state file. They run plan against their own stale or empty local state and either try to recreate the VPC or get a plan that makes no sense next to what actually exists.
CI/CD pipelines have the same problem in a sharper form — a pipeline runner is a fresh, disposable environment with no local state file at all. It needs a canonical copy of state to read from and write to on every run, not a copy that happens to be sitting on someone’s laptop.

The fix is to move state off any individual machine and into a shared backend that every team member and every CI run points at.

Configuring the S3 backend with native locking

The current recommended way to store Terraform state on AWS is an S3 bucket configured as the backend, with native S3 state locking turned on:

terraform {
  backend "s3" {
    bucket       = "my-company-terraform-state"
    key          = "networking/terraform.tfstate"
    region       = "us-east-1"
    use_lockfile = true
  }
}

bucket and key together identify exactly which object in S3 holds this configuration’s state — key is the path within the bucket, so different configurations (or different Terragrunt units) typically use different keys in the same bucket. use_lockfile = true is the important line: Terraform uses a conditional-write lock file stored alongside the state object directly in S3 to coordinate locking. No separate locking infrastructure is required — the S3 bucket alone is enough.

You will still encounter the older pattern in existing codebases, using a separate DynamoDB table for locking:

# Legacy locking pattern — dynamodb_table is deprecated and scheduled for removal.
# Prefer use_lockfile = true (shown above) in new configurations.
terraform {
  backend "s3" {
    bucket         = "my-company-terraform-state"
    key            = "networking/terraform.tfstate"
    region         = "us-east-1"
    dynamodb_table = "terraform-state-lock"
  }
}

That pattern works, and you should not panic if you inherit a codebase that uses it, but dynamodb_table is now a deprecated argument on the S3 backend and is planned for removal in a future Terraform version. New configurations should use use_lockfile = true instead — it does the same job without a second AWS resource to create, pay for, and keep in sync with the state bucket.

Migrating a backend: `init -migrate-state` vs `init -reconfigure`

Changing the backend block — moving to a different bucket, changing key, or moving from local to S3 for the first time — requires re-running terraform init, and Terraform gives you two different flags depending on whether you want the old state carried over. No cloud credentials exist in this course, so the proof below changes a local backend’s path rather than migrating to S3 — the exact same prompt and mechanics apply to a local→S3 move, only the backend type differs.

terraform init -migrate-state asks whether to copy existing state into the new backend location. Real output, changing path = "state/old.tfstate" to path = "state/new.tfstate" in a backend "local" block:

$ terraform init -migrate-state
Initializing the backend...
Backend configuration changed!

Terraform has detected that the configuration specified for the backend
has changed. Terraform will now check for existing state in the backends.
Do you want to copy existing state to the new backend?
  Pre-existing state was found while migrating the previous "local" backend to the
  newly configured "local" backend. No existing state was found in the newly
  configured "local" backend. Do you want to copy this state to the new "local"
  backend? Enter "yes" to copy and "no" to start with an empty state.

  Enter a value: yes

Successfully configured the backend "local"! Terraform will automatically
use this backend unless the backend configuration changes.

Answering yes really copies the resource into the new location — state/new.tfstate contains the same local_file resource the old path had. terraform init -reconfigure skips that prompt entirely and does not carry state over — proof, changing the path again and running -reconfigure instead:

$ terraform init -reconfigure
Initializing the backend...

Successfully configured the backend "local"! Terraform will automatically
use this backend unless the backend configuration changes.

No migration question at all. The next plan proves the difference — Terraform now believes the resource does not exist, because the new backend location was never populated:

$ terraform plan
  # local_file.config will be created
  + resource "local_file" "config" {
      + content  = "listen=8080\n"
      + filename = "./generated/app.conf"
    }

Plan: 1 to add, 0 to change, 0 to destroy.

-reconfigure is for “start clean at this new location, I do not want the old state” (or the old backend is gone/unreachable); -migrate-state is for “this is the same infrastructure, just move where its state record lives.” Picking the wrong one against a real S3 bucket either leaves the old state behind unmigrated, or (worse) makes Terraform think a running VPC needs to be recreated from scratch.

Partial backend configuration with `-backend-config`

A backend block does not have to hardcode every argument — leaving it empty and supplying values at init time (from a file or individual -backend-config="key=value" flags) keeps secrets and per-environment values out of committed .tf files:

terraform {
  backend "s3" {}
}

# backend.hcl (not committed, or generated by CI per environment)
bucket = "my-company-terraform-state"
key    = "networking/terraform.tfstate"
region = "us-east-1"

terraform init -backend-config=backend.hcl

Real output against a local backend proving the flag is accepted and merged (path supplied the same way as bucket/key/region would be for S3):

$ terraform init -backend-config=backend.hcl
Initializing the backend...

Successfully configured the backend "local"! Terraform will automatically
use this backend unless the backend configuration changes.

This is exactly the mechanism CI pipelines use to point the same configuration at a different S3 key per environment (dev.hcl, prod.hcl) without duplicating the .tf files.

`terraform force-unlock`: the manual escape hatch

If a apply is killed mid-run (a laptop sleeps, a CI job is cancelled), the lock it acquired can be left behind with nothing left running to release it. terraform force-unlock <LOCK_ID> removes that lock manually. Real -help output from Terraform 1.16.4:

$ terraform force-unlock -help
Usage: terraform [global options] force-unlock LOCK_ID

  Manually unlock the state for the defined configuration.

  This will not modify your infrastructure. This command removes the lock on the
  state for the current workspace. The behavior of this lock is dependent
  on the backend being used. Local state files cannot be unlocked by another
  process.

Options:

  -force                 Don't ask for input for unlock confirmation.

The LOCK_ID is not something you invent — when a lock is already held, the next run that fails to acquire it prints the lock’s ID (and who holds it, and when it was acquired) in its own error message, and that ID is what you paste into force-unlock. A stale lock genuinely cannot be produced offline in this course — the local backend the harness uses has no lock to leave behind the way S3’s .tflock object or a held blob lease can (the -help text above says exactly that: “Local state files cannot be unlocked by another process”). Treat force-unlock as a last resort for a lock you have independently confirmed is stale (the process that held it is actually gone — check first, e.g. that the CI run really did get cancelled), never a reflex for “my apply is being slow.”

Terragrunt: `remote_state` generates the `backend.tf` you never hand-write

Terragrunt’s remote_state block in root.hcl generates the exact backend block above for every unit that includes it, resolving key per-unit with path_relative_to_include() so each unit gets its own state path automatically:

remote_state {
  backend = "s3"
  generate = {
    path      = "backend.tf"
    if_exists = "overwrite"
  }
  config = {
    bucket       = "my-company-terraform-state"
    key          = "${path_relative_to_include()}/terraform.tfstate"
    region       = "us-east-1"
    encrypt      = true
    use_lockfile = true
  }
}

include "root" {
  path = find_in_parent_folders("root.hcl")
}

terraform {
  source = "."
}

Running terragrunt run -- init -backend=false in the vpc unit generates a real backend.tf — every key from config lands verbatim, and key resolves to vpc/terraform.tfstate:

# Generated by Terragrunt. Sig: nIlQXj57tbuaRZEa
terraform {
  backend "s3" {
    bucket       = "my-company-terraform-state"
    encrypt      = true
    key          = "vpc/terraform.tfstate"
    region       = "us-east-1"
    use_lockfile = true
  }
}

Every argument in the config map is emitted as-is — there is no auto-create step on AWS the way GCP’s project/location trigger bucket creation (module-versioning-and-registries.mdx’s GCP counterpart proves that contrast directly). This is the mechanism behind never hand-writing a backend "s3" {} block per environment: one remote_state block in the root config, and every unit underneath gets a correct, per-unit-scoped backend generated for it.

What locking actually prevents

Imagine locking did not exist. Two people, or a person and a CI run, both execute terraform apply against the same S3-backed state at nearly the same moment. Both read the current state, both compute a plan, and both start writing their results back to the same state object. Depending on timing, the second write can overwrite the first — silently dropping the first run’s recorded changes even though the first run’s infrastructure changes actually happened — or the state file can end up corrupted mid-write.

A lock turns that race into a queue. The first apply to start acquires the lock; the second apply either waits for the lock to be released, or fails immediately with a clear “state is locked” error, depending on the flags used. Either way, the second run does not get to write over the first run’s state while it is mid-flight.

# A run that finds the state already locked fails fast with a clear message,
# rather than corrupting the state file
terraform apply
# Error: Error acquiring the state lock

flowchart LR
  eng1["Engineer A: terraform apply"] -->|acquires lock| s3["S3 state + lock file"]
  eng2["CI run: terraform apply"] -->|blocked, waits or fails| s3
  s3 -->|lock released after apply| eng2

Two applies race for the same S3-backed state; the lock lets one through and makes the other wait

Mental Model

Every mechanism in this lesson exists to answer one question honestly: “whose write wins, and does the loser find out?” A shared backend answers “where is the one true copy” so nobody’s laptop is that copy. Locking answers “what happens when two writes race” by turning the race into a queue instead of a coin flip. -migrate-state vs -reconfigure answers “does the old record follow me” when the backend’s location changes. force-unlock exists only because the queue mechanism itself can jam (a lock holder that will never release) — it is a manual override of the queue, not a replacement for it, which is exactly why it demands a real lock ID rather than a “just unlock everything” flag.

Trade-offs

Choice	Benefit	Cost
`use_lockfile = true` over `dynamodb_table`	One fewer AWS resource to create, pay for, and keep in sync with the state bucket (proven above: locking lives in S3 itself)	Deprecated `dynamodb_table` configs must be migrated deliberately, not left to rot indefinitely
`-migrate-state` over `-reconfigure` when the backend location changes	Old state is carried forward — proven above, the resource is still tracked afterward	Requires answering an interactive prompt (or `-input=false` plus care) — easy to fat-finger “no” and lose the link to existing infrastructure
Partial backend config (`-backend-config`)	Keeps environment-specific values and secrets out of committed `.tf` files	The backend block alone, read in isolation, no longer tells you which bucket/key state actually lives in — you must also find the `-backend-config` file or CI step
Terragrunt `remote_state` generating `backend.tf`	One block in `root.hcl` correctly scopes every unit’s state path automatically (proven above with real generated output)	The generated file is not meant to be hand-edited — a manual edit is overwritten the next `run` (`if_exists = "overwrite"`)

Common Misconceptions

“terraform init -reconfigure is just a safer alias for -migrate-state.” Proven above: -reconfigure never prompts and never copies state — the very next plan shows the tracked resource as “to create” again, because the new backend location was never populated.
“A stale lock is common and force-unlock is a routine command.” Proven above via the real -help text: it is a manual escape hatch for a lock whose holder is confirmed gone, not a default response to a slow or in-progress apply.
“dynamodb_table still needs to be configured for S3 locking to work in 2026.” Proven above: use_lockfile = true, added for S3 native locking in Terraform 1.10.0 per the CHANGELOG, needs no DynamoDB table at all — dynamodb_table is the deprecated path now.
“Terragrunt’s generated backend.tf is just a copy-paste convenience — you could write the same file by hand just as correctly.” Proven above: the generated file resolves path_relative_to_include() per unit automatically; hand-writing it means re-deriving and re-typing a correct, unique key for every single unit yourself, with no check that two units never collide on the same key.

Reading AI-generated code

When reviewing generated backend configuration or migration scripts, check for:

dynamodb_table added to a new S3 backend block “because that’s how S3 locking works” — proven above, that argument is deprecated; use_lockfile = true is the current mechanism and needs no second AWS resource.
terraform init -reconfigure used in a runbook step that is supposed to preserve existing state — proven above, -reconfigure does not migrate state at all; the correct flag for that intent is -migrate-state.
A CI script that pipes yes into every terraform init unconditionally to avoid interactive prompts, including the -migrate-state “copy existing state?” prompt — that answer needs to be a deliberate yes (this environment really is a location change) or the script risks silently copying state into the wrong place.
force-unlock called with a made-up or copy-pasted-from-a-tutorial lock ID rather than the one printed in the actual “state is locked” error for the run in question.
A hardcoded bucket/key/region in a shared module meant to be reused across environments, instead of an empty backend "s3" {} filled by -backend-config per environment (or, in Terragrunt, one remote_state block generating it correctly per unit).

🐛 Spot the bug

# CI "cleanup" step run before every apply, to avoid ever seeing a locked-state error
terraform force-unlock -force $(cat last-known-lock-id.txt)
terraform apply -auto-approve

Show the answer

This runs force-unlock unconditionally before every apply, using a lock ID cached from some previous run rather than the ID of any lock that actually exists right now. Proven above: force-unlock “will not modify your infrastructure” by itself, but blindly releasing a lock that is currently, legitimately held — by a teammate’s in-progress apply, or a concurrent CI run — removes the one protection state locking provides. The next line’s apply can now race directly against whatever was holding that lock, which is precisely the corrupted-write scenario locking exists to prevent. The correct use of force-unlock is reactive and manual: read the actual “state is locked” error for the ID it names, confirm independently that the holder is gone, and only then unlock that specific ID — never a pre-emptive step in a script that runs regardless of whether anything is actually locked.

🏗️ The one-line takeaway

A remote backend like S3 gives every teammate and every CI run the same canonical copy of state, instead of each machine holding its own diverging local file. Locking — configured today with use_lockfile = true directly on the S3 backend (added in Terraform 1.10.0), no DynamoDB table required — prevents two concurrent apply runs from writing to that shared state at the same time. terraform init -migrate-state carries old state to a new backend location (proven above, a real interactive copy); -reconfigure does not (proven above, the next plan recreates everything). force-unlock is a manual escape hatch for a confirmed-stale lock, never a routine step — proven above with the real -help text and why a stale lock cannot even be simulated offline. Terragrunt’s remote_state block generates a correct, per-unit backend.tf automatically (proven above with real generated output) instead of every unit hand-writing its own.

Why does a local terraform.tfstate file break down once a second team member starts running Terraform Local state files expire after 24 hours Each machine only knows about the changes applied from that same machine, so a second laptop plans against stale or missing state Terraform refuses to run with more than one contributor Local state files cannot store more than one resource

What does use_lockfile = true provide on an S3 backend, and why is it now preferred over a DynamoDB table It enables versioning on the S3 bucket automatically It provides native state locking using a conditional-write lock file in S3 itself, removing the need for a separate DynamoDB table that the older dynamodb_table argument required It encrypts the state file at rest, which DynamoDB locking cannot do It has no effect; DynamoDB locking is still required for correctness

No migration prompt appeared at all. What does the very next terraform plan show, proven above?

$ terraform init -reconfigure
Initializing the backend...

Successfully configured the backend "local"! Terraform will automatically
use this backend unless the backend configuration changes.

No changes — state was silently migrated in the background The tracked resource shown as "will be created" — -reconfigure does not copy old state into the new backend location An error saying the backend is not yet initialized A prompt asking to confirm the migration retroactively

What is the correct time to run force-unlock, based on this real help text and the lock ID it requires?

$ terraform force-unlock -help
...
  This will not modify your infrastructure. This command removes the lock on the
  state for the current workspace. ... Local state files cannot be unlocked by
  another process.

Any time an apply seems to be taking too long Only after independently confirming the process holding the lock is genuinely gone, using the specific lock ID printed in the actual "state is locked" error As a routine first step before every apply, to guarantee no lock blocks it Only on local backends, since the help text mentions them specifically

A root.hcl remote_state block sets key = "${path_relative_to_include()}/terraform.tfstate" with no other per-unit config. Proven above by running terragrunt run -- init -backend=false in the vpc unit, what key lands in the generated backend.tf? key is left as the literal, unresolved expression string vpc/terraform.tfstate — path_relative_to_include() resolves per unit automatically, with zero extra config for that unit The key is always just terraform.tfstate regardless of which unit runs it Terragrunt refuses to generate the file until key is hardcoded per unit

Why does one Terragrunt remote_state block in root.hcl generating backend.tf scale better than every unit hand-writing its own backend "s3" block? It does not scale better, it is purely a style preference key resolves per-unit automatically via path_relative_to_include(), proven above with vpc/terraform.tfstate — hand-writing means re-deriving a correct unique key for every unit yourself with no collision check Hand-written backend blocks are not supported by Terraform at all Generated backend.tf files are encrypted while hand-written ones are not