Remote Backends and Locking
The idea in one sentence
Section titled “The idea in one sentence”A remote backend stores state in a shared location like Azure Storage instead of on one person’s laptop, and locking stops two people from writing to that shared state at the same time.
Why local state does not work for a team
Section titled “Why local state does not work for a team”By default, terraform.tfstate lives on the machine where you ran apply. That is fine for a solo experiment, but it breaks down the moment a second person joins:
- Teammate A runs
applyand creates a virtual network. Their laptop’s local state file now knows about it. - Teammate B, on a different laptop, has never seen that state file. They run
planagainst their own stale or empty local state and either try to recreate the virtual network or get a plan that makes no sense next to what actually exists. - CI/CD pipelines have the same problem in a sharper form — a pipeline runner is a fresh, disposable environment with no local state file at all. It needs a canonical copy of state to read from and write to on every run, not a copy that happens to be sitting on someone’s laptop.
The fix is to move state off any individual machine and into a shared backend that every team member and every CI run points at.
Configuring the azurerm backend
Section titled “Configuring the azurerm backend”The standard way to store Terraform state on Azure is an Azure Storage container configured as the backend:
terraform { backend "azurerm" { resource_group_name = "tfstate-rg" storage_account_name = "acmetfstate001" container_name = "tfstate" key = "networking.terraform.tfstate" }}resource_group_name and storage_account_name identify the storage account that holds state, container_name is the blob container inside that account, and key is the blob name within the container — so different configurations (or different Terragrunt units) typically use different keys in the same container. Every one of these four arguments is required for the backend to know exactly which blob holds this configuration’s state.
Two authentication arguments matter for how Terraform actually talks to that storage account, rather than legacy account-key access:
terraform { backend "azurerm" { resource_group_name = "tfstate-rg" storage_account_name = "acmetfstate001" container_name = "tfstate" key = "networking.terraform.tfstate" use_azuread_auth = true use_oidc = true }}use_azuread_auth = true authenticates to the storage account’s data plane using Microsoft Entra ID instead of a shared account key or SAS token. use_oidc = true layers OpenID Connect / workload identity federation on top — no long-lived secret to rotate at all, which is why the backend’s own documentation labels it “(Recommended)” for CI/CD, particularly GitHub Actions and Azure DevOps.
Locking is built in, not bolted on
Section titled “Locking is built in, not bolted on”Here is the detail that genuinely differs from some other clouds: the azurerm backend’s state locking is built into how Azure Blob Storage works, with nothing extra to configure. When Terraform writes state through this backend, it takes out a blob lease on the state blob before writing — a native Azure Blob Storage feature that grants exclusive write access to whoever holds the lease. A second apply that tries to acquire the same lease while it is held simply cannot, so it waits or fails cleanly instead of writing over the first run.
There is no separate lock table to create, no extra argument to set, and no additional Azure resource to provision just for locking. The four core arguments in the block above are all the azurerm backend needs — locking comes for free as soon as state is stored in Azure Storage. This is worth calling out precisely because it is simpler than some other clouds’ backends have historically been, where locking has needed a separate resource of its own to hold lock records.
Migrating a backend: init -migrate-state vs init -reconfigure
Section titled “Migrating a backend: init -migrate-state vs init -reconfigure”Changing the backend block — moving to a different storage account, changing key, or moving from local to azurerm for the first time — requires re-running terraform init, and Terraform gives you two different flags depending on whether you want the old state carried over. No cloud credentials exist in this course, so the proof below changes a local backend’s path rather than migrating to Azure Storage — the exact same prompt and mechanics apply to a local→azurerm move, only the backend type differs.
terraform init -migrate-state asks whether to copy existing state into the new backend location. Real output, changing path = "state/old.tfstate" to path = "state/new.tfstate" in a backend "local" block:
$ terraform init -migrate-stateInitializing the backend...Backend configuration changed!
Terraform has detected that the configuration specified for the backendhas changed. Terraform will now check for existing state in the backends.Do you want to copy existing state to the new backend? Pre-existing state was found while migrating the previous "local" backend to the newly configured "local" backend. No existing state was found in the newly configured "local" backend. Do you want to copy this state to the new "local" backend? Enter "yes" to copy and "no" to start with an empty state.
Enter a value: yes
Successfully configured the backend "local"! Terraform will automaticallyuse this backend unless the backend configuration changes.Answering yes really copies the resource into the new location — state/new.tfstate contains the same local_file resource the old path had. terraform init -reconfigure skips that prompt entirely and does not carry state over — proof, changing the path again and running -reconfigure instead:
$ terraform init -reconfigureInitializing the backend...
Successfully configured the backend "local"! Terraform will automaticallyuse this backend unless the backend configuration changes.No migration question at all. The next plan proves the difference — Terraform now believes the resource does not exist, because the new backend location was never populated:
$ terraform plan # local_file.config will be created + resource "local_file" "config" { + content = "listen=8080\n" + filename = "./generated/app.conf" }
Plan: 1 to add, 0 to change, 0 to destroy.-reconfigure is for “start clean at this new location, I do not want the old state” (or the old backend is gone/unreachable); -migrate-state is for “this is the same infrastructure, just move where its state record lives.” Picking the wrong one against a real storage account either leaves the old state behind unmigrated, or (worse) makes Terraform think a running virtual network needs to be recreated from scratch.
Partial backend configuration with -backend-config
Section titled “Partial backend configuration with -backend-config”A backend block does not have to hardcode every argument — leaving it empty and supplying values at init time (from a file or individual -backend-config="key=value" flags) keeps per-environment values out of committed .tf files:
terraform { backend "azurerm" {}}# backend.hcl (not committed, or generated by CI per environment)resource_group_name = "tfstate-rg"storage_account_name = "acmetfstate001"container_name = "tfstate"key = "networking.terraform.tfstate"terraform init -backend-config=backend.hclReal output against a local backend proving the flag is accepted and merged (path supplied the same way as the azurerm arguments would be):
$ terraform init -backend-config=backend.hclInitializing the backend...
Successfully configured the backend "local"! Terraform will automaticallyuse this backend unless the backend configuration changes.This is exactly the mechanism CI pipelines use to point the same configuration at a different storage account or key per environment (dev.hcl, prod.hcl) without duplicating the .tf files.
terraform force-unlock: the manual escape hatch
Section titled “terraform force-unlock: the manual escape hatch”If an apply is killed mid-run (a laptop sleeps, a CI job is cancelled), the blob lease it acquired can be left behind with nothing left running to release it. terraform force-unlock <LOCK_ID> removes that lock manually. Real -help output from Terraform 1.16.4:
$ terraform force-unlock -helpUsage: terraform [global options] force-unlock LOCK_ID
Manually unlock the state for the defined configuration.
This will not modify your infrastructure. This command removes the lock on the state for the current workspace. The behavior of this lock is dependent on the backend being used. Local state files cannot be unlocked by another process.
Options:
-force Don't ask for input for unlock confirmation.The LOCK_ID is not something you invent — when a lock is already held, the next run that fails to acquire it prints the lock’s ID (and who holds it, and when it was acquired) in its own error message, and that ID is what you paste into force-unlock. A stale lock genuinely cannot be produced offline in this course — the local backend the harness uses has no lock to leave behind the way a held blob lease can (the -help text above says exactly that: “Local state files cannot be unlocked by another process”). Treat force-unlock as a last resort for a lock you have independently confirmed is stale (the process that held the lease is actually gone — check first, e.g. that the CI run really did get cancelled), never a reflex for “my apply is being slow.”
Terragrunt: remote_state generates the backend.tf you never hand-write
Section titled “Terragrunt: remote_state generates the backend.tf you never hand-write”Terragrunt’s remote_state block in root.hcl generates the exact backend block above for every unit that includes it, resolving key per-unit with path_relative_to_include() so each unit gets its own state path automatically:
remote_state { backend = "azurerm" generate = { path = "backend.tf" if_exists = "overwrite" } config = { resource_group_name = "terraform-state-rg" storage_account_name = "mycompanytfstate" container_name = "tfstate" key = "${path_relative_to_include()}/terraform.tfstate" use_azuread_auth = true }}include "root" { path = find_in_parent_folders("root.hcl")}
terraform { source = "."}Running terragrunt run -- init -backend=false in the vnet unit generates a real backend.tf — every key from config lands verbatim, and key resolves to vnet/terraform.tfstate:
# Generated by Terragrunt. Sig: nIlQXj57tbuaRZEaterraform { backend "azurerm" { container_name = "tfstate" key = "vnet/terraform.tfstate" resource_group_name = "terraform-state-rg" storage_account_name = "mycompanytfstate" use_azuread_auth = true }}Every argument in the config map is emitted as-is — Terragrunt does not create the storage account or container for you the way it can auto-create a GCS bucket from project/location (module-versioning-and-registries.mdx’s GCP counterpart proves that contrast directly); the resource group, storage account, and container must already exist. This is the mechanism behind never hand-writing an azurerm backend block per environment: one remote_state block in the root config, and every unit underneath gets a correct, per-unit-scoped backend generated for it.
What locking actually prevents
Section titled “What locking actually prevents”Imagine locking did not exist. Two people, or a person and a CI run, both execute terraform apply against the same Azure-Storage-backed state at nearly the same moment. Both read the current state, both compute a plan, and both start writing their results back to the same state blob. Depending on timing, the second write can overwrite the first — silently dropping the first run’s recorded changes even though the first run’s infrastructure changes actually happened — or the state file can end up corrupted mid-write.
A lock turns that race into a queue. The first apply to start acquires the blob lease; the second apply either waits for the lease to be released, or fails immediately with a clear “state is locked” error, depending on the flags used. Either way, the second run does not get to write over the first run’s state while it is mid-flight.
# A run that finds the state already locked fails fast with a clear message,# rather than corrupting the state fileterraform apply# Error: Error acquiring the state lockflowchart LR eng1["Engineer A: terraform apply"] -->|acquires blob lease| storage["Azure Storage: state blob"] eng2["CI run: terraform apply"] -->|blocked, waits or fails| storage storage -->|lease released after apply| eng2
Mental Model
Section titled “Mental Model”Every mechanism in this lesson exists to answer one question honestly: “whose write wins, and does the loser find out?” A shared backend answers “where is the one true copy” so nobody’s laptop is that copy. Locking answers “what happens when two writes race” by turning the race into a queue instead of a coin flip — Azure’s answer happens to be free (a blob lease is part of storage itself, not a bolt-on). -migrate-state vs -reconfigure answers “does the old record follow me” when the backend’s location changes. force-unlock exists only because the queue mechanism itself can jam (a lease holder that will never release) — it is a manual override of the queue, not a replacement for it, which is exactly why it demands a real lock ID rather than a “just unlock everything” flag.
Trade-offs
Section titled “Trade-offs”| Choice | Benefit | Cost |
|---|---|---|
| Blob-lease locking built into azurerm backend | Zero extra resources to provision, pay for, or keep in sync (proven above: only the four core arguments are needed) | No separate audit trail resource the way a dedicated lock table might have its own access logs |
use_azuread_auth + use_oidc over shared account keys | No long-lived secret to rotate or leak; the backend’s own docs label OIDC “(Recommended)” for CI | Requires federated identity / Entra ID setup ahead of time, rather than copy-pasting an account key |
-migrate-state over -reconfigure when the backend location changes | Old state is carried forward — proven above, the resource is still tracked afterward | Requires answering an interactive prompt (or -input=false plus care) — easy to fat-finger “no” and lose the link to existing infrastructure |
Terragrunt remote_state generating backend.tf | One block in root.hcl correctly scopes every unit’s state path automatically (proven above with real generated output) | The generated file is not meant to be hand-edited — a manual edit is overwritten the next run (if_exists = "overwrite"); the storage account/container must already exist, unlike GCP’s bucket auto-create |
Common Misconceptions
Section titled “Common Misconceptions”- “
terraform init -reconfigureis just a safer alias for-migrate-state.” Proven above:-reconfigurenever prompts and never copies state — the very nextplanshows the tracked resource as “to create” again, because the new backend location was never populated. - “A stale lock is common and
force-unlockis a routine command.” Proven above via the real-helptext: it is a manual escape hatch for a lock whose holder is confirmed gone, not a default response to a slow or in-progressapply. - “Azure state locking must use a separate lock file or table, the way S3 uses DynamoDB.” Proven by the azurerm backend’s own docs quoted in
the-state-file.mdx: it locks by taking a blob lease directly on the state blob — there is no separate lock resource to provision. - “Terragrunt’s
remote_stateblock auto-creates the storage account and container the way it can auto-create a GCS bucket.” Proven above: the generatedbackend.tfcontains only the four config keys verbatim — the resource group, storage account, and container must already exist beforeinitcan succeed against them for real.
Reading AI-generated code
Section titled “Reading AI-generated code”When reviewing generated backend configuration or migration scripts, check for:
- A backend block still using a shared storage account key or SAS token where
use_azuread_auth/use_oidcwould remove the long-lived secret entirely — proven above, these are documented and recommended specifically to avoid secret rotation. terraform init -reconfigureused in a runbook step that is supposed to preserve existing state — proven above,-reconfiguredoes not migrate state at all; the correct flag for that intent is-migrate-state.- A CI script that pipes
yesinto everyterraform initunconditionally to avoid interactive prompts, including the-migrate-state“copy existing state?” prompt — that answer needs to be a deliberateyes(this environment really is a location change) or the script risks silently copying state into the wrong place. force-unlockcalled with a made-up or copy-pasted-from-a-tutorial lock ID rather than the one printed in the actual “state is locked” error for the run in question.- A hardcoded
resource_group_name/storage_account_name/container_name/keyin a shared module meant to be reused across environments, instead of an emptybackend "azurerm" {}filled by-backend-configper environment (or, in Terragrunt, oneremote_stateblock generating it correctly per unit).
🐛 Spot the bug
# CI "cleanup" step run before every apply, to avoid ever seeing a locked-state errorterraform force-unlock -force $(cat last-known-lock-id.txt)terraform apply -auto-approveShow the answer
This runs force-unlock unconditionally before every apply, using a lock ID cached from some previous run rather than the ID of any lock that actually exists right now. Proven above: force-unlock “will not modify your infrastructure” by itself, but blindly releasing a blob lease that is currently, legitimately held — by a teammate’s in-progress apply, or a concurrent CI run — removes the one protection state locking provides. The next line’s apply can now race directly against whatever was holding that lease, which is precisely the corrupted-write scenario locking exists to prevent. The correct use of force-unlock is reactive and manual: read the actual “state is locked” error for the ID it names, confirm independently that the holder is gone, and only then unlock that specific ID — never a pre-emptive step in a script that runs regardless of whether anything is actually locked.