Verification tools

Every tool below was actually run against real Terraform 1.16.4 / Terragrunt 1.1.0 configuration with the pinned hashicorp/azurerm 5.7.0 provider and the pinned ghcr.io/terraform-linters/tflint, bridgecrew/checkov, and openpolicyagent/conftest Docker images this course uses — never against a real Azure subscription (every provider config in this course uses features {} + a placeholder subscription_id, and terraform validate never contacts Azure at all; real azurerm plan always needs a real authorizer, so plan-shaped proofs in this course use terraform_data or terraform test with mock_provider, see the AI Bug Catalog’s entry 9). This is the ladder in the order it actually catches things, cheapest and fastest first.

flowchart TD
  A["fmt -check<br/>formatting only"] --> B["validate<br/>schema + type errors + deprecation warnings"]
  B --> C["plan -detailed-exitcode<br/>real diff against real (mock) provider schema"]
  C --> D["show -json + jq<br/>machine-readable plan, what CI/policy actually reads"]
  D --> E["terraform test + mock_provider<br/>assertions against a plan, no real infra"]
  E --> F["tflint / checkov<br/>real rule catalogs, cloud-specific findings"]
  F --> G["conftest<br/>custom Rego policy against show -json"]
  G --> H["terragrunt hcl validate/render/run --all<br/>the live tree, offline"]

The verification ladder — what each rung catches

terraform fmt -check — formatting, not correctness

$ terraform fmt -check -diff main.tf

Exits non-zero and prints a diff if formatting is off; fixes nothing itself unless run without -check. Catches zero semantic bugs from this module’s catalog — a perfectly formatted file can still hardcode a wrong account ID. It’s the cheapest possible CI gate, and the one this course’s own harness runs first, before anything else, over every fence.

terraform validate — schema, types, and (crucially) warnings

$ terraform validate -json

Catches: entry 3 (module + count incompatibility), entry 11 (for_each on an unknown value), entry 13 (hallucinated argument), entry 15 (reference cycle) — all real, hard failures. Also catches entry 2 (deprecated arguments) — but only as a warning, which a CI step that checks terraform validate; echo $? alone silently ignores. The real, useful CI check greps the JSON:

terraform validate -json | jq -e '[.diagnostics[] | select(.severity=="warning")] | length == 0'

validate never contacts a real provider API — it checks the configuration against the provider’s schema, downloaded once at init, entirely offline.

terraform plan -detailed-exitcode — a real diff, not just a syntax check

$ terraform plan -detailed-exitcode

Exit code 0 = no changes, 1 = error, 2 = changes present — the exit code CI actually branches on, instead of parsing plan text. Real azurerm plan always needs a real Azure authorizer (proven directly in the AI Bug Catalog’s entry 9), so the plan-shaped proofs in this catalog — entry 10’s count index-shift, entry 7’s nullable substitution — use terraform test with mock_provider "azurerm" {} instead of a raw plan, which is genuinely offline for any provider. Catches the same class of bug validate alone cannot see, because it only shows up as a diff, not a schema violation.

terraform show -json + jq — what CI and policy tools actually read

$ terraform plan -out=plan.out && terraform show -json plan.out > plan.json
$ jq '.resource_changes[] | select(.change.actions[0] == "delete") | .address' plan.json

This is the schema conftest/OPA (below) and this course’s own harness read — never plan’s human-readable text. Proven directly in entry 8: an ephemeral variable’s literal default shows up in this exact JSON’s variables block even though it’s correctly kept out of terraform.tfstate. Any secret-shaped string an archived show -json plan artifact might leak is caught here, not in the CLI’s redacted output.

terraform test with mock_provider

# tests/example.tftest.hcl (fragment — see modules/testing-modules for a full worked example)
mock_provider "azurerm" {}

run "plan_only" {
  command = plan
  assert {
    condition     = azurerm_linux_virtual_machine.app.size == "Standard_B1s"
    error_message = "unexpected size"
  }
}

terraform test (1.6+) with mock_provider (1.7+) runs full plan-time assertions with zero real provider calls at all — this is exactly what makes it useful for azurerm resources, where a real plan needs a real subscription and a real authorizer (see Testing Modules for a full worked example, including override_resource). This is the only rung on this ladder that works identically well across all three clouds regardless of what a real plan can reach.

tflint — real, current findings against a real rule catalog

$ docker run --rm -v "$(pwd)":/data -v <plugin-cache>:/root/.tflint.d -w /data ghcr.io/terraform-linters/tflint --init
$ docker run --rm -v "$(pwd)":/data -v <plugin-cache>:/root/.tflint.d -w /data ghcr.io/terraform-linters/tflint

Real output against a bare azurerm_linux_virtual_machine with no required_providers/required_version block:

2 issue(s) found:
Warning: Missing version constraint for provider "azurerm" in `required_providers` (terraform_required_providers)
Warning: terraform "required_version" attribute is required (terraform_required_version)

Catches genuinely current findings — real rule IDs proven against the pinned tflint-ruleset-azurerm docs. tflint has no equivalent of entry 21 for azure — that catalog entry is a checkov skip-comment mismatch instead (#checkov:skip=CKV_AZURE_35 silencing nothing while CKV_AZURE_59 still fails), proven below. --init must persist its downloaded plugin across runs — mount a real volume for ~/.tflint.d, not a fresh one per docker run --rm, or every run re-downloads and a stale cache silently masks plugin-install failures.

checkov — real, current findings

$ docker run --rm -v "$(pwd)":/tf bridgecrew/checkov -d /tf --compact --quiet

Real output against a bare azurerm_key_vault:

terraform scan results:
Passed checks: 1, Failed checks: 5, Skipped checks: 0
Check: CKV_AZURE_189: "Ensure that Azure Key Vault disables public network access"
Check: CKV_AZURE_42: "Ensure the key vault is recoverable"
Check: CKV_AZURE_110: "Ensure that key vault enables purge protection"

Same caveat as tflint: a checkov skip comment naming a real-but-wrong check ID is silently accepted rather than rejected — proven directly in this catalog’s entry 21, where #checkov:skip=CKV_AZURE_35 leaves the actually-failing CKV_AZURE_59 untouched, still reported, exactly as if no skip comment existed.

conftest — custom Rego policy against show -json

$ docker run --rm -v "$(pwd)":/project openpolicyagent/conftest test --policy policy plan.json

Reads exactly the show -json output from the rung above — proven throughout entry 20’s real FAIL/PASS output. Current conftest defaults to Rego v1 syntax (import rego.v1, contains/if): pre-v1 deny[msg] { ... } fails to even load, with a real rego_parse_error, not a rule that silently never fires.

terragrunt hcl validate / render / run --all plan — the live tree, offline

$ terragrunt hcl validate --working-dir live/prod/vpc      # recursive HCL syntax check
$ terragrunt render --format json --working-dir live/dev/vpc   # resolved config, offline
$ terragrunt run --all plan --working-dir .                 # dependency-ordered, offline with mock_outputs

hcl validate only checks that the HCL parses — proven directly in entry 17, it passes on two terragrunt.hcl files that are byte-identical copy-pastes deploying against the wrong account. render resolves every include/local/function call to its final value without ever calling terraform — the fastest way to actually see whether two environments’ resolved inputs differ. run --all plan requires a root.hcl (or a --terragrunt flag on this course’s own harness) and runs every unit in real dependency order — run-all/plan-all (entry 16) are both unknown commands as of the 1.1 CLI redesign; this is the real, current replacement.

This course’s own harness: tools/verify-snippets.mjs

Every hcl fence in this course whose first line is # <path> ending .tf/.hcl/.tftest.hcl/.tfvars is collected as a real file at that path (relative to its lesson) and run through terraform init -backend=false + validate + fmt -check, plus test/plan/terragrunt hcl validate/render under the --test/--plan/--terragrunt flags. Honestly, two real limitations:

• -backend=false always. Any backend "s3"/"azurerm"/"gcs" block a lesson declares is never actually initialized — this course never has real backend credentials, so entry 5’s backend-locking claims are docs-sourced, not harness-run (see that entry for the exact citation).
• azurerm cannot plan offline for most resources. This harness always tries terraform plan and pattern-matches known offline-auth-failure phrases to print not possible offline: <reason> instead of a raw stack trace — real: azurerm always tries to build a Resource Manager authorizer, even for a plan -destroy -refresh=false against a hand-authored state, proven directly in the AI Bug Catalog’s entry 9 (unable to build authorizer for Resource Manager API ... exec: "az": executable file not found). aws’s mock credentials (skip_credentials_validation, skip_requesting_account_id, skip_metadata_api_check, skip_region_validation) are what let aws plan proceed fully offline where azurerm generally cannot.

A fence whose first line contains @expect-error (used throughout the AI Bug Catalog for every deliberately-broken snippet) is deliberately skipped by this harness — the lesson prose carries the real, already-captured error instead of asking the harness to re-derive it on every run.

Mental Model

The verification ladder, cheapest and fastest first:

1. fmt -check           → formatting only, zero semantic coverage
2. validate              → schema/type errors + warnings (check warnings explicitly!)
3. plan -detailed-exitcode → a real diff — count/for_each churn, lifecycle scope, nullable substitution
4. show -json + jq        → what CI/policy tools actually read, not plan's human text
5. test + mock_provider   → plan-time assertions, zero real provider calls, works for any cloud
6. tflint / checkov        → real rule catalogs — but silent on an invented rule ID
7. conftest                → custom policy against show -json — v1 Rego syntax required
8. terragrunt hcl validate/render/run --all → the live tree, offline, dependency-ordered

Passing every rung proves the configuration is well-formed and internally consistent.
It does not prove prevent_destroy protects the right resource, or that dev and prod
actually deploy against different accounts — those need a human reading the checklist.

Common Misconceptions

• “terraform validate passing means the config is correct.” It means the config is schema-valid. Proven throughout the catalog: 15 of 23 entries validate and plan cleanly.
• “A green tflint/checkov run means every intended check actually ran.” Proven directly in entry 21: an invented rule ID produces the identical “success” output as a correctly-named one — the tool has no way to tell “this rule doesn’t exist” from “this rule found nothing to flag.”
• “-backend=false is just a harness limitation, not a real concern.” It’s also the honest state of most CI validate/plan steps before a real apply stage — which is exactly why entry 5’s backend-locking drift is invisible to validate/plan in real pipelines too, not just this course’s harness.

🏗️ The one-line takeaway

Each rung of this ladder catches a specific, narrow class of bug — fmt catches formatting, validate catches schema errors and (if you check for them) warnings, plan catches real diffs, test/tflint/checkov/conftest catch policy and provider-specific findings, and terragrunt catches the live tree’s structure — but passing all of them together still does not prove intent: that the right resource is protected, that two environments actually differ, or that a policy rule you wrote is spelled correctly enough to actually run.

Why does this lesson say terraform validate passing is not enough on its own, even though it does catch real errors like entry 3's module+count conflict?

validate doesn't actually catch any real errors validate only fails on hard schema/type errors — a deprecated argument (entry 2) or a hallucinated-but-schema-valid choice still validates successfully, and validate's own warnings are easy to miss if CI only checks the exit code validate requires real cloud credentials to run at all validate and plan always produce identical results

Why can aws's terraform plan run fully offline in this course while azurerm generally cannot, per the harness's own documented limitation?

azurerm and google providers are broken aws's provider config sets skip_credentials_validation, skip_requesting_account_id, and skip_metadata_api_check, letting most resource types plan without any real AWS call — azurerm needs a real Resource Manager authorizer for essentially every plan, with no equivalent skip flags on the provider block Terraform plan never needs any credentials for any provider This course never runs azurerm/google plan at all, even with mock_provider

What does a real checkov run report when a #checkov:skip= comment names a real, current check ID that simply isn't the one actually failing?

An explicit 'wrong check ID' error that fails the run The actually-failing check still shows up in the report, unaffected — proven directly, checkov has no way to know a skip comment's target ID doesn't match reviewer intent checkov refuses to start entirely The skip comment silences every finding on that resource

Submit

Mark as complete