Skip to content

Helm and Package Management

Helm is the package manager for Kubernetes: it turns a directory of templated manifests into a versioned, parameterized Chart that gets installed as a tracked Release.

The problem: raw YAML does not parameterize or version well

Section titled “The problem: raw YAML does not parameterize or version well”

A handful of Deployment, Service, and ConfigMap YAML files works fine for one environment. It falls apart the moment there is more than one: dev, staging, and prod need different replica counts, different resource limits, different image tags, and different hostnames, but the underlying shape of the manifests is identical. Copy-pasting the YAML three times and hand-editing each copy means every future change has to be applied three times, correctly, in sync. There is also no built-in notion of “this set of 12 manifests is one versioned unit” — kubectl apply treats each file independently, with no shared history and no easy way to package the whole set for someone else to install.

Helm solves both problems at once. It packages a set of related manifests into a single unit called a Chart, replaces hardcoded values with template placeholders, and tracks every install of that Chart as a Release with its own version history.

A Chart is a directory with three key pieces:

mychart/
├── Chart.yaml # name, version, and other chart metadata
├── values.yaml # default configuration values
└── templates/
├── deployment.yaml
├── service.yaml
└── configmap.yaml

Chart.yaml holds metadata (name, version, description). values.yaml holds the default configuration. Everything under templates/ is a Kubernetes manifest written with Go template syntax, where {{ .Values.xxx }} pulls a value in from values.yaml at render time:

# templates/deployment.yaml (excerpt)
apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ .Release.Name }}-web
spec:
replicas: {{ .Values.replicaCount }}
template:
spec:
containers:
- name: web
image: '{{ .Values.image.repository }}:{{ .Values.image.tag }}'
values.yaml
replicaCount: 2
image:
repository: myregistry/web
tag: '1.4.0'

helm template (renders locally without installing) or helm install fills in those placeholders and produces plain Kubernetes YAML — Helm never invents a new deployment mechanism, it just generates the manifests kubectl would apply anyway.

Supplying environment-specific configuration

Section titled “Supplying environment-specific configuration”

The same Chart deploys to dev, staging, and prod by overriding values.yaml at install time, either with a dedicated override file or individual --set flags:

Terminal window
# Render the manifests without installing, to sanity-check the output
helm template myapp ./mychart -f values-prod.yaml
# Install into prod using a prod-specific values file
helm install myapp ./mychart -f values-prod.yaml
# Override a single value on the command line instead
helm install myapp ./mychart --set replicaCount=5
values-prod.yaml
replicaCount: 5
image:
tag: '1.4.0'

The Chart’s templates never change between environments — only the values fed into them do.

Every helm install creates a Release: one named, tracked instance of a Chart running in the cluster. Helm keeps a revision history for each Release, so upgrades and rollbacks operate on the whole set of resources the Chart manages, not on one Deployment at a time — conceptually the same idea as kubectl rollout history for a single Deployment, just scoped to everything a Chart installed together.

Terminal window
# Upgrade an existing Release to a new chart version or new values
helm upgrade myapp ./mychart -f values-prod.yaml
# Roll an entire Release back to a previous revision
helm rollback myapp 2
# See the revision history for a Release
helm history myapp
# Remove a Release and the resources it created
helm uninstall myapp

Helm 4 versus Helm 3 — what actually changed

Section titled “Helm 4 versus Helm 3 — what actually changed”

This course is written against Helm 4.3.0. Helm 4’s own GitHub release for v4.0.0 is explicit about scope: “Helm v4 is a major version with backward incompatible changes including to the flags and output of the Helm CLI and to the SDK,” but adds that “the changes are not as extensive as those from Helm v2 to v3, with the goal that the majority of workflows remain compatible between Helm v3 and v4.” Existing Chart apiVersion v2 charts — the vast majority in the wild — “will continue to be supported in Helm v4” with no rewrite required. The release notes list, among other things, a redesigned plugin system (with WebAssembly-based plugin support), post-renderers implemented as plugins, first-class server-side apply support, kstatus-based resource-readiness watching, and local content-based chart caching.

Two concrete CLI differences a course or an AI assistant trained mostly on Helm 3 will get wrong, proven directly against the installed v4.3.0 binary:

$ helm install --help | grep -A1 "server-side"
--server-side object updates run in the server instead of the client (default true)
$ helm upgrade --help | grep -A1 "take-ownership"
--take-ownership if set, upgrade will ignore the check for helm annotations and take
ownership of the existing resources

Server-side apply is the default in Helm 4 (--server-side defaults to true on helm install; helm upgrade takes a tri-state --server-side auto|true|false) — the exact same Server-Side Apply mechanism this course already covers for kubectl apply --server-side, with the same managedFields/field-manager machinery and the same --force-conflicts escape hatch (Helm’s flag is spelled --force-conflicts too). --take-ownership is new in Helm 4: it lets helm install/helm upgrade adopt resources that already exist in the cluster without the meta.helm.sh/release-name annotation Helm normally requires before it will touch an object — useful for importing hand-applied manifests into a Release, dangerous if pointed at the wrong object.

A third change is a real, working-directory-breaking one, proven by running the command directly:

$ helm upgrade --help | grep -A1 -- "--atomic"
Error: unknown flag: --atomic
Flag --atomic has been deprecated, use --rollback-on-failure instead

--atomic still parses (it is accepted with a deprecation warning) but its replacement, --rollback-on-failure, is the flag Helm 4 documents going forward — a Helm-3-trained AI assistant reaching for --atomic in new pipeline code is not wrong, just behind.

helm template versus helm install —dry-run=server: a real, provable diff

Section titled “helm template versus helm install —dry-run=server: a real, provable diff”

helm template never talks to a cluster — it is a pure client-side Go-template render. helm install --dry-run=server renders the same manifests but then actually sends them to the API server with dry-run=server, so schema validation, admission plugins, and defaulting all run for real, exactly like kubectl apply --dry-run=server elsewhere in this course. Run back-to-back against the identical Chart and values:

orders-api/templates/service.yaml
$ helm template orders ./orders-api
---
apiVersion: v1
kind: Service
...
---
# Source: orders-api/templates/deployment.yaml
apiVersion: apps/v1
kind: Deployment
...
---
# Source: orders-api/templates/tests/test-connection.yaml
apiVersion: v1
kind: Pod
metadata:
name: "orders-orders-api-test-connection"
annotations:
"helm.sh/hook": test
...
$ helm install orders ./orders-api --dry-run=server
NAME: orders
LAST DEPLOYED: Sat Sep 26 18:15:10 2026
NAMESPACE: default
STATUS: pending-install
REVISION: 1
DESCRIPTION: Dry run complete
HOOKS:
---
# Source: orders-api/templates/tests/test-connection.yaml
apiVersion: v1
kind: Pod
metadata:
name: "orders-orders-api-test-connection"
annotations:
"helm.sh/hook": test
...
MANIFEST:
---
# Source: orders-api/templates/service.yaml
...
---
# Source: orders-api/templates/deployment.yaml
...

helm template dumps every rendered document — including the helm.sh/hook: test Pod — as one flat stream with no notion of Release state. helm install --dry-run=server separates HOOKS from MANIFEST (a hook is not part of the tracked Release manifest, it only runs on demand via helm test) and reports the exact NAME/STATUS/REVISION metadata a real install would produce, because it genuinely round-trips through the API server rather than just filling in a template.

Values precedence, proven with a real chart

Section titled “Values precedence, proven with a real chart”

Helm merges values from lowest to highest precedence: the chart’s own values.yaml, then any -f/--values files in the order given (later files win on overlapping keys), then --set/--set-string/--set-file last. Proven with a chart whose values.yaml sets replicaCount: 2 and image.tag: 1.27-alpine, an override file setting replicaCount: 3 and image.tag: "1.27", and a --set flag overriding the tag once more:

$ cat values-prod.yaml
replicaCount: 3
image:
tag: "1.27"
$ helm upgrade --install orders ./orders-api -f values-prod.yaml --set image.tag=1.27-alpine --rollback-on-failure --wait
Release "orders" has been upgraded. Happy Helming!
NAME: orders
...
REVISION: 6
$ helm get values orders
USER-SUPPLIED VALUES:
image:
tag: 1.27-alpine
replicaCount: 3
$ helm get values orders -a
COMPUTED VALUES:
image:
repository: nginx
tag: 1.27-alpine
replicaCount: 3
resources:
requests:
cpu: 100m
memory: 64Mi

replicaCount ends up 3 (the -f file beat the chart default of 2, and no --set touched it), while image.tag ends up 1.27-alpine (the --set flag beat the -f file’s "1.27", which had already beaten the chart default) — exactly the documented values.yaml → -f (in order) → --set precedence, not a guess about it.

helm rollback and a real, messy revision history

Section titled “helm rollback and a real, messy revision history”

helm upgrade --install --rollback-on-failure --wait (the Helm 4 name for --atomic) automatically rolls a failed upgrade back — proven here across two real, back-to-back upgrade failures on the exact same Release (the first failure was worker nodes not yet Ready, the second was a genuinely bad, non-existent image tag causing ImagePullBackOff):

$ helm history orders
REVISION UPDATED STATUS CHART APP VERSION DESCRIPTION
1 Sat Sep 26 18:15:18 2026 superseded orders-api-0.1.0 1.0.0 Install complete
2 Sat Sep 26 18:15:25 2026 failed orders-api-0.1.0 1.0.0 Upgrade "orders" failed: resource Deployment/default/orders-orders-api not ready...
3 Sat Sep 26 18:16:55 2026 superseded orders-api-0.1.0 1.0.0 Rollback to 1
4 Sat Sep 26 18:17:08 2026 failed orders-api-0.1.0 1.0.0 Upgrade "orders" failed: resource Deployment/default/orders-orders-api not ready...
5 Sat Sep 26 18:20:08 2026 superseded orders-api-0.1.0 1.0.0 Rollback to 3
6 Sat Sep 26 18:20:28 2026 deployed orders-api-0.1.0 1.0.0 Upgrade complete

Every failed upgrade (revisions 2 and 4) produced its own automatic rollback revision (3 and 5) with no operator intervention — --rollback-on-failure genuinely protects a Release from being left half-upgraded, but it is not a substitute for fixing the underlying cause: an upgrade with a typo’d, non-existent image tag will loop through exactly this fail-then-rollback cycle forever if a CI pipeline blindly retries it, since the rollback restores the previous working state, not the intended new one. A manual helm rollback orders <revision> does the same thing on demand, without needing a failure to trigger it.

helm lint statically checks a chart’s structure and templates before anything is rendered against a cluster:

$ helm lint ./orders-api
==> Linting orders-api
[INFO] Chart.yaml: icon is recommended
1 chart(s) linted, 0 chart(s) failed

A chart can also ship its own helm.sh/hook: test Pod under templates/tests/ — helm test runs it against the already-installed Release and reports pass/fail, which is a real smoke test of the live Service, not a lint of the template source:

$ helm test orders
NAME: orders
...
TEST SUITE: orders-orders-api-test-connection
Last Started: Sat Sep 26 18:23:09 2026
Last Completed: Sat Sep 26 18:23:16 2026
Phase: Succeeded

helm diff: seeing the change before you make it

Section titled “helm diff: seeing the change before you make it”

The built-in helm upgrade already shows what would change via --dry-run, but the community helm-diff plugin renders a proper unified diff against the live Release. It is not bundled — installing it in Helm 4 surfaces the plugin system’s new signature-verification step directly:

$ helm plugin install https://github.com/databus23/helm-diff
Error: plugin source does not support verification. Use --verify=false to skip verification
$ helm plugin install https://github.com/databus23/helm-diff --verify=false
WARNING: Skipping plugin signature verification
Downloading https://github.com/databus23/helm-diff/releases/download/v3.15.15/helm-diff-macos-arm64.tgz
Installed plugin: diff
$ helm diff upgrade orders ./orders-api -f values-prod.yaml --set image.tag=1.27-alpine --set replicaCount=4
default, orders-orders-api, Deployment (apps) has changed:
spec:
- replicas: 3
+ replicas: 4
...

Helm 4 verifies plugin signatures by default and refuses an unsigned source outright rather than silently trusting it — --verify=false is an explicit, auditable opt-out, not a default you fall into.

Modern Helm distributes charts the same way container images are distributed — as OCI artifacts, addressed with an oci:// reference instead of a classic chart-repository index. Proven end-to-end against a throwaway local registry:

$ helm package ./orders-api -d .
Successfully packaged chart and saved it to: orders-api-0.1.0.tgz
$ helm push orders-api-0.1.0.tgz oci://localhost:5050/charts --plain-http
Pushed: localhost:5050/charts/orders-api:0.1.0
Digest: sha256:23c5fd204fbcc25d18e0111a2b1846d6335a2657e98c7644635fb3213b1a95cb
$ helm pull oci://localhost:5050/charts/orders-api --version 0.1.0 -d ./pulled --plain-http
Pulled: localhost:5050/charts/orders-api:0.1.0
Digest: sha256:23c5fd204fbcc25d18e0111a2b1846d6335a2657e98c7644635fb3213b1a95cb
$ helm show chart oci://localhost:5050/charts/orders-api --version 0.1.0 --plain-http
apiVersion: v2
appVersion: 1.0.0
name: orders-api
version: 0.1.0

--plain-http is only needed here because the demo registry has no TLS certificate — a real registry (ghcr.io, ECR, Artifact Registry, a private Harbor) is addressed the same way over HTTPS with no extra flag, and helm install/helm upgrade accept an oci:// reference exactly like a classic repo/chart reference.

A tiny chart, a library chart, and why the templates themselves are excerpts

Section titled “A tiny chart, a library chart, and why the templates themselves are excerpts”

The chart proven throughout this lesson is genuinely two charts: orders-api (the application chart) declaring a dependencies: entry on a second chart, common, whose Chart.yaml sets type: library. A library chart ships no installable templates of its own — only reusable, defined helpers other charts include:

# charts/common/templates/_helpers.tpl — excerpt, not a standalone manifest
{{- define "common.fullname" -}}
{{ .Release.Name }}-{{ .Chart.Name }}
{{- end -}}
{{- define "common.labels" -}}
app.kubernetes.io/name: {{ .Chart.Name }}
app.kubernetes.io/instance: {{ .Release.Name }}
{{- end -}}

orders-api’s own templates/deployment.yaml then calls {{ include "common.fullname" . }} and {{- include "common.labels" . | nindent 4 }} — every .tpl helper file and every chart template containing {{ }} Go-template syntax is an intentional excerpt in this lesson, not a complete standalone manifest: the harness that validates this course’s # k8s/... fences against a real API server skips them on purpose, because {{ .Release.Name }} is not valid YAML until Helm has rendered it. The proof that the templates are correct is the rendered helm template/helm install --dry-run=server output shown above, not a raw apply of the template source.

flowchart LR
  values["values.yaml (defaults)"] --> render["helm install / helm template"]
  overrides["values-prod.yaml or --set"] --> render
  templates["templates/ (Go-templated manifests)"] --> render
  render --> manifests["Rendered Kubernetes manifests"]
  manifests --> cluster["Applied to cluster as Release myapp, revision N"]
A Chart's values and templates render into manifests, applied to the cluster as a tracked Release

Helm is a template renderer plus a revision tracker, nothing more mystical than that: values.yaml (and its overrides) is the only input that ever changes, templates/ is the fixed Go-template source, and every helm install/upgrade/rollback is really “render, then let the API server reconcile the result,” proven above by helm install --dry-run=server genuinely round-tripping through the API. Helm 4 changed how carefully that reconciliation happens (server-side apply by default, a signature-verified plugin system, --rollback-on-failure in place of --atomic) but not the basic model — a Release’s revision history is exactly kubectl rollout history scoped to everything the Chart manages, not a separate concept.

ChoiceBenefitCost
helm templateFast, no cluster needed, safe to run anywhereNever catches a real API-server rejection — proven above, only --dry-run=server does
helm install/upgrade --dry-run=serverProven above: real schema validation and a real NAME/STATUS/REVISION previewRequires live cluster access, slower than a pure template render
--rollback-on-failure (Helm 4 name for --atomic)Proven above: automatically undoes a failed upgrade, no half-deployed ReleaseProven above: a bad image tag loops fail→rollback forever if blindly retried — it protects state, it does not diagnose the cause
helm-diff pluginProven above: real unified diff against the live Release before upgradingThird-party, unsigned by default — proven above, Helm 4 requires an explicit --verify=false to install it
Library chart (type: library) for shared helpersOne _helpers.tpl, reused by every consuming chart, no copy-pasteCannot be installed on its own — it only exists as a dependency, and its own templates are unrenderable excerpts by design
OCI (oci://) chart registryProven above: same registry infrastructure as container images, no separate chart-repo index to hostNeeds --plain-http or a real TLS cert — an OCI registry with neither simply refuses the connection
  • “helm template and helm install —dry-run=server prove the same thing.” Proven above: helm template never contacts the cluster at all; only --dry-run=server sends the render to the real API server and gets real schema validation and admission behavior back.
  • “—atomic is still the flag to reach for in Helm 4.” Proven above with the real CLI: --atomic still parses with a deprecation warning, but Helm 4’s documented flag is --rollback-on-failure.
  • “Values from a -f file always win over —set, since the file is more explicit.” Proven above: --set is the highest-precedence source, applied after every -f file, not before.
  • “A library chart is just a normal chart with no default values.” Proven above: type: library in Chart.yaml means the chart has no installable templates of its own at all — only defined helpers, never applied directly to a cluster.
  • “helm lint would have caught the ImagePullBackOff from a bad image tag.” helm lint only checks chart structure and template syntax statically — proven above, it never contacts a registry or a cluster, so a non-existent image tag lints clean and only fails at real helm upgrade --wait time.

When reviewing generated Helm charts, CI pipeline steps, or upgrade scripts, check for:

  • helm install/helm upgrade with no --rollback-on-failure (or the older --atomic) in a CI pipeline — proven above, a failed upgrade with neither flag leaves the Release in a failed state and the old Pods potentially already torn down, with nothing rolling it back automatically.
  • --atomic used in new Helm 4 pipeline code — proven above, it still works but is deprecated; --rollback-on-failure is the current flag.
  • A chart template (.tpl file, or any file under templates/ containing {{ }}) applied directly with kubectl apply -f instead of rendered first — Go-template syntax is not valid YAML, proven above by this course’s own harness skipping such fences on purpose.
  • helm plugin install from a URL with --verify=false used as a default habit rather than a deliberate, reviewed exception — proven above, Helm 4’s default is to require a verifiable signature.
  • A chart with no helm.sh/hook: test Pod and no helm test step in the deploy pipeline — the difference between “the chart rendered” and “the running Service actually answers a request” is exactly what helm test proves and a template-only review cannot.
  • helm upgrade piped straight from CI with no preceding helm diff (or at minimum --dry-run) step — proven above, helm diff shows the exact live-Release change before it happens, catching an unintended replicas/image drop before it reaches production.

🐛 Spot the bug

scripts/deploy-orders-api.sh
#!/usr/bin/env bash
set -euo pipefail
helm upgrade --install orders ./orders-api \
-f values-prod.yaml \
--set image.tag="${IMAGE_TAG}" \
--wait --timeout 5m
Show the answer

This upgrade has no --rollback-on-failure (Helm 4’s current name; --atomic still works but is deprecated). Proven above with this course’s own Release history: two real upgrade failures each needed an automatic rollback to avoid leaving the Deployment half-updated — without either flag, a failed --wait here (a bad IMAGE_TAG, a probe that never turns healthy) leaves the Release in STATUS: failed with whatever partial rollout already happened, and nothing reverts it. The fix is --rollback-on-failure added to the same command.

What is the difference between a Helm Chart and a Helm Release
How is environment-specific configuration typically supplied to a Chart
How do helm upgrade and helm rollback relate to kubectl rollout for a single Deployment
What does helm template do differently from helm install --dry-run=server
What should an AI-generated Helm 4 deploy script use instead of --atomic
$ helm upgrade --help | grep -A1 -- "--atomic"
Flag --atomic has been deprecated, use --rollback-on-failure instead
Why did the plain helm plugin install command fail in Helm 4
$ helm plugin install https://github.com/databus23/helm-diff
Error: plugin source does not support verification. Use --verify=false to skip verification