deploy

A Go CLI tool for deploying Helm charts and raw Kubernetes manifests via kapp and vals.
Supports multiple environments, a cluster-level and an environment-level deployment file, namespace filtering, optional CRD deployment and a --dry-run mode that shows what would change without applying anything.

---

Requirements

Tool	Purpose
helm	Render Helm charts
vals	Resolve secret references in YAML
kapp	Apply / delete Kubernetes resources (including Namespace objects)

All three tools must be available in $PATH. kubectl is no longer needed.

---

Build

go build -o deploy .

No external Go dependencies – only the standard library is used.

Tests

go test ./...

The test suite installs fake helm/vals/kapp binaries on $PATH, so neither the real tools nor a cluster are needed.

---

Usage

./deploy [flags]

Flag	Short	Default	Description
--env	-e	staging	Environment name (folder under --cluster and --deploy, e.g. dev, staging, production)
--deploy-file	-f	_deployment.json	Name of the deployment file looked up in <cluster>/<env>/ and <deploy>/<env>/. Must be a plain file name, no path
--namespace	-n	(all)	Only process the matching namespace group(s) in the deployment file(s)
--deploy	-d	deploy	Root folder containing the per-environment deployment folders (<deploy-root>/<env>)
--apps	-a	apps	Directory holding shared/per-app values files. Use e.g. --apps base for the legacy layout — there is no automatic fallback
--cluster	-c	cluster	Directory holding cluster-wide values (default.yaml/.yml + <env>/) and the cluster-level deployment file (<env>/_deployment.json)
--vars	-v	(none)	Comma-separated list of KEY=VALUE env files, loaded into the process environment before anything runs (see Env files below)
--only	-o	(both)	Only process the cluster or the deploy deployment file. Error if the selected file does not exist
--dry-run	(none)	false	Show what would change without applying anything
--help	-h	–	Print usage and exit

The short forms are full aliases for the same flag — -e staging and --env staging do exactly the same thing, and both forms can be freely mixed in one call. --dry-run deliberately has no short form (to avoid clashing with -d/--deploy).

Every values file the tool looks up accepts either a .yaml or a .yml extension (both are checked; if both exist, both are applied, .yaml first).

Examples

# Deploy staging: cluster/staging/_deployment.json, then deploy/staging/_deployment.json
./deploy

# Deploy a specific environment
./deploy --env production

# Use another deployment file name in both locations
./deploy --env production --deploy-file _deployment_monitoring.json
./deploy -e production -f _deployment_monitoring.json

# Deploy only the monitoring namespace (in both files)
./deploy --namespace monitoring

# Dry-run for production
./deploy --env production --dry-run

# Custom folder layout
./deploy --env production --apps charts --cluster cluster-config --deploy environments

# Load environment variables from one or more files (later files win)
./deploy --env production --vars ./secrets/common.env,./secrets/production.env

# Two-stage rollout: first namespaces, RBAC (user-management) etc., then the apps
./deploy -e production -o cluster
./deploy -e production -o deploy

# Short flags (--dry-run has no short form)
./deploy -e production -n monitoring -v ./secrets/common.env,./secrets/production.env --dry-run

---

Deployment files

The tool looks for exactly one file name (default _deployment.json, overridable with --deploy-file / -f) in two places and processes them in this order:

Order	File	Allowed content	kapp app ID
1	<cluster>/<env>/<deploy-file>	Namespaces with global_manifest, namespaced_manifest and applications	cluster-<env>-<namespace> (namespace manifests) / cluster-<env>-<namespace>-<name> (apps)
2	<deploy>/<env>/<deploy-file>	Namespaces with global_manifest, namespaced_manifest and applications	<env>-<namespace> (namespace manifests) / <env>-<namespace>-<name> (apps)

• Both files are optional, but at least one must exist — otherwise the tool exits with no deployment file "<name>" found in ....
• All namespaces of the cluster file are fully processed first, then the deploy file.
• If the cluster file fails (invalid content, parse error, missing manifest, failed kapp deploy), the deploy file is not processed.
• --deploy-file applies to both locations. With -f custom.json, _deployment.json is ignored in both places.
• _deployment_*.json files are no longer picked up automatically. To use one, name it explicitly via -f.
• --namespace filters groups in both files.
• --only cluster / --only deploy (-o) processes only one of the two files; the other one is not even parsed. It is an error if the selected file does not exist. Combinable with -n, -f and --dry-run.

Cluster file (<cluster>/<env>/_deployment.json)

The cluster file sets up namespaces, namespace-wide resources (pull secrets, network policies, quotas, …) and cluster-wide applications such as RBAC (ClusterRoles, Roles, bindings). It uses the same format as the deploy file:

{
  "global_manifest": ["manifests/imagePullSecret.yaml"],
  "app_k8s_deploy_helm_charts": [
    {
      "namespace": "monitoring",
      "namespaced_manifest": ["namespace.yaml", "monitoring-network-policy.yaml"]
    },
    {
      "namespace": "user-management",
      "namespaced_manifest": ["namespace.yaml"],
      "applications": [
        {
          "name": "user-management",
          "chart_ref": "oci://reg.h2-invent.com/charts/user-management",
          "version": "1.0.0",
          "state": "present",
          "manifest": ["group-bindings.yaml"]
        }
      ]
    }
  ]
}

Rules (checked before anything is deployed; violations abort the run):

• Every group needs a non-empty namespace, and each namespace may only appear once.

Per namespace group, the tool deploys:

1. one kapp app cluster-<env>-<namespace> with all global_manifest entries, then all namespaced_manifest entries of that group (only if at least one exists)
2. one kapp app per application cluster-<env>-<namespace>-<name>

A group without manifests and applications deploys nothing (a warning is printed).

Cluster apps get the same values files as deploy-file apps (see Directory structure); only the per-namespace override is looked up in <cluster>/<env>/<namespace>/<name>.yaml|.yml.

RBAC example (user-management): the chart renders the ClusterRoles/Roles; the group bindings are listed as the app's manifest (<cluster>/<env>/user-management/group-bindings.yaml), so roles and bindings live in the same kapp app and are deployed and deleted together. Bindings with their own metadata.namespace need that namespace to exist — define it in the cluster file as well, since the cluster file runs first. The deploying identity needs bind/escalate on the referenced roles (or cluster-admin).

kapp app names are stored as Kubernetes object names, which must not contain _. The prefix is therefore cluster-<env> (e.g. cluster-staging-monitoring), not cluster_<env>.

Deploy file (<deploy>/<env>/_deployment.json)

Namespaces, global_manifest, namespaced_manifest and applications as described in Deployment file format. Per namespace, the tool deploys the app <env>-<namespace> (manifests, if any) first, then the group's applications.

Namespaces

The tool never creates namespaces automatically — in neither file, and no kubectl is involved. If a namespace is needed, define it explicitly as a manifest, typically in namespaced_manifest:

# <cluster>/<env>/monitoring/namespace.yaml
apiVersion: v1
kind: Namespace
metadata:
  name: monitoring
  labels:
    pod-security.kubernetes.io/enforce: baseline

• kapp applies Namespace objects before namespaced resources within an app, so the namespace can sit in the same app as e.g. its pull secret.
• Order across apps: cluster file → deploy file, and within the deploy file the namespace app → applications. So a namespace defined in either file exists before the deploy file's Helm charts are applied.
• Define each Namespace in only one place (cluster file or deploy file), otherwise both kapp apps claim it.
• A namespace without a manifest must already exist; otherwise the kapp deploy of its resources fails.
• Namespaces previously created via kubectl are adopted by kapp on the first run that contains their Namespace manifest.
• Caution: the kapp app that owns a Namespace deletes it when the app is deleted (kapp delete) or the manifest is removed — and with it everything inside, including resources of other apps.

Path resolution per file

Relative paths inside a deployment file are resolved relative to the folder of that file:

Entry	Cluster file	Deploy file
global_manifest	<cluster>/<env>/<entry>	<deploy>/<env>/<entry>
namespaced_manifest	<cluster>/<env>/<namespace>/<entry>	<deploy>/<env>/<namespace>/<entry>
app manifest	<cluster>/<env>/<namespace>/<entry>	<deploy>/<env>/<namespace>/<entry>
per-namespace values override	<cluster>/<env>/<namespace>/<name>.yaml|.yml	<deploy>/<env>/<namespace>/<name>.yaml|.yml

Caution for the cluster file: every *.yaml/*.yml directly in <cluster>/<env>/ is applied as Helm values to every app (of both files). Put global_manifest files of the cluster file into a subfolder (e.g. manifests/imagePullSecret.yaml), otherwise they are passed to helm template as values too. The deployment JSON file itself and subfolders are never used as values.

Moving namespaces into the cluster file

• kapp ownership is per resource (label kapp.k14s.io/app), not per app name. A manifest must therefore be listed in only one of the two files — the same resource in both would conflict on every run.
• Manifests that move from the deploy file to the cluster file are still owned by <env>-<namespace> on the first run, so the cluster app fails with an ownership conflict. Without downtime:
  1. Deploy the cluster app once with --dangerous-override-ownership-of-existing-resources (the new app takes over the resources).
  2. If the deploy file has no global_manifest/namespaced_manifest left for that namespace, delete the old, now empty app: kapp delete -a <env>-<namespace> --app-namespace gitops (it owns nothing anymore, so nothing is removed). Otherwise the next deploy-file run simply drops the moved resources from it.
• Manifests that stay in the deploy file need no action.

---

Responsibilities (who maintains what)

When cluster/, apps/ and deploy/ are maintained by different teams, use this split:

Folder	Team	Content
cluster/	Platform / Ops	Namespaces, RBAC (e.g. the user-management chart and group bindings), pull secrets, NetworkPolicies, quotas, cluster-wide values (registry, StorageClass, IngressClass, base domain)
apps/	Application / Dev	Env-independent values per app: image repository, resource defaults, probes, feature flags
deploy/	Release / environment operations	Which app runs with which chart version in which namespace, environment-specific overrides (replicas, hostnames, secret references), app-specific manifests

Details per file:

File	Team	Content
cluster/default.yaml|.yml	Platform	Values for every app in every environment
cluster/<env>/*.yaml|.yml	Platform	Values for every app in this environment
cluster/<env>/_deployment.json	Platform	All Namespace objects, namespace-wide manifests, cluster-wide apps (RBAC)
cluster/<env>/<namespace>/*	Platform	Manifests and value overrides for the cluster file
apps/default.yaml|.yml	–	Overlaps with cluster/default.yaml; recommended to leave it out and use cluster/default.yaml instead
apps/<name>.yaml|.yml	App team	Values of one app, shared across all environments
deploy/<env>/_deployment.json	Release	applications and app-specific manifests — no Namespace objects
deploy/<env>/<namespace>/*	Release	Per-environment value overrides and manifests of the apps

Precedence: later values files win — deploy overrides apps, apps overrides cluster. The platform team therefore only sets defaults and cannot enforce values. Settings that must be enforced (e.g. security contexts) belong in an admission policy (Kyverno, OPA Gatekeeper, …), not in values.

Rules to avoid conflicts:

• Define every Namespace only in the cluster file, so each namespace has exactly one owner.
• Keys in cluster/default.yaml and cluster/<env>/*.yaml reach every chart. Agree on a convention (e.g. only global.*) so they do not collide with chart-specific keys.
• RoleBindings in app namespaces belong to the platform team (cluster file); their target namespace must be defined there as well.
• Roll out in two stages: the platform team runs ./deploy -o cluster, then the release team runs ./deploy -o deploy.
• Enforce the ownership in the repository via CODEOWNERS, e.g.:

/cluster/  @platform-team
/apps/     @app-team
/deploy/   @release-team

• Cluster permissions: only the identity running -o cluster needs cluster-wide rights (Namespaces, ClusterRoles, bind/escalate). The identity running -o deploy only needs rights in the app namespaces plus the gitops namespace (kapp bookkeeping). Note: kapp lists resources cluster-wide by default; with namespace-only rights it may need read access to cluster-scoped resources or kapp's --dangerous-scope-to-fallback-allowed-namespaces — verify this before restricting the identity.

---

Env files (--vars)

--vars takes a comma-separated list of plain KEY=VALUE files and loads them into the tool's own process environment before anything else runs — before namespace creation, before helm template, before vals eval.

Override order: files are loaded left to right. If the same key appears in more than one file, the last file listed wins. Given:

# common.env
VAULT_ADDR=https://vault.internal:8200
REGION=eu

# production.env
REGION=eu-prod

--vars common.env,production.env results in REGION=eu-prod and VAULT_ADDR from common.env.

File format:

• One assignment per line: KEY=VALUE
• Blank lines and lines starting with # are ignored
• An optional leading export is stripped
• A value may be wrapped in matching single or double quotes — the quotes are stripped
• Everything after the first = is the value (values containing = are preserved)
• A line without =, or with an empty key, is a hard error

Scope: the variables only exist in the ./deploy process and are inherited by every command it execs (helm, vals, kapp). They are not visible in the calling shell and nothing is written to disk.

---

Directory structure

.
├── main.go
├── cluster/                       # overridable via --cluster
│   ├── default.yaml|.yml          # Global cluster values (applies to every env)
│   ├── staging/
│   │   ├── *.yaml|*.yml           # Environment-wide values, applied to every app
│   │   ├── _deployment.json       # Cluster-level deployment file (processed FIRST)
│   │   ├── manifests/             # global_manifest entries of the cluster file
│   │   └── <namespace>/
│   │       ├── <name>.yaml|.yml   # Per-app override for cluster-file apps
│   │       └── <manifest>.yaml    # namespaced_manifest / app manifest entries of the cluster file
│   └── production/
│       └── ...
├── apps/                          # overridable via --apps (no automatic fallback)
│   ├── default.yaml|.yml          # Applied to every application
│   ├── prometheus.yaml|.yml       # Applied only to the app named "prometheus"
│   └── prometheus-common.yaml|.yml # Shared file, used via the `base` field
└── deploy/                        # overridable via --deploy
    ├── staging/
    │   ├── _deployment.json       # Environment-level deployment file (processed SECOND)
    │   ├── manifests/             # global_manifest entries of the deploy file
    │   └── <namespace>/
    │       ├── <name>.yaml|.yml   # Per-app override for deploy-file apps
    │       └── <manifest>.yaml    # namespaced_manifest / app manifest entries of the deploy file
    └── production/
        └── ...

global_manifest entries are only taken from what the deployment file lists — no folder is scanned automatically. The manifests/ subfolder is a convention for both files: in cluster/<env>/ it is required in practice (files directly there are used as values for every app), in deploy/<env>/ it simply keeps the layout identical.

Important asymmetry, by design: cluster/<env>/ is a folder listing — every *.yaml/*.yml file directly inside it is applied to every app in that environment (of both deployment files). apps/<name> is the opposite: an exact single-file match (apps/<name>.yaml and/or .yml), never a folder listing. Do not create an apps/<some-name>/ folder to group several apps' files — it is ignored. This exists because a folder sharing one app's exact name once caused a real incident: another app's file got merged in and the wrong container image was deployed.

Values files are loaded in this order (later files take precedence):

1. <cluster>/default.yaml|.yml — global cluster values
2. <cluster>/<env>/*.yaml and *.yml — environment-wide values, alphabetically
3. <apps>/default.yaml|.yml — defaults for every application
4. <apps>/<name>.yaml|.yml — this app's own file (or <apps>/<base>.yaml|.yml if base is set)
5. <deploy>/<env>/<namespace>/<name>.yaml|.yml — per-app, per-namespace override (for apps of the cluster file: <cluster>/<env>/<namespace>/<name>.yaml|.yml)

Migrating

• Namespaces are no longer created automatically (no kubectl). Add a Namespace manifest (e.g. namespace.yaml in namespaced_manifest) for every namespace the tool should manage; existing namespaces are adopted by kapp. See Namespaces.
• _deployment_*.json are no longer auto-discovered. Either merge their namespace groups into _deployment.json, or call them explicitly: ./deploy -f _deployment_monitoring.json. With -f, the same name is also looked up in <cluster>/<env>/.
• global_manifest is now resolved relative to the folder of the deployment file (<cluster>/<env>/ or <deploy>/<env>/), no longer relative to <cluster>/. For an env-independent file use ../, e.g. "global_manifest": ["../manifests/imagePullSecret.yaml"] → <cluster>/manifests/imagePullSecret.yaml (from the cluster file).
• Deploy file: the namespace-manifests app was renamed from <env>-<namespace>-namespace-manifests to <env>-<namespace>. Existing resources are still owned by the old app, so the first run fails with an ownership conflict. Migrate once per namespace: deploy the new app with --dangerous-override-ownership-of-existing-resources, then kapp delete -a <env>-<namespace>-namespace-manifests --app-namespace gitops (the old app owns nothing anymore, so nothing is removed).
• <cluster>/cluster.yaml|.yml was renamed to <cluster>/default.yaml|.yml (same naming as <apps>/default.yaml|.yml). There is no fallback: rename the file, otherwise its values are no longer applied.
• The cluster file now allows applications (e.g. RBAC charts). Their kapp app ID is cluster-<env>-<namespace>-<name>. Existing cluster files keep working unchanged.
• New flag --only / -o (cluster or deploy) to roll out in two stages.
• Namespaces and namespace-wide manifests can be moved into <cluster>/<env>/_deployment.json. Their kapp app ID is cluster-<env>-<namespace> — see Moving namespaces into the cluster file.
• Cluster values live under cluster/ (overridable with --cluster): move ./base/cluster.yml to ./cluster/default.yaml and environment-specific overrides into ./cluster/<env>/.
• The base/ folder is now called apps/ by default. There is no automatic fallback: pass --apps base explicitly or rename the folder.
• Per-chart values are a single, flat file per app directly under apps/.
• Environment folders have moved into a deploy/ folder (overridable with --deploy).
• manifest accepts a single string or an array.
• Secrets/config previously exported manually can be passed via --vars.

---

Deployment file format

{
  "global_manifest": ["manifests/imagePullSecret.yaml"],
  "app_k8s_deploy_helm_charts": [
    {
      "namespace": "monitoring",
      "namespaced_manifest": ["monitoring-network-policy.yaml"],
      "applications": [
        {
          "name": "kube-prometheus-stack",
          "chart_ref": "prometheus-community/kube-prometheus-stack",
          "state": "present",
          "version": "58.0.0",
          "deploy_crds": true
        },
        {
          "name": "alertmanager",
          "chart_ref": "prometheus-community/alertmanager",
          "base": "prometheus-common",
          "state": "present"
        },
        {
          "name": "gateway-api",
          "chart_ref": "oci://reg.example.com/charts/gateway-api",
          "deploy_crds": true,
          "manifest": "kapp-config.yaml"
        },
        {
          "name": "gateway-api-crds",
          "state": "present",
          "manifest": [
            "my-configmap.yaml",
            "https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.2.0/standard-install.yaml"
          ]
        },
        {
          "name": "old-app",
          "chart_ref": "my-repo/old-app",
          "state": "absent"
        }
      ]
    }
  ]
}

This format applies to both the deploy file and the cluster file (see Cluster file). Unknown keys (e.g. a typo like manfest) are a hard parse error.

Application fields

Field	Type	Required	Description
name	string	✅	Release name and part of the kapp app ID
state	string	✅	present to deploy, absent to delete
chart_ref	string	❌	Helm chart reference. If omitted, Helm is skipped
repo	string	❌	HTTP Helm repo URL (chart_ref is then the chart name)
version	string	❌	Helm chart version. Omit to use latest
manifest	string or array	❌	One or more raw manifests: local path (relative to <file-dir>/<namespace>/) or http(s):// URL. Applied in order
deploy_crds	bool	❌	Pass --include-crds to helm template (default: false)
plain_http	bool	❌	Pass --plain-http to helm template for registries without TLS (default: false)
base	string	❌	Load `/.yaml

manifest field

manifest accepts either a single string or a list:

{ "name": "gateway-api", "manifest": "kapp-config.yaml" }

{ "name": "gateway-api", "manifest": ["kapp-config.yaml", "extra-rbac.yaml"] }

Each entry is either:

• a local path, resolved as <file-dir>/<namespace>/<entry> — must exist, or the deploy fails with manifest file not found: ..., or
• an http:// / https:// URL, downloaded directly (also during --dry-run).

All manifests are piped through vals eval and merged into the same kapp deploy. Every chunk is separated with --- automatically.

base field

By default the tool loads <apps>/<name>.yaml|.yml. Set base to use a different file:

{ "name": "alertmanager", "base": "prometheus-common" }

apps/default.yaml / apps/default.yml

Applied to every application, before the per-chart file and the per-namespace override.

global_manifest / namespaced_manifest

Raw manifests per namespace, independent of any single application — e.g. an image-pull secret, a default NetworkPolicy or a ResourceQuota.

Field	Scope	Resolved relative to
global_manifest	top-level, applied to every namespace in that file	<file-dir>/<entry>
namespaced_manifest	per namespace group, only that namespace	<file-dir>/<namespace>/<entry>

Deploy file: for a given namespace, both lists are combined into one additional kapp app (<env>-<namespace>), deployed before that namespace's applications. If neither list has entries, no extra app is created.

Cluster file: both lists go into cluster-<env>-<namespace> (only if at least one has entries), deployed before that namespace's applications.

global_manifest only applies to the namespaces of the file it is defined in. If the cluster file and the deploy file both need e.g. the image-pull secret, list it in both.

<file-dir> is <cluster>/<env> for the cluster file and <deploy>/<env> for the deploy file — the same base as namespaced_manifest, just without the <namespace>/ subfolder.

Image-pull secret example — <file-dir>/manifests/imagePullSecret.yaml, e.g. cluster/staging/manifests/imagePullSecret.yaml for the cluster file:

apiVersion: v1
kind: Secret
metadata:
  name: regcred
type: kubernetes.io/dockerconfigjson
data:
  .dockerconfigjson: ref+vault://secret/registry#/dockerconfigjson

{ "global_manifest": ["manifests/imagePullSecret.yaml"], "app_k8s_deploy_helm_charts": [ /* ... */ ] }

Optional: to share one file across all environments, put it into <cluster>/manifests/ and reference it with ../manifests/imagePullSecret.yaml from cluster/<env>/_deployment.json.

Behaviour matrix

chart_ref	manifest	Result
✅	(empty/omitted)	Helm chart rendered and deployed via kapp
✅	✅	Helm chart + all manifests merged into one kapp deploy
❌	✅	Only the raw manifests are deployed via kapp
❌	(empty/omitted)	Entry is skipped with a warning

---

kapp namespaces

Naming note: app IDs are joined with -. A namespace named like <namespace>-<app> of another group (e.g. namespace monitoring-stack vs. app stack in namespace monitoring) would produce the same ID <env>-monitoring-stack. Avoid such combinations.

• -n <namespace> is always the real target namespace of the group — it is the default namespace for resources without their own metadata.namespace (e.g. raw manifests, global_manifest).
• --app-namespace gitops is fixed: kapp's own bookkeeping for every app (cluster and deploy file) is stored in gitops, so kapp ls -n gitops shows everything in one place.

---

Dry-run mode

--dry-run treats rendering and cluster-mutating steps differently:

1. Rendering (helm template, vals eval, reading/downloading manifests) – runs for real, since the diff is based on it.
2. Namespaces – no separate step; explicitly defined Namespace manifests are part of the kapp diff.
3. kapp deploy – runs for real as kapp deploy --diff-run --diff-changes (read-only), showing the real diff.
4. kapp delete (state: absent) – only printed.

This applies to both the cluster file and the deploy file, in the same order as a real run (or only to the file selected via --only). --vars is loaded in both modes.

Note: in a dry-run nothing is applied, so a namespace that is only defined in this run does not exist yet when later apps are diffed. Their diff may then report a non-zero exit, which is shown as a [WARN] and does not abort the dry-run.

---

vals secret resolution

All YAML passed to kapp runs through vals eval -f - first, resolving references such as:

password: ref+vault://secret/myapp#/password
token: ref+awssecrets://myapp/token
var: ref+envsubst://$VAR1

vals reads its backend configuration (e.g. VAULT_ADDR, VAULT_TOKEN) from the environment — which is what --vars is for.

See the vals documentation for supported backends.

---

CRD deployment

Setting deploy_crds: true passes --include-crds to helm template.

Note: For reliable CRD deployment it is recommended to helm pull --untar the chart and apply the crds/ directory separately (e.g. as an app in the cluster-level deployment file, which runs first — or roll out with -o cluster before -o deploy).