Skip to main content
Star us on GitHub Star

Install the Controller in Kubernetes


Version: 1.0.8 Type: application AppVersion: 1.1.3

Host an OpenZiti controller in Kubernetes




This chart runs a Ziti controller in Kubernetes. It uses the custom resources provided by cert-manager and trust-manager, i.e., Issuer, Certificate, and Bundle. Delete the controller pod after an upgrade for the new controller configuration to take effect.

Here's a checklist that must be satisfied for each of the controller's TLS servers: router control plane (ctrlPlane), each web listener(s) (clientApi, at least).

  1. Each of the controller's TLS servers must have an advertised host (FQDN) and TCP port. The chart ensures the DNS SAN is added for this address. The port is always 443 if ingress is enabled for ClusterIP service.
  2. A K8s Service must be configured and reachable by all clients. (i.e., type ClusterIP+Ingress, NodePort, or LoadBalancer)
  3. Each K8s Service must be configured for TLS passthrough (i.e., server TLS must not be terminated by an intermediary LB).
  4. All clients must access the controller through the advertised address(es).


Add the OpenZiti Charts Repo to Helm

helm repo add openziti

This chart requires Certificate, Issuer, and Bundle resources to be applied before installing the chart. Sub-charts cert-manager, and trust-manager will be installed automatically. You may disable the sub-charts if you wish to provide these resources separately, but if you do so then please use the sub-chart values at the foot of Values.yaml to ensure those charts are correctly configured.

Install Required Custom Resource Definitions

This step satisfies Helm's requirement that the CRDs used in the umbrella chart already exist in Kubernetes before installing the controller chart.

kubectl apply -f
kubectl apply -f

Minimal Installation

This first example shows a minimal installation for a Kubernetes distribution that provides TLS pass-through for Service type LoadBalancer, e.g., k3s, k3d, Minikube. This is useful for environments where there's no cost, or justifiable cost, associated with provisioning a LoadBalancer with TLS passthrough.

You must supply one value when you install the chart.

clientApi.advertisedHoststringnilthe DNS name that edge clients and routers will resolve to reach this controller's edge client API
clientApi.advertisedPortstringnilthe TCP port associated with the advertisedHost to advertise to edge clients and routers
helm install \
--namespace ziti-controller ziti-controller-minimal1 \
openziti/ziti-controller \
--set clientApi.advertisedHost="" \
--set clientApi.advertisedPort="443"

A default admin user and password will be generated and saved to a secret during installation. The credentials can be retrieved using this command:

kubectl get secret \
-n ziti-controller ziti-controller-minimal1-admin-secret \
-o go-template='{{range $k,$v := .data}}{{printf "%s: " $k}}{{if not $v}}{{$v}}{{else}}{{$v | base64decode}}{{end}}{{"\n"}}{{end}}'

You may log in the ziti CLI with one command or omit the -p part to prompt:

ziti edge login \
--yes \
--username admin \
--password $(
kubectl -n ziti-controller \
get secrets ziti-controller-minimal1-admin-secret \
-o go-template='{{index .data "admin-password" | base64decode }}'

Using ClusterIP Services with an Ingress Controller

The default K8s service type for this chart is ClusterIP. You can publish these cluster-internal services with an Ingress resource. You need an Ingress Controller. Here's an example of using the community ingress-nginx chart to provision ingresses for the controller's ClusterIP services.

Ensure you have the ingress-nginx chart installed with controller.extraArgs.enable-ssl-passthrough=true. You can verify this feature is enabled by running kubectl describe pods {ingress-nginx-controller pod} and checking the args for --enable-ssl-passthrough=true.

If necessary, patch the ingress-nginx deployment to enable TLS passthrough.

kubectl patch deployment "ingress-nginx-controller" \
--namespace ingress-nginx \
--type json \
--patch '[{"op": "add",
"path": "/spec/template/spec/containers/0/args/-",

Create a Helm chart values file like this.

# /tmp/controller-values.yml
advertisedPort: 443
type: ClusterIP
enabled: true
ingressClassName: nginx
annotations: "false" "true"

Now install or upgrade this controller chart with your values file.

helm install \
--namespace ziti-controller ziti-controller-managed1 \
openziti/ziti-controller \
--values /tmp/controller-values.yml

Expose the Router Control Plane

This is applicable if you have any routers outside the Ziti controller's cluster. You must configure pass-through TLS LoadBalancer or Ingress for the control plane service. Routers running in the same cluster as the controller can use the cluster service named {controller release}-ctrl (the "ctrl" endpoint). This example demonstrates a pass-through Ingress resource for nginx-ingress.

Merge this with your Helm chart values file before installing or upgrading.

advertisedPort: 443
enabled: true
enabled: true
ingressClassName: nginx
annotations: "false" "true"

Extra Security for the Management API

You can split the client and management APIs into separate cluster services by setting managementApi.service.enabled=true. With this configuration, you'll have an additional cluster service named {controller release}-mgmt that is the management API, and the client API will not have management features.

This Helm chart's values allow for both operational scenarios: combined and split. The default choice is to expose the combined client and management APIs as the cluster service named {controller release}-client, which is convenient because you can use the ziti CLI immediately. For additional security, you may shelter the management API by splitting these two sets of features, exposing them as separate API servers. After the split, you can access the management API in several ways:

  • running the web console inside the cluster,
  • hosting a Ziti service, or
  • kubectl port-forward.

Advanced PKI

The default configuration generates a singular PKI root of trust for all the controller's servers and the edge signer CA. Optionally, you may provide the name of a cert-manager Issuer or ClusterIssuer to become the root of trust for the Ziti controller's identity.

Merge this with your Helm chart values file before installing or upgrading.

kind: ClusterIssuer
name: my-alternative-cluster-issuer

You may also configure the Ziti controller to use separate PKI roots of trust for its three main identities: control plane, edge signer, and web bindings.

For example, to use a separate CA for the edge signer function, merge this with your Helm chart values file before installing or upgrading.

enabled: true

Prometheus Monitoring

This chart provides a default disabled ziti-controller-prometheus k8s service for prometheus, which can be enabled with prometheus.service.enabled. Enabling it will create a prometheus ServiceMonitor for configuring the prometheus endpoint. It is also important that you enable for getting a full set of metrics.

For more information, please check here.

Values Reference

additionalConfigsobject{}allow the insertion of raw/unmodified config blocks Additional configs will be appended to the top-level keys of the ziti controller config ( events, edge, web, network, ctrl) These configs allow for OVERRIDING behavior in any top level config block
additionalVolumeslist[]additional volumes to mount to ziti-controller container
affinityobject{}deployment template spec affinity
ca.clusterDomainstring"cluster.local"Set a custom cluster domain if other than cluster.local
ca.durationstring"87840h"Go time.Duration string format
ca.renewBeforestring"720h"Go time.Duration string format
cert-manager.enableCertificateOwnerRefbooltrueclean up secret when certificate is deleted
cert-manager.enabledboolfalseinstall the cert-manager subchart
cert-manager.installCRDsboolfalseCRDs must be applied in advance of installing the parent chart
cert.durationstring"87840h"server certificate duration as Go time.Duration string format
cert.renewBeforestring"720h"rewnew server certificates before expiry as Go time.Duration string format
clientApi.advertisedHoststringnilglobal DNS name by which routers can resolve a reachable IP for this service
clientApi.advertisedPortint443cluster service, node port, load balancer, and ingress port
clientApi.containerPortint1280cluster service target port on the container
clientApi.dnsNameslist[]additional DNS SANs
clientApi.ingress.annotationsstringnilingress annotations, e.g., to configure ingress-nginx
clientApi.ingress.enabledboolfalsecreate an ingress for the cluster service
clientApi.service.enabledbooltruecreate a cluster service for the deployment
clientApi.service.typestring"LoadBalancer"expose the service as a ClusterIP, NodePort, or LoadBalancer
ctrlPlane.advertisedHoststringnilglobal DNS name by which routers can resolve a reachable IP for this service: default is cluster service DNS name which assumes all routers are inside the same cluster
ctrlPlane.advertisedPortint443cluster service, node port, load balancer, and ingress port
ctrlPlane.alternativeIssuerstringnilkind and name of alternative issuer for the controller's identity
ctrlPlane.containerPortint6262cluster service target port on the container
ctrlPlane.dnsNameslist[]additional DNS SANs
ctrlPlane.ingress.annotationsstringnilingress annotations, e.g., to configure ingress-nginx
ctrlPlane.ingress.enabledboolfalsecreate an ingress for the cluster service
ctrlPlane.service.enabledbooltruecreate a cluster service for the deployment
ctrlPlane.service.typestring"ClusterIP"expose the service as a ClusterIP, NodePort, or LoadBalancer
ctrlPlaneCasBundle.namespaceSelectorobject{}namespaces where trust-manager will create the Bundle resource containing Ziti's trusted CA certs (default: empty means all namespaces)
dbFilestring"ctrl.db"name of the BoltDB file
edgeSignerPki.admin_client_cert.durationstring"8760h"admin client certificate duration as Go time.Duration
edgeSignerPki.admin_client_cert.renewBeforestring"720h"renew admin client certificate before expiry as Go time.Duration
edgeSignerPki.enabledbooltruegenerate a separate PKI root of trust for the edge signer CA
envstringnilset name to value in containers' environment fabric event logger and file handler"fabric-events.json""/var/run/ziti""5s"matching interval age and reporting interval ensures coherent metrics from fabric events"5s"matching interval age and reporting interval ensures coherent metrics from fabric events[0].typestring"fabric.circuits"[1].typestring"fabric.links"[2].typestring"fabric.routers"[3].typestring"fabric.terminators"[4].metricFilterstring".*"[4].sourceFilterstring".*"[4].typestring"metrics"[5].typestring"edge.sessions"[6].typestring"edge.apiSessions"[7].typestring"fabric.usage"[7].versionint3[8].typestring"services"[9].intervalstring"5s"[9].typestring"edge.entityCounts"
highAvailability.modestring"standalone"Ziti controller HA mode
highAvailability.replicasint1Ziti controller HA swarm replicas
image.additionalArgslist[]additional arguments can be passed directly to the container to modify ziti runtime arguments
image.argslist["{{ include \"configMountDir\" . }}/ziti-controller.yaml"]args for the entrypoint command
image.commandlist["ziti","controller","run"]container entrypoint command
image.homeDirstring"/home/ziggy"homeDir for admin login shell must align with container image's ~/.bashrc for ziti CLI auto-complete to work
image.pullPolicystring"IfNotPresent"deployment image pull policy
image.repositorystring""container image repository for app deployment
image.tagstring""override the container image tag specified in the chart
ingress-nginx.controller.extraArgs.enable-ssl-passthroughstring"true"configure subchart ingress-nginx to enable the pass-through TLS feature
ingress-nginx.enabledboolfalseinstall the ingress-nginx subchart
managementApi.advertisedHoststringnilglobal DNS name by which routers can resolve a reachable IP for this service
managementApi.advertisedPortint443cluster service, node port, load balancer, and ingress port
managementApi.containerPortint1281cluster service target port on the container
managementApi.dnsNameslist[]additional DNS SANs
managementApi.ingress.annotationsstringnilingress annotations, e.g., to configure ingress-nginx
managementApi.ingress.enabledboolfalsecreate an ingress for the cluster service
managementApi.service.enabledboolfalsecreate a cluster service for the deployment
managementApi.service.typestring"ClusterIP"expose the service as a ClusterIP, NodePort, or LoadBalancer
network.createCircuitRetriesint2createCircuitRetries controls the number of retries that will be attempted to create a path (and terminate it) for new circuits.
network.cycleSecondsint15Defines the period that the controller re-evaluates the performance of all of the circuits running on the network.
network.initialLinkLatencystring"65s"Sets the latency of link when it's first created. Will be overwritten as soon as latency from the link is actually reported from the routers. Defaults to 65 seconds.
network.minRouterCostint10Sets router minimum cost. Defaults to 10
network.pendingLinkTimeoutSecondsint10pendingLinkTimeoutSeconds controls how long we'll wait before creating a new link between routers where there isn't an established link, but a link request has been sent
network.routeTimeoutSecondsint10routeTimeoutSeconds controls the number of seconds the controller will wait for a route attempt to succeed.
network.routerConnectChurnLimitstring"1m"Sets how often a new control channel connection can take over for a router with an existing control channel connection Defaults to 1 minute the hard upper limit of underperforming circuits that are candidates to be re-routed. If smart routing detects 100 circuits that are underperforming, and smart.rerouteCap is set to 1, and smart.rerouteFraction is set to 0.02, then the upper limit of circuits that will be re-routed in this cycleSeconds period will be limited to 1. the fractional upper limit of underperforming circuits that are candidates to be re-routed. If smart routing detects 100 circuits that are underperforming, and smart.rerouteFraction is set to 0.02, then the upper limit of circuits that will be re-routed in this cycleSeconds period will be limited to 2 (2% of 100).
nodeSelectorobject{}deployment template spec node selector
persistence.VolumeNamestringnilPVC volume name
persistence.accessModestring"ReadWriteOnce"PVC access mode: ReadWriteOnce (concurrent mounts not allowed), ReadWriteMany (concurrent allowed)
persistence.annotationsobject{}annotations for the PVC
persistence.enabledbooltruerequired: place a storage claim for the BoltDB persistent volume
persistence.existingClaimstring""A manually managed Persistent Volume and Claim Requires persistence.enabled=true. If defined, PVC must be created manually before volume will be bound.
persistence.sizestring"2Gi"2GiB is enough for tens of thousands of entities, but feel free to make it larger
persistence.storageClassstringnilStorage class of PV to bind. By default it looks for the default storage class. If the PV uses a different storage class, specify that here.
podAnnotationsobject{}annotations to apply to all pods deployed by this chart
podSecurityContextobject{"fsGroup":2171}deployment template spec security context
podSecurityContext.fsGroupint2171the GID of the group that should own any files created by the container, especially the BoltDB file
prometheus.advertisedHoststring""DNS name to advertise in place of the default internal cluster name built from the Helm release name
prometheus.advertisedPortint443cluster service, node port, load balancer, and ingress port
prometheus.containerPortint9090cluster service target port on the container
prometheus.service.enabledboolfalsecreate a cluster service for the deployment
prometheus.service.labelsobject{"app":"prometheus"}extra labels for matching only this service, ie. serviceMonitor
prometheus.service.typestring"ClusterIP"expose the service as a ClusterIP, NodePort, or LoadBalancer
prometheus.serviceMonitor.annotationsobject{}ServiceMonitor annotations
prometheus.serviceMonitor.enabledbooltrueIf enabled, and prometheus service is enabled, ServiceMonitor resources for Prometheus Operator are created
prometheus.serviceMonitor.intervalstringnilServiceMonitor scrape interval
prometheus.serviceMonitor.labelsobject{}Additional ServiceMonitor labels
prometheus.serviceMonitor.metricRelabelingslist[]ServiceMonitor relabel configs to apply to samples as the last step before ingestion (defines metric_relabel_configs)
prometheus.serviceMonitor.namespacestringnilAlternative namespace for ServiceMonitor resources
prometheus.serviceMonitor.namespaceSelectorobject{}Namespace selector for ServiceMonitor resources
prometheus.serviceMonitor.relabelingslist[]ServiceMonitor relabel configs to apply to samples before scraping (defines relabel_configs)
prometheus.serviceMonitor.schemestring"https"ServiceMonitor will use http by default, but you can pick https as well
prometheus.serviceMonitor.scrapeTimeoutstringnilServiceMonitor scrape timeout in Go duration format (e.g. 15s)
prometheus.serviceMonitor.targetLabelslist[]ServiceMonitor will add labels from the service to the Prometheus metric
prometheus.serviceMonitor.tlsConfigobject{"insecureSkipVerify":true}ServiceMonitor will use these tlsConfig settings to make the health check requests
prometheus.serviceMonitor.tlsConfig.insecureSkipVerifybooltrueset TLS skip verify, because the SAN will not match with the pod IP
resourcesobject{}deployment container resources
securityContextobject{}deployment container security context
spireAgent.enabledboolfalseif you are running a container with the spire-agent binary installed then this will allow you to add the hostpath necessary for connecting to the spire socket
spireAgent.spireSocketMntstring"/run/spire/sockets"file path of the spire socket mount
tolerationslist[]deployment template spec tolerations"{{ .Release.Namespace }}"trust-manager needs to be configured to trust the namespace in which the controller is deployed so that it will create the Bundle resource for the ctrl plane trust bundle
trust-manager.crds.enabledboolfalseCRDs must be applied in advance of installing the parent chart
trust-manager.enabledboolfalseinstall the trust-manager subchart
webBindingPki.enabledbooltruegenerate a separate PKI root of trust for web bindings, i.e., client, management, and prometheus APIs


  • replicas - Each controller replica needs to be it's own HA member. We have to wait until HA is officially released.
  • lower CA / Cert lifetime; how to refresh stuff when Certs are updated?
  • Deploy Prometheus scraper configuration when prometheus.enabled = true
  • cert-manager allows issuing only one cert per key, i.e., ClientCertKeyReuseIssue prevents us from issuing a user cert and server cert backed by same private key, hence the controller config.yaml re-uses server certs in place of user certs to allow startup and testing to continue