Hardening CI Workloads with Kubernetes User Namespaces in the GitLab Runner Helm Chart

Container isolation is a fundamental part of running CI workloads securely. But running a container as root inside a Kubernetes pod does not necessarily mean that its root identity is isolated from the host in the way operators expect.

Kubernetes User Namespaces help address this by mapping user IDs inside a pod to different user IDs on the host. With hostUsers: false, containers can run as root inside their user namespace while that identity maps to an unprivileged identity on the host node.

The GitLab Runner binary already supported the Kubernetes executor settings host_users and host_users_overwrite_allowed. However, the official GitLab Runner Helm chart did not expose corresponding configuration values.

In GitLab Runner Helm Chart MR !565, addressing issue #440, I added Helm chart support for configuring Kubernetes User Namespaces for both the Runner Manager pod and the CI job pods it creates.

The Gap: Runner Support Without Helm Chart Configuration

The Runner binary already understood these settings under [runners.kubernetes]:

  • host_users
  • host_users_overwrite_allowed

But users deploying GitLab Runner through the official Helm chart did not have a direct way to configure these options through chart values.

This created two configuration requirements:

  1. Configure hostUsers on the Runner Manager pod itself.
  2. Pass the Kubernetes executor settings into the Runner container so that spawned CI job pods can use the same user namespace configuration.

The chart needed to expose both paths without changing the default behavior for existing installations.

1. The Helm Template Challenge: Explicit false Is a Valid Value

One of the most important implementation details was handling Boolean values correctly in Helm templates.

Consider a conventional conditional:

{{- if .Values.hostUsers }} hostUsers: {{ .Values.hostUsers }} {{- end }} 

This looks reasonable, but it has a problem: Helm treats false as a falsy value.

If an operator explicitly configures:

hostUsers: false 

the conditional does not render the field.

That is not the desired behavior. There is an important difference between:

  • A value that has not been configured.
  • A value explicitly configured as false.

For Kubernetes security settings, silently dropping an explicit Boolean value can produce a deployment different from the one an operator intended.

Checking whether the key exists

The template needs to check whether the value was supplied, rather than whether the value evaluates to true.

The implementation uses:

{{- if and (hasKey .Values "hostUsers") (not (kindIs "invalid" .Values.hostUsers)) }} hostUsers: {{ .Values.hostUsers }} {{- end }} 

This handles the relevant cases:

Helm value Template behavior
Key absent Field omitted
hostUsers: null Field omitted
hostUsers: false Renders hostUsers: false
hostUsers: true Renders hostUsers: true

The combination of hasKey and kindIs distinguishes an absent or null value from an explicitly configured Boolean.

This is a useful Helm templating pattern whenever a chart needs to preserve explicit false while still allowing a value to remain unset.

2. Configure the Runner Manager Pod

The first configuration path is in:

templates/deployment.yaml

The top-level chart value:

hostUsers: false 

controls the spec.hostUsers field on the Runner Manager pod.

This is important because the Runner Manager itself is a Kubernetes workload. Its pod-level security configuration should be independently configurable through the chart.

The template renders the field only when the value is explicitly configured and non-null.

That preserves the chart's existing default behavior while allowing operators to opt into user namespace isolation.

3. Configure Kubernetes Executor Job Pods

Configuring the Runner Manager pod is only part of the solution.

The Runner Manager creates CI job pods through the Kubernetes executor. Those pods need their own executor configuration.

The chart therefore also exposes:

  • runners.hostUsers
  • runners.hostUsersOverwriteAllowed

These values are handled in:

templates/_env_vars.tpl

They are translated into the corresponding Runner environment variables:

KUBERNETES_HOST_USERS KUBERNETES_HOST_USERS_OVERWRITE_ALLOWED 

This allows the Runner process to receive the Kubernetes executor configuration through its environment.

The two configuration paths serve different purposes:

Configuration Target Purpose
Top-level hostUsers Runner Manager pod Controls spec.hostUsers
runners.hostUsers Kubernetes executor Configures user namespace behavior for job pods
runners.hostUsersOverwriteAllowed Kubernetes executor Controls whether job-level configuration may override the setting

Keeping these paths separate makes the chart configuration explicit: one value controls the Manager pod, while the nested Runner settings configure the executor that creates CI job pods.

4. Validate Boolean Serialization with Helm Unit Tests

Helm templates can look correct while still producing unexpected YAML or environment variable values.

To protect the new behavior, I added nine test cases in:

tests/deployment_test.yaml

The tests cover two important areas.

Pod specification rendering

The tests verify that:

  • spec.hostUsers is omitted when the value is unset.
  • spec.hostUsers renders as false when explicitly configured as false.
  • spec.hostUsers renders as true when explicitly configured as true.

Runner environment variables

The tests also verify that:

  • KUBERNETES_HOST_USERS is omitted by default.
  • KUBERNETES_HOST_USERS_OVERWRITE_ALLOWED is omitted by default.
  • Explicit Boolean values are serialized as the expected strings, "false" and "true".

This distinction matters because Kubernetes pod specifications use Boolean values, while container environment variables are strings.

The tests therefore validate both the generated pod configuration and the environment variables passed to the Runner process.

5. Why This Matters for CI Security

Kubernetes User Namespaces provide an additional isolation boundary between identities inside a pod and identities on the host.

Without user namespace isolation, a process running as UID 0 inside a container has a root identity within the container's user namespace, which can carry security implications if isolation boundaries are compromised.

With hostUsers: false, Kubernetes can map the pod's user namespace to a separate range of host UIDs and GIDs. Container root is therefore not equivalent to host root.

For CI environments, this is particularly relevant because job containers execute project-controlled build scripts and dependencies.

Exposing the setting through the Helm chart makes the capability easier to configure in real deployments instead of requiring users to work around the chart's missing values.

Kubernetes version and cluster configuration still matter: user namespace support must be available in the cluster, and the relevant runtime and workload configuration must support it.

Engineering Takeaways

Do not use truthiness when the distinction between unset and false matters. Helm conditionals can accidentally remove explicitly configured Boolean values.

Separate Manager configuration from executor configuration. The Runner Manager pod and the CI job pods it creates are different workloads with different configuration paths.

Environment variables require explicit serialization. Kubernetes pod fields use Boolean values, while environment variables represent those values as strings.

Test rendered output, not just template logic. Helm unit tests help catch missing fields, incorrect Boolean handling, and unexpected environment variable output.

Expose existing binary capabilities through deployment tooling. A feature supported by the application is not fully accessible to users if the official deployment chart cannot configure it.

Conclusion

GitLab Runner already supported Kubernetes executor user namespace settings, but the official Helm chart did not expose the configuration needed to use them conveniently.

MR !565 closed that gap by adding hostUsers support for the Runner Manager pod and environment-variable configuration for Kubernetes executor job pods.

The most subtle part was preserving explicit Boolean false values in Helm templates. By checking key existence and value type rather than relying on truthiness, the implementation supports unset, null, false, and true configurations correctly.

This change demonstrates how deployment tooling contributes to application security: exposing the right Kubernetes capabilities, preserving configuration semantics, and testing the rendered result are all part of making secure CI workloads practical to operate.

References

Author: Manish Kumar

GitLab: @manishpatel00