Help us learn about your current experience with the documentation. Take the survey.

The GitLabCore reconciler

internal/controller/gitlabcore reconciles the apps.gitlab.com/v2alpha1 GitLabCore resource, the resource Bridge configures a GitLab instance through. For the design of the resource, see ADR 26.

The reconciler is alpha. The controllers and helm packages keep serving the v1beta1 GitLab resource, and the two paths share no code.

Enable the reconciler

The reconciler rides with Bridge and is gated twice over, like the Bridge server itself:

  • The bridge build tag compiles it in. Without the tag, gitlabcore_stub.go takes its place and neither the reconciler nor the v2alpha1 types reach the binary, so public images carry neither.
  • ENABLE_BRIDGE=true registers it at runtime, even in a tagged build. This gate is not only a feature switch: the GitLabCore definition ships in no release, and a watch on a definition the cluster does not serve fails the manager on start.

task install_v2alpha1_crds installs the definitions and grants the manager ServiceAccount what the reconciler needs beyond the chart permissions: gitlabcores, its status and finalizers subresources, and PodDisruptionBudgets. The chart grants the v1beta1 GitLab permissions only, and no PodDisruptionBudget permissions at all, because the v1beta1 controller applies none. The grant goes into a role of its own, so task deploy_operator does not revert it.

Install both and run the Operator against a development cluster:

task install_v2alpha1_crds
ENABLE_BRIDGE=true HELM_CHARTS=$(pwd)/charts task run_bridge
kubectl apply -f config/samples/gitlabcore_v2alpha1.yaml

task run builds without the tag, so the reconciler is absent there. Build an image with task docker-build-bridge, which uses Dockerfile.bridge.

The sample points at no infrastructure. For a resource that is wired to PostgreSQL, Redis, and object storage, run bash scripts/dev_dependencies.sh setup: it provisions them and writes an external-deps-v2alpha1.yaml to apply as it is. For more information, see External dependencies.

What one reconcile does

  1. Reads the capabilities of the cluster with internal/render/capabilities.
  2. Renders the GitLab chart with internal/render, using the release name and namespace of the resource.
  3. Runs the pre-install hooks with internal/render/hookexec, RBAC excluded, and only when the hooks of this release have not run yet.
  4. Applies the rendered objects server-side, definitions and RBAC excluded.
  5. Reports the readiness of the rendered Deployments and StatefulSets through status.conditions.

pre-install is the only hook event that runs. The post-install hooks of the chart belong to the NGINX admission webhook patch and the Traefik dashboard, and the Operator overrides both subcharts off, so no release renders one. A test asserts this against every chart version the Operator carries.

The hooks run once per release, not once per pass. ConditionInitialized carries the generation whose hooks completed, and the generation covers the whole specification, so the hooks run again when an administrator changes the resource and not otherwise. An apply that fails is retried without them.

This matters because of what a rerun costs. The shared secrets Job carries helm.sh/hook-delete-policy: hook-succeeded,before-hook-creation, so running the event again deletes the Job, recreates it, and waits for a pod to complete.

Every successful reconcile asks for the next one, 30 seconds later, so the loop runs for as long as the resource exists. Nothing else triggers it: the reconciler watches the GitLabCore resource and none of the objects it applies. A release is hundreds of objects, of kinds the Operator does not know ahead of time, and a reconcile renders the whole chart, so a watch on them would mean an informer per kind and a full render on every status update they make. An object that is deleted or edited by hand is restored on the next pass instead, within the requeue delay.

The chart comes from the charts directory of the Operator, which HELM_CHARTS points at and the image bakes in. Nothing is pulled over the network, so a spec.chart.version the Operator does not carry is a configuration error. The error names the versions that are available.

The cluster is the only source of capabilities. GITLAB_OPERATOR_KUBERNETES_VERSION and GITLAB_OPERATOR_KUBERNETES_API_VERSIONS configure the frozen renderer of the v1beta1 path and do not reach the reconciler. A render that claims capabilities the cluster does not have produces objects that cannot be applied, and applying them is what this reconciler does with the result.

Every object is applied under the field manager gitlabcore-controller, with ownership forced. A field another manager took is taken back, and a field the chart stops rendering is removed. For the rules a consumer of internal/render follows, see Apply rendered objects.

Effective values

EffectiveValues builds the values in three layers: the values derived from the structured fields, then spec.chart.values merged over them, then the Operator overrides.

FieldChart values
spec.hostnameglobal.hosts.gitlab.name, and global.hosts.domain from the parent domain
spec.license.secretRefglobal.gitlab.license.secret, global.gitlab.license.key

A hostname of gitlab.example.com therefore yields a domain of example.com, and with it the sibling hosts registry.example.com and kas.example.com. An apex hostname is its own domain, because stripping its first label would leave the public suffix.

The derived layer also mirrors the shared secrets defaults of the v1beta1 controller:

DefaultReason
shared-secrets.serviceAccount.create: false, name: $GITLAB_MANAGER_SERVICE_ACCOUNTThe Job runs under the ServiceAccount of the Operator, which the Operator installation provisions.
shared-secrets.rbac.create: falseThe Operator creates no RBAC, and the Job needs none: its account already has it.
shared-secrets.securityContext.runAsUser: "", fsGroup: ""Keeps the Job compatible with the OpenShift nonroot SecurityContextConstraint, which assigns both itself.

That ServiceAccount has to exist in the namespace of the resource, with permission to manage Secrets there, before the first reconcile. The v1beta1 controller reaches the same result by applying only the ConfigMap and the Job of that component.

The free-form values win over the derived ones, as ADR 26 decides, which keeps them a working escape hatch. The Operator overrides win over both:

OverrideReason
installCertmanager: falsecert-manager is a prerequisite of the Operator. The cluster administrator installs it once, and it serves every instance.
gitlab-runner.install: falseThe GitLab Runner has a lifecycle of its own and is deployed through the Runner Operator.
nginx-ingress.enabled: false, nginx-ingress-geo.enabled: false, haproxy.install: false, traefik.install: false, global.gatewayApi.installEnvoy: falseThe Operator installs no networking controllers/Operators.

An override is a setting an instance may not choose, because the Operator, not the chart, owns what it configures. Keep the list short: every entry is a value an administrator sets and does not get.

Status

ConditionMeaning
InitializedThe chart resolved and rendered, and its hooks ran.
AvailableEvery rendered workload has its desired replicas ready.

status.phase reports Preparing, Running, or Failed, and status.version records the chart version that was applied.

The reconciler installs no definitions and no RBAC

Two classes of rendered object are never applied. The cluster administrator provisions both.

  • Definitions. No CustomResourceDefinition is applied, neither the ones the chart renders from templates, such as the Gateway API and Envoy Gateway definitions, nor the ones Result.CRDs carries from the crds/ directories. A definition is cluster-wide and outlives every release that uses it, so installing one from a namespaced resource would let one GitLab instance change an API that another instance, and other operators, depend on.
  • RBAC. Nothing in the rbac.authorization.k8s.io group is applied: no Role, RoleBinding, ClusterRole, or ClusterRoleBinding. RBAC grants permissions, so applying it would turn the right to write a GitLabCore into the right to grant any permission the Operator holds, which is cluster-wide.

An object that needs an API the cluster does not serve fails to apply, with the kind named in the error and in the Available condition. Either have the cluster administrator install that API, or turn the component off in spec.chart.values. The chart routes through the Gateway API by default, so a cluster without the Gateway API and Envoy Gateway needs global.gatewayApi.enabled: false.

A component whose RBAC is missing starts and then fails against the API server, which no reconcile repairs either.

The hooks go through the same filter. The chart declares its RBAC as hooks too, and the Operator has no permission to create it, so a release whose hook RBAC was applied would fail on the first hook:

roles.rbac.authorization.k8s.io "gitlab-shared-secrets" is forbidden: User
"system:serviceaccount:gitlab-system:gitlab-manager" cannot delete resource "roles"

The shared secrets Job does not need that RBAC, because the mirrored defaults run it under the ServiceAccount of the Operator.

Deletion

Objects in the namespace of the resource carry a controller reference to it, so Kubernetes deletes them. The rest cannot be owned, because the API server rejects an owner it cannot resolve: a namespaced resource may own neither a cluster-scoped object nor an object of another namespace. A finalizer deletes those by the release labels internal/render stamps:

operator.gitlab.com/release-name        the name of the resource
operator.gitlab.com/release-namespace   its namespace

The kinds to sweep come from rendering the release once more, because nothing records what was applied. The sweep is best effort and never blocks the deletion: a chart the Operator no longer carries, or values that no longer render, must not leave a resource that cannot be deleted. What is left behind is logged with the label selector that finds it.

Known limits

  • A reconcile blocks while the hooks of an event run. The shared secrets Job dominates the first reconcile of an instance, and its worker is held for that time.
  • Nothing reruns the hooks when their output is gone. Deleting a generated Secret by hand leaves the instance without it until an administrator changes the resource, because only that changes the generation the marker compares against.
  • The loop polls. Drift is repaired within the requeue delay rather than on the event that caused it, and each pass renders the chart and applies every object again, whether or not anything changed.
  • Nothing runs the pre-upgrade or pre-delete hooks, and no upgrade path is modeled: every render is composed as revision 1 of an install.
  • The chart values.schema.json is not validated against the effective values yet. That is the job of the validating webhook ADR 26 describes.

Run the tests

The unit tests are Ginkgo specs that need the chart archive on disk and no cluster:

task retrieve-charts
HELM_CHARTS=$(pwd)/charts CHART_VERSION=$(head -n1 CHART_VERSIONS) \
  task unit-tests TEST_PKGS="./internal/controller/..."

Run the end-to-end test

internal/controller/gitlabcore/e2e_test.go creates a GitLabCore in a throwaway namespace, calls Reconcile the way the manager does, and asserts on what reaches the cluster: the secrets the hooks generated, the owner references and release labels of the applied objects, the status conditions, and the sweep the finalizer performs. It is gated behind the e2e build tag, so the unit tests never pick it up.

Point kubectl at a throwaway cluster and install the definitions first, otherwise the test skips:

task install_v2alpha1_crds
task e2e-tests TEST_PKGS="./internal/controller/..."

The release renders into the test namespace and nowhere else: NGINX, Prometheus, and the Gateway API are off through the values, and cert-manager and the Runner through the Operator overrides, so every object is namespaced and of a built-in kind. The pods never become ready, because there is no PostgreSQL, Redis, or object storage, which the test asserts rather than waits for.

VariableDescription
E2E_CLUSTER_SCOPED=1Enables the NGINX Ingress controller of the chart, so the release also carries RBAC and cluster-scoped objects. The IngressClass and the admission webhook are applied, the RBAC is skipped, and the finalizer sweeps what it applied.
E2E_KEEP_NAMESPACE=1Keeps the namespace and the cluster-scoped objects for inspection.

After a run with E2E_KEEP_NAMESPACE=1, delete the cluster-scoped objects by hand. An admission webhook whose backing service is gone rejects unrelated writes across the whole cluster.