OpenBao chart

  • Tier: Premium, Ultimate
  • Offering: GitLab Self-Managed
  • Status: Beta

The availability of this feature is controlled by a feature flag. For more information, see the history.

You can use the OpenBao chart to install OpenBao, which is required to enable the GitLab secrets manager.

Known limitations

  • You can’t upgrade OpenBao without downtime. Zero downtime upgrades are proposed in issue 595721.
  • You can’t deploy OpenBao with GitLab Operator. Support is proposed in issue 1933.
  • A FIPS variant of the OpenBao image is already being built, but OpenBao is not FIPS validated. FIPS validation is tracked in GitLab issue 574875.
  • When failing over to a Geo secondary site that uses a different domain (instead of updating DNS to point the primary domain to the secondary site), OpenBao requires manual re-provisioning of JWT authentication for all projects and groups where GitLab Secrets Manager is enabled. This process can be time-consuming for large deployments. A migration tool is being proposed in GitLab issue 595722. Until then, the recommended approach is to update DNS records so the primary domain points to the promoted secondary site.

Setup GitLab secret manager and OpenBao

  1. On an existing GitLab instance, enable OpenBao and configure its database connection. OpenBao requires a separate PostgreSQL database that you must create first. For the database and password secret, see Database configuration.

    # Enable OpenBao integration
    global:
      openbao:
        enabled: true
        # Connect to the separate OpenBao database you created
        psql:
          host: "psql.openbao.example.com"
          password:
            secret: openbao-db-password
            key: password
    # Install bundled OpenBao
    openbao:
      install: true
  2. In GitLab, on the top bar, select Search or go to and find your project.

  3. Select Settings > General.

  4. Expand Visibility, project features, permissions.

  5. Turn on the Secrets Manager toggle, and wait for the Secrets Manager to be provisioned.

Rolling back OpenBao upgrades

OpenBao upgrades can make changes to the PostgreSQL data that are not backwards compatible, which can cause compatibility issues if the OpenBao upgrade must be rolled back.

You should always back up before upgrading OpenBao. If you need to roll back an OpenBao upgrade, also restore the database backup matching the OpenBao version.

For more information, see OpenBao upgrade documentation.

Back up OpenBao

A complete OpenBao backup includes the:

  • OpenBao unseal key
  • OpenBao PostgreSQL database

The Toolbox backs up the OpenBao database as part of the standard GitLab backup, when backup credentials are configured.

Back up the OpenBao database at the same time as the main GitLab PostgreSQL database to avoid data inconsistencies.

Restore OpenBao

The Toolbox restores the OpenBao database as part of the standard GitLab restore. See OpenBao database credentials to configure restore credentials.

Before restoring an OpenBao backup:

  1. Note the current replica count so you can restore it afterward.

  2. Scale down OpenBao so the running pod does not race the DROP TABLE and CREATE TABLE statements in the dump:

    kubectl get deploy -lapp.kubernetes.io/name=openbao,app.kubernetes.io/instance=<helm release name> -n <namespace>
    kubectl scale deploy -lapp.kubernetes.io/name=openbao,app.kubernetes.io/instance=<helm release name> -n <namespace> --replicas=0

After the restore completes, scale OpenBao back to its previous replica count so it reads from the restored database:

kubectl scale deploy -lapp.kubernetes.io/name=openbao,app.kubernetes.io/instance=<helm release name> -n <namespace> --replicas=<previous count>

OpenBao configuration options

The following tables list all available OpenBao configuration options. Global chart settings, such as Ingress configuration and image tag suffixes, also apply to OpenBao.

Installation command-line options

The table below contains all the possible charts configurations that can be supplied to the helm install command using the --set flags.

ParameterDefaultDescription
config.logLevelinfoOpenBao log level.
config.logRequestsLeveloffOpenBao request log level. To enable request logging set this to the same value as config.logLevel or a higher level.
config.logFormatjsonOpenBao log format. Either json or standard.
nameOverrideOverride the chart name.
fullnameOverrideOverride the fully qualified app name.
serviceAccount.createtrueCreate a service account for OpenBao.
serviceAccount.automounttrue
serviceAccount.annotations{}Additional service account annotations.
serviceAccount.nameOverride the generated service account name.
role.createCreate a role with necessary RBAC permissions.
securityContext.capabilities{ drop: ["ALL"] }
securityContext.runAsNonRoottrue
securityContext.allowPrivilegeEscalationfalse
securityContext.runAsUser1000
podSecurityContext.seccompProfileRuntimeDefault
podSecurityContext.runAsUser1000
podSecurityContext.fsGroup1000
serviceActive.typeClusterIPService type of the active OpenBao pod.
serviceActive.annotations{}Service annotations of the active OpenBao pod.
serviceInactive.typeClusterIPService type of the standby OpenBao pods.
serviceInactive.annotations{}Service annotations of the standby OpenBao pods.
resources{}Resource limits and requests.
autoscaling.minReplicas2Minimum OpenBao replicas.
autoscaling.maxReplicas2Maximum OpenBao replicas.
autoscaling.targetCPUUtilizationPercentage80Target CPU utilization for autoscaling.
autoscaling.targetCPUMemoryPercentageTarget memory utilization for autoscaling.
livenessProbeOpenBao liveness probe. Check OpenBao values for the default.
readinessProbeOpenBao readiness probe. Check OpenBao values for the default.
nodeSelector{}Node selector labels.
tolerations[]Toleration labels for pod assignment.
affinity{}Affinity labels for pod assignment.
podAnnotations{}Annotations to add to the OpenBao pods.
podLabels{}Labels to add to the OpenBao pods.
extraVolumesAdditional volumes for the OpenBao pods.
extraVolumeMountsAdditional volume mounts for the OpenBao container.
config.uifalseEnable the OpenBao UI.
config.clusterPort8201OpenBao cluster port.
config.apiPort8200OpenBao API port.
config.cacheSize2560Size of the read cache used by the physical storage subsystem as a number of entries.
config.maxRequestSize786432Maximum request size in bytes. Default is 768KB.
config.maxRequestJsonMemory1048576Maximum size of the JSON-parsed request body in bytes. Default is 1MB.

Container image options

The OpenBao chart deploys a cloud-native GitLab container image to deploy OpenBao. The OpenBao build includes modifications from the upstream version. As a result, some functionality may differ from the standard OpenBao releases.

ParameterDefaultDescription
image.repositoryregistry.gitlab.com/gitlab-org/build/cng/gitlab-openbaoRepository of the OpenBao image.
image.pullPolicyIfNotPresentImage pull policy.
image.tagOverride this to deploy a custom OpenBao version.
imagePullSecrets[]Secrets to pull images from private repositories.

Ingress and TLS configuration options

The OpenBao chart defaults to Ingress-terminated TLS encryption.

ParameterDefaultDescription
global.openbao.hostopenbao.<GitLab Domain>OpenBao host. Used to configure GitLab webservice and the OpenBao chart.
global.openbao.urlDerived from hostOpenBao URL for GitLab. If present, must be a complete URI.
global.openbao.jwt_audienceSame as urlJWT audience claim for OpenBao authentication. Must match OpenBao bound_audiences. For Geo deployments, see Geo deployment.
global.openbao.psql{}OpenBao database config (host, database, username, password).
ingress.enabledtrueEnable the OpenBao Ingress to allow Runner to reach OpenBao.
ingress.hostnameExternal OpenBao host based on global hosts config.Hostname the Ingress should match.
ingress.tls.enabledtrueEnable Ingress TLS.
ingress.tls.secretNameName of the Kubernetes TLS Secret. Managed by certmanager by default.
tlsSecretNameName of the TLS secret mounted internally. Defaults to the Ingress TLS secret.
ingress.annotationstrueAnnotations rendered to the Ingress. Use this to configure OpenBao for any non-NGINX Ingress controllers.
ingress.configureCertmanagerGlobal certmanager configUse certmanager to manage the TLS certificate.
ingress.certmanagerIssuer<release>-issuerName of the certmanager issuer.
ingress.sslPassthroughNginxfalseAnnotate the Ingress to pass through incoming TLS connections to OpenBao. If certmanager is configured, new HTTP01 challanges will be through another Ingress.
config.tlsDisabletrueDisable internal TLS. If disabled, Ingress TLS passthrough is also disabled.
config.metricsListener.tlsDisabletrueDisable internal TLS of the metrics listener.

You should operate OpenBao with end-to-end encrypted TLS. To enable end-to-end TLS configure, OpenBao to expect a TLS connection and pass the TLS connection through NGINX Ingress:

global:
  ingress:
    useNewIngressForCerts: true
config:
  tlsDisable: false
ingress:
  sslPassthroughNginx: true

Enabling SSL passthrough requires cert-manager to create another Ingress to complete HTTP01 challanges. If you use the bundled certmanager and Issuer, make sure the Issuer sets the correct IngressClass by configuring global.ingress.useNewIngressForCerts.

Gateway API

The OpenBao chart allows to expose traffic via an HTTPRoute. If Gateway API is enabled globally, a listener for OpenBao will be created in the managed Gateway resource.

ParameterDefaultDescription
gatewayRoute.enabledDefaults to value of global.gatewayApi.enabledEnable exposing OpenBao via a HTTPRoute.
gatewayRoute.sectionNameopenbao-webGateway section to be used by the HTTPRoute.
gatewayRoute.gatewayNameGitLab chart managed GatewayGateway name to be used by the HTTPRoute.
gatewayRoute.annotations{}Extra annotations for the HTTPRoute.
gatewayRoute.timeouts{}Custom timeout config for the HTTPRoute.

Monitoring configuration options

OpenBao is preconfigured to expose Prometheus metrics which will be scraped by the bundled Prometheus subchart.

ParameterDefaultDescription
config.telemetry.enabledtrueEnable telemetry and monitoring.
config.telemetry.disableHostnametruePrefix gauge values with local hostname.
config.telemetry.prometheusRetentionTime24hMetrics retention time.
config.telemetry.metricsPrefixopenbaoPrefix for all metrics.
config.telemetry.usageGaugePeriod0Interval at which high-cardinality usage data is collected, such as token counts, entity counts, and secret counts.
config.telemetry.numLeaseMetricsBuckets1Number of expiry buckets for leases.
config.telemetry.prefixFilterMetric prefixes to include (+) or exclude (-) from telemetry. Check OpenBao values for the default.
config.metricsListener.enabledtrueEnable a second API port to serve requests for metrics. The listener can serve all API requests, but serves requests for metrics without authentication.
config.metricsListener.tlsDisabletrueDisable internal TLS of the metrics listener.
config.metricsListener.port8209Port of the metrics listener.
config.metricsListener.unauthenticatedMetricsAccesstrueAllow requests for metrics to be served without authentication.
podMonitor.enabledfalseEnable PodMonitor resource for Prometheus Operator. Requires Prometheus Operator to be installed in the cluster.
podMonitor.additionalLabels{}Additional labels to add to the PodMonitor resource.
podMonitor.selectorLabels{}Additional selector labels to filter which pods to scrape.
podMonitor.endpointConfig{}Additional endpoint configuration (for example, interval, scrapeTimeout).

Unsealing and initialization options

The OpenBao chart supports two mutually exclusive auto-unseal methods:

It also uses OpenBao declarative self initialization.

During initialization, OpenBao generates recovery keys for emergency access when JWT authentication is unavailable. To store and use them, see recovery key management.

ParameterDefaultDescription
config.unseal.static.enabledtrueEnable static auto unsealing.
config.unseal.static.currentKeyIdstatic-unseal-0ID of the current static unsealing key.
config.unseal.static.currentKey/srv/openbao/keys/static-unseal-1Path of the current static unsealing key.
config.unseal.static.previousKeyIdID of the previous static unsealing key.
config.unseal.static.previousKey/srv/openbao/keys/static-unseal-0Path of the previous static unsealing key. Only rendered if previous key ID is also set.
config.unseal.awskms.enabledfalseEnable AWS KMS auto-unsealing.
config.unseal.awskms.kmsKeyIdKMS key ID, ARN, or alias (for example, alias/my-openbao-key). Required when config.unseal.awskms.enabled is true.
config.unseal.awskms.regionAWS region where the KMS key resides.
config.unseal.awskms.endpointOptional custom KMS endpoint URL (for example, a VPC endpoint).
config.initialize.enabledtrueEnable OpenBao self initialization.
config.initialize.oidcDiscoveryUrlExternal GitLab hostOIDC discovery URL. Defaults to the external GitLab hostname.
config.initialize.boundIssuerExternal GitLab hostIssuer URL. Defaults to the external GitLab hostname.
config.initialize.boundAudiencesExternal OpenBao hostOIDC role audiences. Defaults to the external OpenBao hostname.
staticUnsealSecret.generatefalseGenerate a static key to auto unseal OpenBao. Defaults to false as managed by GitLab charts shared-secret chart.
initializeTplTemplate passed to self initialize OpenBao. Check OpenBao values for the default.

AWS KMS unsealing

AWS KMS unsealing delegates the unseal key to an AWS KMS key, removing the need to manage a static key secret.

When running on AWS (EKS, EC2), use IRSA or an instance profile so that no explicit AWS credentials are required. Annotate the OpenBao service account with the IAM role ARN:

openbao:
  serviceAccount:
    annotations:
      eks.amazonaws.com/role-arn: "arn:aws:iam::<account-id>:role/<role-name>"
  config:
    unseal:
      static:
        enabled: false
      awskms:
        enabled: true
        kmsKeyId: "alias/my-openbao-key"
        region: "us-east-1"

The IAM role must have kms:Encrypt, kms:Decrypt, and kms:DescribeKey permissions on the KMS key.

Audit event streaming options

The OpenBao chart configures auditing devices to stream events to GitLab.

ParameterDefaultDescription
global.openbao.httpAudit.secret<release>-openbao-audit-secretName of the secret storing the token shared between OpenBao and GitLab.
global.openbao.httpAudit.keytokenSecret key storing the shared token.
config.audit.http.enabledtrueEnable streaming of auditing events by using HTTP to GitLab.
config.audit.http.streamingUriInternal workhorse URLEndpoint to stream auditing events to.
config.audit.http.authTokenPath/srv/openbao/audit/gitlab-authPath the token shared with GitLab is mounted at.
config.audit.http.requestTimeoutGo duration string (for example, 5s) that bounds each audit-event HTTP POST to the streaming endpoint. Empty, the default, means no timeout.
auditTplTemplate that configures the OpenBao audit devices. Check OpenBao values for the default.
httpAuditSecret.generatefalseGenerate a secret to be shared with GitLab for authenticated auditing. Defaults to false as managed by GitLab charts shared-secret chart.
initializeTplTemplate passed to configure OpenBao auditing. Check OpenBao values for the default.

Change an existing HTTP audit device

OpenBao registers the HTTP audit device declaratively and does not support modifying a registered device in place. To change an audit device, remove the existing audit device and add a new device. Make all changes through the chart values.

The chart supports adding or removing config.audit.http.requestTimeout directly. It switches the audit device path so setting or clearing the timeout is a remove and add operation, not an in-place modification.

To change the timeout from one value to another, use two deploys to force that path switch:

  1. First set requestTimeout to empty.
  2. Set the new value.

Database configuration

OpenBao uses a separate logical database (openbao by default) for data isolation from the Rails backend.

Configure global.openbao.psql or openbao.config.storage.postgresql.connection with host, database, username, and password. You must create the database manually. Password is required and is not inherited from the main GitLab database.

To configure an external database:

  1. Create a PostgreSQL user and database on your database server:

    -- Create the OpenBao user
    CREATE USER openbao WITH PASSWORD '<password>';
    
    -- Create the OpenBao database
    CREATE DATABASE openbao OWNER openbao;
  2. Create a Kubernetes secret containing the password:

    kubectl create secret -n bao generic openbao-db-password --from-literal=password="<password>"
  3. Configure OpenBao to connect to your external database:

    global:
      openbao:
        psql:
          host: "psql.openbao.example.com"
          port: 5432
          database: openbao
          username: openbao
          password:
            secret: openbao-db-password
            key: password

    This uses global.openbao.psql, which is the preferred location because it is also accessible by Toolbox for backup and restore operations. To set advanced connection options (such as sslMode, connectTimeout, or keepalive tuning), use openbao.config.storage.postgresql.connection alongside the global settings.

  4. Deploy or upgrade OpenBao. When starting, OpenBao automatically creates its database schema in the specified database.

PostgreSQL storage options

The following options tune the OpenBao PostgreSQL storage backend. Set them under openbao.config.storage.postgresql, in addition to the connection details in global.openbao.psql.

ParameterDefaultDescription
config.storage.postgresql.haEnabledtrueEnable high availability mode for PostgreSQL storage.
config.storage.postgresql.haTablenullTable used for high availability locks. Uses the OpenBao default when unset.
config.storage.postgresql.maxConnectRetries20Maximum number of database connection retry attempts.
config.storage.postgresql.maxIdleConnections2Maximum number of idle database connections.
config.storage.postgresql.maxParallel5Maximum number of parallel database operations.
config.storage.postgresql.skipCreateTablenullSkip automatic creation of the storage tables. Create them manually when set to true.
config.storage.postgresql.connection.keepalivesnullEnable TCP keepalive probes. Set to 1 to enable or 0 to disable.
config.storage.postgresql.connection.keepalivesCountnullNumber of TCP keepalive probes to send before dropping a connection.
config.storage.postgresql.connection.keepalivesIdlenullSeconds of inactivity before the first TCP keepalive probe.
config.storage.postgresql.connection.keepalivesIntervalnullSeconds between TCP keepalive probes.
config.storage.postgresql.connection.sslModedisablePostgreSQL SSL mode, such as disable, require, or verify-full.