• Metadata database feature support
• Create the database
• Enable the metadata database for Helm charts installations
  • Before you start
  • New installations
  • Existing registries
    • One-step migration
    • Three-step migration

Manage the container registry metadata database

The metadata database enables many new registry features, including online garbage collection, and increases the efficiency of many registry operations. This page contains information on how to create the database.

Metadata database feature support

You can migrate existing registries to the metadata database, and use online garbage collection.

Some database-enabled features are only enabled for GitLab.com and automatic database provisioning for the registry database is not available. Review the feature support table in the feedback issue for the status of features related to the container registry database.

Create the database

Follow the steps below to manually create the database and role.

1. Create the secret with the database password:

   kubectl create secret generic RELEASE_NAME-registry-database-password --from-literal=password=randomstring
   

2. Log into your database instance:

   kubectl exec -it $(kubectl get pods -l app.kubernetes.io/name=postgresql -o custom-columns=NAME:.metadata.name --no-headers) -- bash
   

   PGPASSWORD=${POSTGRES_POSTGRES_PASSWORD} psql -U postgres -d template1
   

3. Create the database user:

   CREATE ROLE registry WITH LOGIN;
   

4. Set the database user password.

   1. Fetch the password:

      kubectl get secret RELEASE_NAME-registry-database-password -o jsonpath="{.data.password}" | base64 --decode
      

   2. Set the password in the psql prompt:

      \password registry
      

5. Create the database:

   CREATE DATABASE registry WITH OWNER registry;
   

6. Safely exit from the PostgreSQL command line and then from the container using exit:

   template1=# exit
   ...@gitlab-postgresql-0/$ exit
   

Enable the metadata database for Helm charts installations

Prerequisites:

• GitLab 16.4 or later.
• PostgreSQL database version 12 or later, accessible from the registry pods.
• Access to the Kubernetes cluster and the Helm deployment locally.
• SSH access to the registry pods.

Follow the instructions that match your situation:

• New installation or enabling the container registry for the first time.
• Migrate existing container images to the metadata database:
  • One-step migration. Only recommended for relatively small registries or no requirement to avoid downtime.
  • Three-step migration. Recommended for larger container registries.

Before you start

Read the before you start section of the Registry administration guide.

New installations

To enable the database:

1. Create the database and Kubernetes secret.

2. Get the current Helm values for your release and save them to a file. For example, for a release named gitlab and a file named values.yml:

   helm get values gitlab > values.yml
   

3. Add the following lines to your values.yml file:

   registry:
     enabled: true
     database:
       enabled: true
       name: registry  # must match the database name you created above
       user: registry  # must match the database username you created above
       password:
         secret: gitlab-registry-database-password # must match the secret name
         key: password  # must match the secret key to read the password from
       sslmode: verify-full
       ssl:
         secret: gitlab-registry-postgresql-ssl # you will need to create this secret manually
         clientKey: client-key.pem
         clientCertificate: client-cert.pem
         serverCA: server-ca.pem
       migrations:
         enabled: true  # this option will execute the schema migration as part of the registry deployment
   

4. Optional. You can verify the schema migrations have been applied properly. You can either:

   • Review the log output of the migrations job, for example:

     kubectl logs jobs/gitlab-registry-migrations-1
     ...
     OK: applied 154 migrations in 13.752s
     

   • Or, connect to the Postgres database and query the schema_migrations table:

     SELECT * FROM schema_migrations;
     

     Ensure the applied_at column timestamp is filled for all rows.

The registry is ready to use the metadata database!

Existing registries

You can migrate your existing container registry data in one step or three steps. A few factors affect the duration of the migration:

• The size of your existing registry data.
• The specifications of your PostgresSQL instance.
• The number of registry pods running in your cluster.
• Network latency between the registry, PostgresSQL and your configured Object Storage.

One-step migration

When doing a one-step migration, be aware that:

• The registry must remain in read-only mode during the migration.
• If the Pod where the migration is being executed is terminated, you have to completely restart the process. The work to improve this process is tracked in issue 5293.

1. Create the database and Kubernetes secret.

2. Get the current Helm values for your release and save them into a file. For example, for a release named gitlab and a file named values.yml:

   helm get values gitlab > values.yml
   

3. Find the registry: section in the values.yml file and add the database section, set the maintenance.readonly.enabled flag to true, and migrations.enabled to true:

   registry:
     priorityClassName: system-node-critical
     enabled: true
     maintenance:
       readonly:
         enabled: true  # must remain set to true while the migration is executed
     database:
       enabled: true
       name: registry  # must match the database name you created above
       user: registry  # must match the database username you created above
       password:
         secret: gitlab-registry-database-password # must match the secret name
         key: password  # must match the secret key to read the password from
           sslmode: verify-full
       ssl:
         secret: gitlab-registry-postgresql-ssl  # you will need to create this secret manually
         clientKey: client-key.pem
         clientCertificate: client-cert.pem
         serverCA: server-ca.pem
       migrations:
         enabled: true  # this option will execute the schema migration as part of the registry deployment
   

4. Upgrade your Helm installation to apply changes in your deployment:

   helm upgrade gitlab gitlab/gitlab -f values.yml
   

5. Connect to one of the registry pods via SSH, for example for a pod named gitlab-registry-5ddcd9f486-bvb57:

   kubectl exec -ti gitlab-registry-5ddcd9f486-bvb57 bash
   

6. Change to the home directory and then run the following command:

   cd ~
   /usr/bin/registry database import /etc/docker/registry/config.yml
   

7. Update the registry configuration to disable read-only mode:

   registry:
     enabled: true
     maintenance:
       readonly:
         enabled: false
     database:
       enabled: true
       name: registry
       user: registry
       password:
         secret: gitlab-registry-database-password
         key: password
       migrations:
         enabled: true
   

8. Upgrade your Helm installation to apply changes in your deployment:

   helm upgrade gitlab gitlab/gitlab -f values.yml
   

You can now use the metadata database for all operations!

Three-step migration