Using the Praefect chart

The Praefect chart is used to manage Gitaly Cluster (Praefect) inside a GitLab installment deployed with the Helm charts.

Known issues

1. The database has to be manually created.
2. The cluster size is fixed: Gitaly Cluster (Praefect) does not support autoscaling.
3. Using a Praefect instance in the cluster to manage Gitaly instances outside the cluster is not supported.

Requirements

This chart consumes the Gitaly chart. Settings from global.gitaly are used to configure the instances created by this chart. Documentation of these settings can be found in Gitaly chart documentation.

Important: global.gitaly.tls is independent of global.praefect.tls. They are configured separately.

By default, this chart will create 3 Gitaly Replicas.

Configuration

The chart is disabled by default. To enable it as part of a chart deploy set global.praefect.enabled=true.

Replicas

The default number of replicas to deploy is 3. This can be changed by setting global.praefect.virtualStorages[].gitalyReplicas with the desired number of replicas. For example:

global:
  praefect:
    enabled: true
    virtualStorages:
    - name: default
      gitalyReplicas: 4
      maxUnavailable: 1

Multiple virtual storages

Multiple virtual storages can be configured (see Gitaly Cluster (Praefect) documentation). For example:

global:
  praefect:
    enabled: true
    virtualStorages:
    - name: default
      gitalyReplicas: 4
      maxUnavailable: 1
    - name: vs2
      gitalyReplicas: 5
      maxUnavailable: 2

This will create two sets of resources for Gitaly. This includes two Gitaly StatefulSets (one per virtual storage).

Administrators can then configure where new repositories are stored.

Persistence

It is possible to provide persistence configuration per virtual storage.

global:
  praefect:
    enabled: true
    virtualStorages:
    - name: default
      gitalyReplicas: 4
      maxUnavailable: 1
      persistence:
        enabled: true
        size: 50Gi
        accessMode: ReadWriteOnce
        storageClass: storageclass1
    - name: vs2
      gitalyReplicas: 5
      maxUnavailable: 2
      persistence:
        enabled: true
        size: 100Gi
        accessMode: ReadWriteOnce
        storageClass: storageclass2

defaultReplicationFactor

defaultReplicationFactor can be configured on each virtual storages. (see configure replication-factor documentation).

global:
  praefect:
    enabled: true
    virtualStorages:
    - name: default
      gitalyReplicas: 5
      maxUnavailable: 2
      defaultReplicationFactor: 3
    - name: secondary
      gitalyReplicas: 4
      maxUnavailable: 1
      defaultReplicationFactor: 2

Migrating to Praefect

When migrating from standalone Gitaly instances to a Praefect setup, global.praefect.replaceInternalGitaly can be set to false. This ensures that the existing Gitaly instances are preserved while the new Praefect-managed Gitaly instances are created.

global:
  praefect:
    enabled: true
    replaceInternalGitaly: false
    virtualStorages:
    - name: virtualStorage2
      gitalyReplicas: 5
      maxUnavailable: 2

The instructions to migrate to Gitaly Cluster (Praefect) can then be followed to move data from the default storage to virtualStorage2. If additional storages were defined under global.gitaly.internal.names, be sure to migrate repositories from those storages as well.

After the repositories have been migrated to virtualStorage2, replaceInternalGitaly can be set back to true if a storage named default is added in the Praefect configuration.

global:
  praefect:
    enabled: true
    replaceInternalGitaly: true
    virtualStorages:
    - name: default
      gitalyReplicas: 4
      maxUnavailable: 1
    - name: virtualStorage2
      gitalyReplicas: 5
      maxUnavailable: 2

The instructions to migrate to Gitaly Cluster (Praefect) can be followed again to move data from virtualStorage2 to the newly-added default storage if desired.

Finally, see the repository storage paths documentation to configure where new repositories are stored.

Creating the database

Praefect uses its own database to track its state. This has to be manually created in order for Praefect to be functional.

1. Log into your database instance:

   kubectl exec -it $(kubectl get pods -l app.kubernetes.io/name=postgresql -o custom-columns=NAME:.metadata.name --no-headers) -- bash

   PGPASSWORD=$(echo $POSTGRES_POSTGRES_PASSWORD) psql -U postgres -d template1

2. Create the database user:

   CREATE ROLE praefect WITH LOGIN;

3. Set the database user password.

   By default, the shared-secrets Job will generate a secret for you.

   1. Fetch the password:

      kubectl get secret RELEASE_NAME-praefect-dbsecret -o jsonpath="{.data.secret}" | base64 --decode

   2. Set the password in the psql prompt:

      \password praefect

4. Create the database:

   CREATE DATABASE praefect WITH OWNER praefect;

Running Praefect over TLS

Praefect supports communicating with client and Gitaly nodes over TLS. This is controlled by the settings global.praefect.tls.enabled and global.praefect.tls.secretName. To run Praefect over TLS follow these steps:

1. The Helm chart expects a certificate to be provided for communicating over TLS with Praefect. This certificate should apply to all the Praefect nodes that are present. Hence all hostnames of each of these nodes should be added as a Subject Alternate Name (SAN) to the certificate or alternatively, you can use wildcards.

   To know the hostnames to use, check the file /srv/gitlab/config/gitlab.yml file in the Toolbox Pod and check the various gitaly_address fields specified under repositories.storages key within it.

   kubectl exec -it <Toolbox Pod> -- grep gitaly_address /srv/gitlab/config/gitlab.yml

1. Create a TLS Secret using the certificate created.

   kubectl create secret tls <secret name> --cert=praefect.crt --key=praefect.key

2. Redeploy the Helm chart by passing --set global.praefect.tls.enabled=true.

When running Gitaly over TLS, a secret name must be provided for each virtual storage.

global:
  gitaly:
    tls:
      enabled: true
  praefect:
    enabled: true
    tls:
      enabled: true
      secretName: praefect-tls
    virtualStorages:
    - name: default
      gitalyReplicas: 4
      maxUnavailable: 1
      tlsSecretName: default-tls
    - name: vs2
      gitalyReplicas: 5
      maxUnavailable: 2
      tlsSecretName: vs2-tls

Installation command line options

The table below contains all the possible charts configurations that can be supplied to the helm install command using the --set flags.

serviceAccount

This section controls if a ServiceAccount should be created and if the default access token should be mounted in pods.

affinity

For more information, see affinity.