Toolbox

The Toolbox Pod is used to execute periodic housekeeping tasks within the GitLab application. These tasks include backups, Sidekiq maintenance, and Rake tasks.

Configuration

The following configuration settings are the default settings provided by the Toolbox chart:

gitlab:
  ## doc/charts/gitlab/toolbox
  toolbox:
    enabled: true
    replicas: 1
    backups:
      cron:
        enabled: false
        concurrencyPolicy: Replace
        failedJobsHistoryLimit: 1
        schedule: "0 1 * * *"
        successfulJobsHistoryLimit: 3
        suspend: false
        backoffLimit: 6
        restartPolicy: "OnFailure"
        resources:
          requests:
            cpu: 50m
            memory: 350M
        persistence:
          enabled: false
          accessMode: ReadWriteOnce
          size: 10Gi
      objectStorage:
        backend: s3
        config: {}
    persistence:
      enabled: false
      accessMode: 'ReadWriteOnce'
      size: '10Gi'
    resources:
      requests:
        cpu: '50m'
        memory: '350M'
    securityContext:
      fsGroup: '1000'
      runAsUser: '1000'
ParameterDescriptionDefault
annotationsAnnotations to add to the Toolbox Pods and Jobs{}
common.labelsSupplemental labels that are applied to all objects created by this chart.{}
antiAffinityLabels.matchLabelsLabels for setting anti-affinity options 
backups.cron.activeDeadlineSecondsBackup CronJob active deadline seconds (if null, no active deadline is applied)null
backups.cron.backoffLimitBackup CronJob backoff limit6
backups.cron.concurrencyPolicyKubernetes Job concurrency policyReplace
backups.cron.enabledBackup CronJob enabled flagfalse
backups.cron.extraArgsString of arguments to pass to the backup utility 
backups.cron.failedJobsHistoryLimitNumber of failed backup jobs list in history1
backups.cron.persistence.accessModeBackup cron persistence access modeReadWriteOnce
backups.cron.persistence.enabledBackup cron enable persistence flagfalse
backups.cron.persistence.matchExpressionsLabel-expression matches to bind 
backups.cron.persistence.matchLabelsLabel-value matches to bind 
backups.cron.persistence.sizeBackup cron persistence volume size10Gi
backups.cron.persistence.storageClassStorageClass name for provisioning 
backups.cron.persistence.subPathBackup cron persistence volume mount path 
backups.cron.persistence.volumeNameExisting persistent volume name 
backups.cron.resources.requests.cpuBackup cron minimum needed CPU50m
backups.cron.resources.requests.memoryBackup cron minimum needed memory350M
backups.cron.restartPolicyBackup cron restart policy (Never or OnFailure)OnFailure
backups.cron.scheduleCron style schedule string0 1 * * *
backups.cron.startingDeadlineSecondsBackup cron job starting deadline, in seconds (if null, no starting deadline is applied)null
backups.cron.successfulJobsHistoryLimitNumber of successful backup jobs list in history3
backups.cron.suspendBackup cron job is suspendedfalse
backups.objectStorage.backendObject storage provider to use (s3 or gcs)s3
backups.objectStorage.config.gcpProjectGCP Project to use when backend is gcs ””
backups.objectStorage.config.keyKey containing credentials in secret””
backups.objectStorage.config.secretObject storage credentials secret””
common.labelsSupplemental labels that are applied to all objects created by this chart.{}
deployment.strategyAllows one to configure the update strategy utilized by the deployment{ type: Recreate }
enabledToolbox enablement flagtrue
extraYAML block for extra gitlab.yml configuration {}
image.pullPolicyToolbox image pull policyIfNotPresent
image.pullSecretsToolbox image pull secrets 
image.repositoryToolbox image repositoryregistry.gitlab.com/gitlab-org/build/cng/gitlab-toolbox-ee
image.tagToolbox image tagmaster
init.image.repositoryToolbox init image repository 
init.image.tagToolbox init image tag 
init.resourcesToolbox init container resource requirements{ requests: { cpu: 50m }}
nodeSelectorToolbox and backup job node selection 
persistence.accessModeToolbox persistence access modeReadWriteOnce
persistence.enabledToolbox enable persistence flagfalse
persistence.matchExpressionsLabel-expression matches to bind 
persistence.matchLabelsLabel-value matches to bind 
persistence.sizeToolbox persistence volume size10Gi
persistence.storageClassStorageClass name for provisioning 
persistence.subPathToolbox persistence volume mount path 
persistence.volumeNameExisting PersistentVolume name 
podLabelsLabels for running Toolbox Pods{}
replicasNumber of Toolbox Pods to run1
resources.requestsToolbox minimum requested resources{ cpu: 50m, memory: 350M
securityContext.fsGroupGroup ID under which the pod should be started1000
securityContext.runAsUserUser ID under which the pod should be started1000
serviceAccount.annotationsAnnotations for ServiceAccount{}
serviceAccount.enabledFlag for using ServiceAccountfalse
serviceAccount.createFlag for creating a ServiceAccountfalse
serviceAccount.nameName of ServiceAccount to use 
tolerationsTolerations to add to the Toolbox 
extraEnvFromList of extra environment variables from other data sources to expose 

Configuring backups

Information concerning configuring backups in the backup and restore documentation. Additional information about the technical implementation of how the backups are performed can be found in the backup and restore architecture documentation.]

Persistence configuration

The persistent stores for backups and restorations are configured separately. Please review the following considerations when configuring GitLab for backup and restore operations.

Backups use the backups.cron.persistence.* properties and restorations use the persistence.* properties. Further descriptions concerning the configuration of a persistence store will use just the final property key (e.g. .enabled or .size) and the appropriate prefix will need to be added.

The persistence stores are disabled by default, thus .enabled needs to be set to true for a backup or restoration of any appreciable size. In addition, either .storageClass needs to be specified for a PersistentVolume to be created by Kubernetes or a PersistentVolume needs to be manually created. If .storageClass is specified as ‘-‘, then the PersistentVolume will be created using the default StorageClass as specified in the Kubernetes cluster.

If the PersistentVolume is created manually, then the volume can be specified using the .volumeName property or by using the selector .matchLables / .matchExpressions properties.

In most cases the default value of .accessMode will provide adequate controls for only Toolbox accessing the PersistentVolumes. Please consult the documentation for the CSI driver installed in the Kubernetes cluster to ensure that the setting is correct.

Backup considerations

A backup operation needs an amount of disk space to hold the individual components that are being backed up before they are written to the backup object store. The amount of disk space depends on the following factors:

  • Number of projects and the amount of data stored under each project
  • Size of the PostgresSQL database (issues, MRs, etc.)
  • Size of each object store backend

Once the rough size has been determined, the backups.cron.persistence.size property can be set so that backups can commence.

Restore considerations

During the restoration of a backup, the backup needs to be extracted to disk before the files are replaced on the running instance. The size of this restoration disk space is controlled by the persistence.size property. Be mindful that as the size of the GitLab installation grows the size of the restoration disk space also needs to grow accordingly. In most cases the size of the restoration disk space should be the same size as the backup disk space.

Toolbox included tools

The Toolbox container contains useful GitLab tools such as Rails console, Rake tasks, etc. These commands allow one to check the status of the database migrations, execute Rake tasks for administrative tasks, interact with the Rails console:

# locate the Toolbox pod
kubectl get pods -lapp=toolbox

# Launch a shell inside the pod
kubectl exec -it <Toolbox pod name> -- bash

# open Rails console
gitlab-rails console -e production

# execute a Rake task
gitlab-rake gitlab:env:info