Upgrading GitLab

Tier: Free, Premium, Ultimate Offering: Self-managed

Upgrading GitLab is a relatively straightforward process, but the complexity can increase based on:

  • The installation method you have used.
  • How old your GitLab version is.
  • If you’re upgrading to a major version.

If possible, you should test out the upgrade in a test environment before updating your production instance. Your test environment should mimic your production environment as closely as possible.

Make sure to read the whole page as it contains information related to every upgrade method.

Upgrade GitLab

To upgrade GitLab:

  1. Create an upgrade plan to document your upgrade steps.
  2. Familiarize yourself with the maintenance policy documentation.
  3. Read the release posts for versions you’re passing over. In particular, deprecations, removals, and important notes on upgrading.
  4. Determine what upgrade path you should take. If your upgrade path includes required upgrade stops, you might have to perform multiple upgrades to move from your current version to your target version. If relevant, check OS compatibility with the target GitLab version.
  5. Check for background migrations. All migrations must finish running before each upgrade. You must spread out upgrades between major and minor releases to allow time for background migrations to finish.
  6. Test your upgrade in a test environment first, and have a rollback plan to reduce the risk of unplanned outages and extended downtime.
  7. If available in your starting version, consider turning on maintenance mode during the upgrade.
  8. Consult changes for different versions of GitLab to ensure compatibility before upgrading:
  9. Perform pre-upgrade checks.
  10. Pause running CI/CD pipelines and jobs.
  11. If relevant, follow upgrade steps for additional features:
  12. Follow the upgrade steps based on your installation method.
  13. If your GitLab instance has any runners associated with it, upgrade them to match the current GitLab version. This step ensures compatibility with GitLab versions.
  14. If you encounter problems with the upgrade, get support.
  15. Disable maintenance mode if you had enabled it.
  16. Unpause running CI/CD pipelines and jobs.
  17. Perform post-upgrade checks.

Upgrade based on installation method

Depending on the installation method and your GitLab version, there are multiple official ways to upgrade GitLab:

Linux packages (Omnibus)

As part of a GitLab upgrade, the Linux package upgrade guide contains the specific steps to follow to upgrade a Linux package instance.

Helm chart (Kubernetes)

GitLab can be deployed into a Kubernetes cluster using Helm. For production deployments, the setup follows the Cloud Native Hybrid guidance where stateless components of cloud-native GitLab run in Kubernetes with the GitLab Helm chart, and stateful components are deployed in compute VMs with the Linux package.

Use the version mapping from the chart version to GitLab version to determine the upgrade path.

Follow Multi-node upgrades with downtime to perform the upgrade in a Cloud Native Hybrid setup.

A full cloud-native deployment is not supported for production. However, instructions on how to upgrade such an environment are in a separate document.

Docker

GitLab provides official Docker images for both Community and Enterprise editions, and they are based on the Omnibus package. See how to install GitLab using Docker.

Self-compiled (source)

In the past we used separate documents for the upgrading instructions, but we have switched to using a single document. The old upgrading guidelines can still be found in the Git repository:

Pre-upgrade and post-upgrade checks

Immediately before and after the upgrade, perform the pre-upgrade and post-upgrade checks to ensure the major components of GitLab are working:

  1. Check the general configuration: 

    sudo gitlab-rake gitlab:check

  2. Confirm that encrypted database values can be decrypted: 

    sudo gitlab-rake gitlab:doctor:secrets
  3. In GitLab UI, check that:
    • Users can sign in.
    • The project list is visible.
    • Project issues and merge requests are accessible.
    • Users can clone repositories from GitLab.
    • Users can push commits to GitLab.
  4. For GitLab CI/CD, check that:
    • Runners pick up jobs.
    • Docker images can be pushed and pulled from the registry.

  5. If using Geo, run the relevant checks on the primary and each secondary: 

    sudo gitlab-rake gitlab:geo:check
  6. If using Elasticsearch, verify that searches are successful.

If something goes wrong, get support.

CI/CD pipelines and jobs during upgrades

If you upgrade your GitLab instance while the GitLab Runner is processing jobs, the trace updates fail. When GitLab is back online, the trace updates should self-heal. However, depending on the error, the GitLab Runner either retries, or eventually terminates, job handling.

As for the artifacts, the GitLab Runner attempts to upload them three times, after which the job eventually fails.

To address the above two scenarios, it is advised to do the following prior to upgrading:

  1. Plan your maintenance.

  2. Pause your runners, or block new jobs from starting by adding the following to your /etc/gitlab/gitlab.rb: 

    nginx['custom_gitlab_server_config'] = "location ^~ /api/v4/jobs/request {\n deny all;\n return 503;\n}\n"

    And reconfigure GitLab with: 

    sudo gitlab-ctl reconfigure
  3. Wait until all jobs are finished.
  4. Upgrade GitLab.
  5. Upgrade GitLab Runner to the same version as your GitLab version. Both versions should be the same.
  6. Unpause your runners and unblock new jobs from starting by reverting the previous /etc/gitlab/gitlab.rb change.

Upgrading between editions

GitLab comes in two flavors: Community Edition which is MIT licensed, and Enterprise Edition which builds on top of the Community Edition and includes extra features mainly aimed at organizations with more than 100 users.

Below you can find some guides to help you change GitLab editions.

Community to Enterprise Edition

note
The following guides are for subscribers of the Enterprise Edition only.

If you wish to upgrade your GitLab installation from Community to Enterprise Edition, follow the guides below based on the installation method:

  • Source CE to EE upgrade guides - The steps are very similar to a version upgrade: stop the server, get the code, update configuration files for the new functionality, install libraries and do migrations, update the init script, start the application and check its status.
  • Omnibus CE to EE - Follow this guide to upgrade your Omnibus GitLab Community Edition to the Enterprise Edition.
  • Docker CE to EE - Follow this guide to upgrade your GitLab Community Edition container to an Enterprise Edition container.
  • Helm chart (Kubernetes) CE to EE - Follow this guide to upgrade your GitLab Community Edition Helm deployment to Enterprise Edition.

Enterprise to Community Edition

To downgrade your Enterprise Edition installation back to Community Edition, you can follow this guide to make the process as smooth as possible.

Upgrade steps for additional features

Some GitLab features have additional steps.

External Gitaly

Upgrade Gitaly servers to the newer version before upgrading the application server. This prevents the gRPC client on the application server from sending RPCs that the old Gitaly version does not support.

Geo

If you’re using Geo:

GitLab agent for Kubernetes

If you have Kubernetes clusters connected with GitLab, upgrade your GitLab agents for Kubernetes to match your new GitLab version.

Elasticsearch

Before updating GitLab, confirm advanced search migrations are complete by checking for pending advanced search migrations.

After updating GitLab, you may have to upgrade Elasticsearch if the new version breaks compatibility. Updating Elasticsearch is out of scope for GitLab Support.

Getting support

If something goes wrong:

  • Copy any errors and gather any logs to later analyze, and then roll back to the last working version. You can use the following tools to help you gather data:
    • gitlabsos if you installed GitLab using the Linux package or Docker.
    • kubesos if you installed GitLab using the Helm Charts.

For support: