Create a GitLab upgrade plan

Tier: Free, Premium, Ultimate Offering: Self-managed

This document serves as a guide to create a strong plan to upgrade a self-managed GitLab instance.

General notes:

  • If possible, you should test out the upgrade in a test environment before updating your production instance. Ideally, your test environment should mimic your production environment as closely as possible.
  • If working with Support to create your plan, share details of your architecture, including:
    • How is GitLab installed?
    • What is the operating system of the node? (check OS versions that are no longer supported to confirm that later updates are available).
    • Is it a single-node or a multi-node setup? If multi-node, share any architectural details about each node with us.
    • Are you using GitLab Geo? If so, share any architectural details about each secondary node.
    • What else might be unique or interesting in your setup that might be important for us to understand?
    • Are you running into any known issues with your current version of GitLab?

Pre-upgrade and post-upgrade checks

Immediately before and after the upgrade, perform the pre-upgrade and post-upgrade checks to ensure the major components of GitLab are working:

  1. Check the general configuration: 

    sudo gitlab-rake gitlab:check

  2. Confirm that encrypted database values can be decrypted: 

    sudo gitlab-rake gitlab:doctor:secrets
  3. In GitLab UI, check that:
    • Users can sign in.
    • The project list is visible.
    • Project issues and merge requests are accessible.
    • Users can clone repositories from GitLab.
    • Users can push commits to GitLab.
  4. For GitLab CI/CD, check that:
    • Runners pick up jobs.
    • Docker images can be pushed and pulled from the registry.

  5. If using Geo, run the relevant checks on the primary and each secondary: 

    sudo gitlab-rake gitlab:geo:check
  6. If using Elasticsearch, verify that searches are successful.

If in any case something goes wrong, see how to troubleshoot.

Rollback plan

It’s possible that something may go wrong during an upgrade, so it’s critical that a rollback plan be present for that scenario. A proper rollback plan creates a clear path to bring the instance back to its last working state. It is comprised of a way to back up the instance and a way to restore it.

Back up GitLab

Create a backup of GitLab and all its data (database, repositories, uploads, builds, artifacts, LFS objects, registry, pages). This is vital for making it possible to roll back GitLab to a working state if there’s a problem with the upgrade:

  • Create a GitLab backup. Make sure to follow the instructions based on your installation method. Don’t forget to back up the secrets and configuration files.
  • Alternatively, create a snapshot of your instance. If this is a multi-node installation, you must snapshot every node. This process is out of scope for GitLab Support.

Restore GitLab

If you have a test environment that mimics your production one, you should test the restoration to ensure that everything works as you expect.

To restore your GitLab backup:

  • Before restoring, make sure to read about the prerequisites, most importantly, the versions of the backed up and the new GitLab instance must be the same.
  • Restore GitLab. Make sure to follow the instructions based on your installation method. Confirm that the secrets and configuration files are also restored.
  • If restoring from a snapshot, know the steps to do this. This process is out of scope for GitLab Support.

Upgrade plan

For the upgrade plan, start by creating an outline of a plan that best applies to your instance and then upgrade it for any relevant features you’re using.

Additional features

Apart from all the generic information above, you may have enabled some features that require special planning.

Feel free to ignore sections about features that are inapplicable to your setup, such as Geo, external Gitaly, or Elasticsearch.

External Gitaly

If you’re using an external Gitaly server, it must be upgraded to the newer version prior to upgrading the application server.

Geo

If you’re using Geo:

Runners

After updating GitLab, upgrade your runners to match your new GitLab version.

GitLab agent for Kubernetes

If you have Kubernetes clusters connected with GitLab, upgrade your GitLab agents for Kubernetes to match your new GitLab version.

Elasticsearch

Before updating GitLab, confirm advanced search migrations are complete by checking for pending advanced search migrations.

After updating GitLab, you may have to upgrade Elasticsearch if the new version breaks compatibility. Updating Elasticsearch is out of scope for GitLab Support.

Troubleshooting

If anything doesn’t go as planned: