- Configure the former primary node to be a secondary node
- Promote the secondary node to primary node
- Restore the secondary node
After a failover, it is possible to fail back to the demoted primary node to restore your original configuration. This process consists of two steps:
- Making the old primary node a secondary node.
- Promoting a secondary node to a primary node.
Since the former primary node will be out of sync with the current primary node, the first step is to bring the former primary node up to date. Note, deletion of data stored on disk like repositories and uploads will not be replayed when bringing the former primary node back into sync, which may result in increased disk usage. Alternatively, you can set up a new secondary GitLab instance to avoid this.
To bring the former primary node up to date:
- SSH into the former primary node that has fallen behind.
Make sure all the services are up:
sudo gitlab-ctl startIf you disabled the primary node permanently, you need to undo those steps now. For Debian/Ubuntu you just need to run
sudo systemctl enable gitlab-runsvdir. For CentOS 6, you need to install the GitLab instance from scratch and set it up as a secondary node by following Setup instructions. In this case, you don’t need to follow the next step.
Set up database replication. In this case, the secondary node
refers to the former primary node.
- If PgBouncer was enabled on the current secondary node
(when it was a primary node) disable it by editing
sudo gitlab-ctl reconfigure.
- You can then set up database replication on the secondary node.
- If PgBouncer was enabled on the current secondary node (when it was a primary node) disable it by editing
If you have lost your original primary node, follow the setup instructions to set up a new secondary node.
When the initial replication is complete and the primary node and secondary node are closely in sync, you can do a planned failover.
If your objective is to have two nodes again, you need to bring your secondary node back online as well by repeating the first step (configure the former primary node to be a secondary node) for the secondary node.