Migrating from Helm v2 to Helm v3
You can use Helm 2to3 plugin to migrate Helm 2 GitLab releases to Helm 3:
helm 2to3 convert YOUR-GITLAB-RELEASE
Known Issues
“UPGRADE FAILED: cannot patch” error is shown after the migration
After migration the subsequent upgrades may fail with an error similar to the following:
Error: UPGRADE FAILED: cannot patch "..." with kind Deployment: Deployment.apps "..." is invalid: spec.selector:
Invalid value: v1.LabelSelector{...}: field is immutable
or
Error: UPGRADE FAILED: cannot patch "..." with kind StatefulSet: StatefulSet.apps "..." is invalid:
spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'template', and 'updateStrategy' are forbidden
This is due to known issues with Helm 2 to 3 migration in Cert Manager
and Redis dependencies. In a nutshell, the heritage
label
on some Deployments and StatefulSets are immutable and can not be changed from Tiller
(set by Helm 2) to Helm
(set by Helm 3). So they must be replaced forcefully.
To work around this use the following instructions:
- Replace cert-manager Deployments (when enabled).
kubectl get deployments -l app=cert-manager -o yaml | sed "s/Tiller/Helm/g" | kubectl replace --force=true -f -
kubectl get deployments -l app=cainjector -o yaml | sed "s/Tiller/Helm/g" | kubectl replace --force=true -f -
- (Optional) Set
persistentVolumeReclaimPolicy
toRetain
on the PV that is claimed by Redis StatefulSet. This is to ensure that the PV won’t be deleted inadvertently.
kubectl patch pv <PV-NAME> -p '{"spec":{"persistentVolumeReclaimPolicy":"Retain"}}'
- Set
heritage
label of the existing Redis PVC toHelm
.
kubectl label pvc -l app=redis --overwrite heritage=Helm
- Replace Redis StatefulSet without cascading.
kubectl get statefulsets.apps -l app=redis -o yaml | sed "s/Tiller/Helm/g" | kubectl replace --force=true --cascade=false -f -
RBAC issues after the migration when running Helm upgrade
You may face the following error when running Helm upgrade after the conversion has been completed:
Error: UPGRADE FAILED: pre-upgrade hooks failed: warning: Hook pre-upgrade gitlab/templates/shared-secrets/rbac-config.yaml failed: roles.rbac.authorization.k8s.io "gitlab-shared-secrets" is forbidden: user "your-user-name@domain.tld" (groups=["system:authenticated"]) is attempting to grant RBAC permissions not currently held:
{APIGroups:[""], Resources:["secrets"], Verbs:["get" "list" "create" "patch"]}
Helm2 used the Tiller service account to perform such operations. Helm3 does not use Tiller anymore, and your user account should have proper RBAC permissions to run the command even if you are running helm upgrade
as a cluster admin. To grant full RBAC permissions to yourself, run:
kubectl create clusterrolebinding cluster-admin-binding --clusterrole=cluster-admin --user=your-user-name@domain.tld
After that, helm upgrade
should work fine.