- IAM role
- Chart configuration
The IAM role will need read, write and list permissions on the S3 buckets. You can choose to have a role per bucket or combine them.
IAM roles can be specified by adding annotations and changing the secrets, as specified below:
An IAM role can be specified via the annotations key:
--set registry.annotations."iam\.amazonaws\.com/role"=<role name>
When creating the
registry-storage.yaml secret, omit the access and secret key:
s3: bucket: gitlab-registry v4auth: true region: us-east-1
Note: If you provide the key pair, IAM role will be ignored. See AWS documentation for more details.
For LFS, artifacts, uploads, packages and pseudonymizer an IAM role can be specified via the annotations key in the
--set gitlab.sidekiq.annotations."iam\.amazonaws\.com/role"=<role name> --set gitlab.webservice.annotations."iam\.amazonaws\.com/role"=<role name>
provider: AWS use_iam_profile: true region: us-east-1
The Task Runner configuration allows for annotations to be set to upload backups to S3:
--set gitlab.task-runner.annotations."iam\.amazonaws\.com/role"=<role name>
s3cmd.config secret is to be created without the access and secret keys:
[default] bucket_location = us-east-1
If GitLab is running in an AWS EKS cluster (version 1.14 or greater), you can use an AWS IAM role to authenticate to the S3 object storage without the need of generating or storing access tokens. More information regarding using IAM roles in an EKS cluster can be found in the Introducing fine-grained IAM roles for service accounts documentation from AWS.
Appropriate IRSA annotations for roles can be applied to ServiceAccounts throughout this Helm chart in one of two ways:
- ServiceAccounts that have been pre-created as described in the above AWS documentation. This will ensure the proper annotations on the ServiceAccount and the linked OIDC provider.
- Chart-generated ServiceAccounts with annotations defined. We allow for the configuration of annotations on ServiceAccounts both globally and on a per-chart basis.
backup-utilityas specified in the backup documentation does not properly copy the backup file to the S3 bucket. The
s3cmdto perform the copy of the backup file and it has a known issue of not supporting OIDC authentication. There is a pull request to mitigate this issue, but it has yet to be accepted into the
Set the following options when the GitLab chart is deployed. It is important to note that the ServiceAccount is enabled but not created.
global: serviceAccount: enabled: true create: false name: <SERVICE ACCT NAME>
Fine-grained ServiceAccounts control is also available:
registry: serviceAccount: create: false name: gitlab-registry gitlab: webservice: serviceAccount: create: false name: gitlab-webservice sidekiq: serviceAccount: create: false name: gitlab-sidekiq task-runner: serviceAccount: create: false name: gitlab-task-runner
eks.amazonaws.com/role-arn annotation can be applied to all ServiceAccounts
created by GitLab owned charts by configuring
global: serviceAccount: annotations: eks.amazonaws.com/role-arn: arn:aws:iam::xxxxxxxxxxxx:role/name
Annotations can also be added on a per ServiceAccount basis, but adding the matching definition for each chart. These can be the same role, or individual roles.
registry: serviceAccount: annotations: eks.amazonaws.com/role-arn: arn:aws:iam::xxxxxxxxxxxx:role/gitlab-registry gitlab: webservice: serviceAccount: annotations: eks.amazonaws.com/role-arn: arn:aws:iam::xxxxxxxxxxxx:role/gitlab sidekiq: serviceAccount: annotations: eks.amazonaws.com/role-arn: arn:aws:iam::xxxxxxxxxxxx:role/gitlab task-runner: serviceAccount: annotations: eks.amazonaws.com/role-arn: arn:aws:iam::xxxxxxxxxxxx:role/gitlab-task-runner
You can test if the IAM role is correctly set up and that GitLab is accessing
S3 using the IAM role by logging into the
taskrunner pod and installing the
awscli Python package:
kubectl exec -it <TASK RUNNER POD> -- bash pip install awscli
awscli package installed, verify that you are able to communicate
with the AWS API:
/home/git/.local/bin/aws sts get-caller-identity
awscommand is not in the path so it is necessary to use the full path to execute the command.
A normal response showing the temporary user ID, account number and IAM
ARN (this will not be the IAM ARN for the role used to access S3) will be
returned if connection to the AWS API was successful. An unsuccessful
connection will require more troubleshooting to determine why the
pod is not able to communicate with the AWS APIs.
If connecting to the AWS APIs is successful, then the following command
will assume the IAM role that was created and verify that a STS token can
be retrieved for accessing S3. The
variables are defined in the environment when IAM role annotation has been
added to the pod and do not require that they be defined:
/home/git/.local/bin/aws sts assume-role-with-web-identity --role-arn $AWS_ROLE_ARN --role-session-name gitlab --web-identity-token file://$AWS_WEB_IDENTITY_TOKEN_FILE
If the IAM role could not be assumed then an error message similar to the following will be displayed:
An error occurred (AccessDenied) when calling the AssumeRoleWithWebIdentity operation: Not authorized to perform sts:AssumeRoleWithWebIdentity
Otherwise, the STS credentials and IAM role information will be displayed.