• Passing properties to GitLab Runner Operator
• Operator properties
• Cache properties
  • S3 cache
  • gcs cache
  • Azure cache
• Configure a proxy environment
• Customize config.toml with a configuration template
• Configure a custom TLS cert
• Configure the CPU and memory size of runner pods
• Configure job concurrency per runner based on cluster resources
• Troubleshooting
  • Root vs non-root
    • Overriding the HOME environment variable
  • Overriding locked variable
    • Watch out for security context constraints
    • Run as anyuid security context constraints
    • Configure SETFCAP
  • Using FIPS Compliant GitLab Runner
    • Register GitLab Runner by using a self-signed certificate
    • Register GitLab Runner with an external URL that points to an IP address

Configuring GitLab Runner on OpenShift

This document explains how to configure GitLab Runner on OpenShift.

Passing properties to GitLab Runner Operator

When creating a Runner, you can configure it by setting properties in its spec. For example, you can specify the GitLab URL where the runner is registered, or the name of the secret that contains the registration token:

apiVersion: apps.gitlab.com/v1beta2
kind: Runner
metadata:
 name: dev
spec:
 gitlabUrl: https://gitlab.example.com
 token: gitlab-runner-secret # Name of the secret containing the Runner token

Read about all the available properties in Operator properties.

Operator properties

The following properties can be passed to the Operator.

Some properties are only available with more recent versions of the Operator.

Cache properties

S3 cache

gcs cache

Azure cache

Configure a proxy environment

To create a proxy environment:

1. Edit the custom-env.yaml file. For example:

   apiVersion: v1
   data:
     HTTP_PROXY: example.com
   kind: ConfigMap
   metadata:
     name: custom-env
   

2. Update OpenShift to apply the changes.

   oc apply -f custom-env.yaml
   

3. Update your gitlab-runner.yml file.

   apiVersion: apps.gitlab.com/v1beta2
   kind: Runner
   metadata:
     name: dev
   spec:
     gitlabUrl: https://gitlab.example.com
     token: gitlab-runner-secret # Name of the secret containing the Runner token
     env: custom-env
   

If the proxy can’t reach the Kubernetes API, you might see an error in your CI/CD job:

ERROR: Job failed (system failure): prepare environment: setting up credentials: Post https://172.21.0.1:443/api/v1/namespaces/<KUBERNETES_NAMESPACE>/secrets: net/http: TLS handshake timeout. Check https://docs.gitlab.com/runner/shells/index.html#shell-profile-loading for more information

To resolve this error, add the IP address of the Kubernetes API to NO_PROXY configuration in the custom-env.yaml file:

   apiVersion: v1
   data:
     NO_PROXY: 172.21.0.1
     HTTP_PROXY: example.com
   kind: ConfigMap
   metadata:
     name: custom-env

You can verify the IP address of the Kubernetes API by running:

oc get services --namespace default --field-selector='metadata.name=kubernetes' | grep -v NAME | awk '{print $3}'

Customize config.toml with a configuration template

You can customize the runner’s config.toml file by using the configuration template.

1. Create a custom configuration template file. For example, let’s instruct our runner to mount an EmptyDir volume. Create the custom-config.toml file:

   [[runners]]
     [runners.kubernetes]
       [runners.kubernetes.volumes]
         [[runners.kubernetes.volumes.empty_dir]]
           name = "empty-dir"
           mount_path = "/path/to/empty_dir"
           medium = "Memory"
   

2. Create a ConfigMap named custom-config-toml from our custom-config.toml file:

    oc create configmap custom-config-toml --from-file config.toml=custom-config.toml
   

3. Set the config property of the Runner:

   apiVersion: apps.gitlab.com/v1beta2
   kind: Runner
   metadata:
     name: dev
   spec:
     gitlabUrl: https://gitlab.example.com
     token: gitlab-runner-secret
     config: custom-config-toml
   

Configure a custom TLS cert

1. To set a custom TLS cert, create a secret with key tls.crt. In this example, the file is named custom-tls-ca-secret.yaml:

   apiVersion: v1
   kind: Secret
   metadata:
       name: custom-tls-ca
   type: Opaque
   stringData:
       tls.crt: |
           -----BEGIN CERTIFICATE-----
           MIIEczCCA1ugAwIBAgIBADANBgkqhkiG9w0BAQQFAD..AkGA1UEBhMCR0Ix
           .....
           7vQMfXdGsRrXNGRGnX+vWDZ3/zWI0joDtCkNnqEpVn..HoX
           -----END CERTIFICATE-----
   

2. Create the secret:

   oc apply -f custom-tls-ca-secret.yaml
   

3. Set the ca key in the runner.yaml to the same name as the name of our secret:

   apiVersion: apps.gitlab.com/v1beta2
   kind: Runner
   metadata:
     name: dev
   spec:
     gitlabUrl: https://gitlab.example.com
     token: gitlab-runner-secret
     ca: custom-tls-ca
   

Configure the CPU and memory size of runner pods

To set CPU limits and memory limits in a custom config.toml file, follow the instructions in this topic.

Configure job concurrency per runner based on cluster resources

Set the concurrent property of the Runner resource:

apiVersion: apps.gitlab.com/v1beta2
kind: Runner
metadata:
  name: dev
spec:
  gitlabUrl: https://gitlab.example.com
  token: gitlab-runner-secret
  concurrent: 2

Job concurrency is dictated by the requirements of the project.

1. Start by trying to determine the compute and memory resources required to execute a CI job.
2. Calculate how many times that job would be able to execute given the resources in the cluster.

If you set a high concurrency value, the Kubernetes executor processes the jobs as soon as it can. However, the Kubernetes cluster’s scheduler capacity determines when the job is scheduled.

Troubleshooting

Root vs non-root

The GitLab Runner Operator and the GitLab Runner pod run as non-root users. As a result, the build image used in the job must run as a non-root user to be able to complete successfully. This ensures that jobs can run successfully with the least permission.

To make this work, make sure that the build image used for CI/CD jobs:

• Runs as non-root
• Does not write to restricted filesystems

Most container filesystems on an OpenShift cluster are read-only, except:

• Mounted volumes
• /var/tmp
• /tmp
• Other volumes mounted on root filesystems as tmpfs

Overriding the HOME environment variable

If creating a custom build image or overriding environment variables, ensure that the HOME environment variables is not set to / which would be read-only. Especially if your jobs would need to write files to the home directory. You could create a directory under /home for example /home/ci and set ENV HOME=/home/ci in your Dockerfile.

For the runner pods it’s expected that HOME would be set to /home/gitlab-runner. If this variable is changed, the new location must have the proper permissions. These guidelines are also documented in the Red Hat Container Platform documentation.

Overriding locked variable

When you register a runner token, if you set the locked variable to true, the error Runner configuration other than name, description, and exector is reserved and cannot be specified appears.

  locked: true # REQUIRED
  tags: ""
  runUntagged: false
  protected: false
  maximumTimeout: 0

For more information, see issue 472.

Watch out for security context constraints

By default, when installed in a new OpenShift project, the GitLab Runner Operator runs as non-root. Some projects, like the default project, are exceptions where all service accounts have anyuid access. In that case, the user of the image is root. You can check this by running the whoami inside any container shell, for example, a job. Read more about security context constraints in Red Hat Container Platform documentation.

Run as anyuid security context constraints

To run a CI/CD job as the root user or write to root filesystems, set the anyuid security context constraints on the gitlab-runner-app-sa service account. The GitLab Runner container uses this service account.

In OpenShift 4.3.8 and earlier:

oc adm policy add-scc-to-user anyuid -z gitlab-runner-app-sa -n <runner_namespace>

# Check that the anyiud SCC is set:
oc get scc anyuid -o yaml

In OpenShift 4.3.8 and later:

oc create -f - <<EOF
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: scc-anyuid
  namespace: <runner_namespace>
rules:
- apiGroups:
  - security.openshift.io
  resourceNames:
  - anyuid
  resources:
  - securitycontextconstraints
  verbs:
  - use
EOF

oc create -f - <<EOF
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: sa-to-scc-anyuid
  namespace: <runner_namespace>
subjects:
  - kind: ServiceAccount
    name: gitlab-runner-app-sa
roleRef:
  kind: Role
  name: scc-anyuid
  apiGroup: rbac.authorization.k8s.io
EOF

Configure SETFCAP

If you use Red Hat OpenShift Container Platform (RHOCP) 4.11 or later, you may get the following error message:

error reading allowed ID mappings:error reading subuid mappings for user

Some jobs (for example, buildah) need the SETFCAP capability granted to run correctly. To fix this issue:

1. Add the SETFCAP capability to the security context constraints that GitLab Runner is using (replace the gitlab-scc with the security context constraints assigned to your GitLab Runner pod):

   oc patch scc gitlab-scc --type merge -p '{"allowedCapabilities":["SETFCAP"]}'
   

2. Update your config.toml and add the SETFCAP capability under the kubernetes section:

   [[runners]]
     [runners.kubernetes]
     [runners.kubernetes.pod_security_context]
       [runners.kubernetes.build_container_security_context]
       [runners.kubernetes.build_container_security_context.capabilities]
         add = ["SETFCAP"]
   

3. Create a ConfigMap using this config.toml in the namespace where GitLab Runner is deployed:

   oc create configmap custom-config-toml --from-file config.toml=config.toml
   

4. Modify the runner you want to fix, adding the config: parameter to point to the recently created ConfigMap (replace my-runner with the correct runner pod name).

   oc patch runner my-runner --type merge -p '{"spec": {"config": "custom-config-toml"}}'
   

For more information, see the Red Hat documentation.

Using FIPS Compliant GitLab Runner

To use a FIPS compliant GitLab Runner helper, change the helper image as follows:

apiVersion: apps.gitlab.com/v1beta2
kind: Runner
metadata:
 name: dev
spec:
 gitlabUrl: https://gitlab.example.com
 token: gitlab-runner-secret
 helperImage: gitlab/gitlab-runner-helper:ubi-fips
 concurrent: 2

Register GitLab Runner by using a self-signed certificate

To use self-signed certificate with GitLab self-managed, create a secret that contains the CA certificate you used to sign the private certificates.

The name of the secret is then provided as the CA in the Runner spec section:

KIND:     Runner
VERSION:  apps.gitlab.com/v1beta2

FIELD:    ca <string>

DESCRIPTION:
     Name of tls secret containing the custom certificate authority (CA)
     certificates

The secret can be created using the following command:

oc create secret generic mySecret --from-file=tls.crt=myCert.pem -o yaml

Register GitLab Runner with an external URL that points to an IP address

If the runner cannot match the self-signed certificate with the hostname, you might get an error message. This issue occurs when you configure GitLab self-managed to use an IP address (like ###.##.##.##) instead of a hostname:

[31;1mERROR: Registering runner... failed               [0;m  [31;1mrunner[0;m=A5abcdEF [31;1mstatus[0;m=couldn't execute POST against https://###.##.##.##/api/v4/runners:
Post https://###.##.##.##/api/v4/runners: x509: cannot validate certificate for ###.##.##.## because it doesn't contain any IP SANs
[31;1mPANIC: Failed to register the runner. You may be having network problems.[0;m

To fix this issue:

1. On the GitLab self-managed server, modify the openssl to add the IP address to the subjectAltName parameter:

   # vim /etc/pki/tls/openssl.cnf
   
   [ v3_ca ]
   subjectAltName=IP:169.57.64.36 <---- Add this line. 169.57.64.36 is your GitLab server IP.
   

2. Then re-generate a self-signed CA with the commands below:

   # cd /etc/gitlab/ssl
   # openssl req -x509 -nodes -days 3650 -newkey rsa:4096 -keyout /etc/gitlab/ssl/169.57.64.36.key -out /etc/gitlab/ssl/169.57.64.36.crt
   # openssl dhparam -out /etc/gitlab/ssl/dhparam.pem 4096
   # gitlab-ctl restart
   

3. Use this new certificate to generate a new secret.