- Recommended Environment
- Runner Shorts Video Tutorials
- 1. Clone GitLab Runner
- 2. Install dependencies and Go runtime
- 3. Install Rancher Desktop
- 4. Install GitLab Runner dependencies
- 5. Build GitLab Runner
- 6. Run GitLab Runner
- 7. Run test suite locally
- 8. Run tests with helper image version of choice
- 9. Install optional tools
- 10. Contribute
- Managing build dependencies
- Developing for Windows on a non-windows environment
- Other resources
GitLab Runner is a golang binary which can operate in two modes:
- GitLab Runner executing jobs locally (“instance” executor).
- GitLab Runner Manager delegating jobs to an autoscaled environment which uses GitLab Runner Helper to pull artifacts.
For developing GitLab Runner in instance executor mode (1) the only setup required is a working golang environment. For developing GitLab Runner in Manager and Helper mode (2) setup also requires a Docker build environment. Additionally running the Manager or Helper in Kubernetes will require a working cluster.
The following instructions setup your golang environment using
asdf to manage the golang version. If you already have this or otherwise know what you’re doing, you can skip step 2 (“Install dependencies and Go runtime”).
In order to provide Docker and Kubernetes locally Step 3 has you setting Rancher Desktop. If you don’t need one or both you can skip step 3 (“Install Rancher Desktop”) or just disable
k3s (Kubernetes) in Rancher Desktop.
The recommended environment on which to install golang and Rancher Desktop for development is a local laptop or desktop. It is possible to use nested-virtualization to run Rancher Desktop in the cloud (which runs
k3s in a VM) but it’s more tricky to setup.
You can also follow along with the Runner Shorts (~20 minute videos) on setting up and making a change:
- Please read the recommended environment section above before beginning
- Setting up a GitLab Runner development environment
- Code walkthrough of GitLab Runner
- Making and testing locally a GitLab Runner change
git clone https://gitlab.com/gitlab-org/gitlab-runner.git
If you are developing for GitLab Runner in autoscaled mode (Manager and Helper) you might want to check out one or more of Taskscaler, Fleeting and associated plugins. To make local changes from one package visible to the others, use golang workspaces.
git clone https://gitlab.com/gitlab-org/fleeting/taskscaler.git git clone https://gitlab.com/gitlab-org/fleeting/fleeting.git git clone https://gitlab.com/gitlab-org/fleeting/fleeting-plugin-aws.git git clone https://gitlab.com/gitlab-org/fleeting/fleeting-plugin-googlecompute.git go work init go work use gitlab-runner go work use taskscaler go work use fleeting go work use fleeting-plugin-aws go work use fleeting-plugin-googlecompute
The GitLab Runner project uses
asdf to manage dependencies.
The simplest way to get your development environment setup is to use
cd gitlab-runner asdf plugin add golang asdf install
asdf, follow the instructions below for the relevant distribution.
sudo apt-get install -y mercurial git-core wget make build-essential wget https://storage.googleapis.com/golang/go1.18.9.linux-amd64.tar.gz sudo tar -C /usr/local -xzf go*-*.tar.gz export PATH="$(go env GOBIN):$PATH"
sudo yum install mercurial wget make sudo yum groupinstall 'Development Tools' wget https://storage.googleapis.com/golang/go1.18.9.linux-amd64.tar.gz sudo tar -C /usr/local -xzf go*-*.tar.gz export PATH="$(go env GOBIN):$PATH"
Using binary package:
wget https://storage.googleapis.com/golang/go1.18.9.darwin-amd64.tar.gz sudo tar -C /usr/local -xzf go*-*.tar.gz export PATH="$(go env GOBIN):$PATH"
Using installation package:
wget https://storage.googleapis.com/golang/go1.18.9.darwin-amd64.pkg open go*-*.pkg export PATH="$(go env GOBIN):$PATH"
pkg install go-1.18.9 gmake git mercurial export PATH="$(go env GOBIN):$PATH"
The Docker Engine is required to create pre-built image that is embedded into GitLab Runner and loaded when using Docker executor. A local Kubernetes cluster is helpful for developing Kubernetes executor. Rancher Desktop provides both.
To install Rancher Desktop, follow the installation instructions for your OS.
dockerd (moby)and not
make deps asdf reshim
For FreeBSD use
Set image version environment variables:
export ALPINE_312_VERSION="3.12.12" export ALPINE_313_VERSION="3.13.12" export ALPINE_314_VERSION="3.14.8" export ALPINE_315_VERSION="3.15.6" export UBUNTU_VERSION="20.04"
Compile GitLab Runner using the Go toolchain:
make runner-and-helper-bin-host is a superset of
make runner-bin-host which in addition
takes care of building the Runner Helper Docker archive dependencies.
You can use the any of the usual command-line arguments (including
./out/binaries/gitlab-runner --debug run
If you want to build the Docker images, run
make runner-and-helper-docker-host, which will:
gitlab-runner-helperand create a helper Docker image from it.
- Compile GitLab Runner for
- Build a DEB package for Runner. The official GitLab Runner images are based on Alpine and Ubuntu, and the Ubuntu image build uses the DEB package.
- Build the Alpine and Ubuntu versions of the
The Next Runner Auto-scaling Architecture adds a new mechanism for autoscaling which will work with all environments. It will replace all current autoscaling mechanisms (e.g. Docker Machine). This new mechanism is in a pre-alpha state and actively being developed. There are two new libraries being used in GitLab Runner:
You don’t need to check out these libraries to use GitLab Runner at HEAD, but some development in the autoscaling space may take place there. In addition Taskscaler and Fleeting, there are a number of Fleeting Plugins which adapt GitLab Runner to a specific cloud providers (e.g. Google Computer or AWS EC2). The written instructions above (“Clone GitLab Runner”) show how to check out the code and the videos (“Runner Shorts”) show how to use it. These instructions show how to use GitLab Runner with a plugin.
Each plugin will come with instructions on how to build the binary and configure the underlying instance group. This work is being done in this issue. The canonical build and configuration instructions will live with each plugin, but in the meantime, here are some general instructions.
Each plugin can be built with
go build -o <plugin-name> ./cmd/.
The resulting binary should be placed somewhere on the local
GitLab Runner is started in the usual way but specifies an
It also specifies under
connector_config an Instance Group, its location, and some details about how to connect to the underlying instances.
GitLab Runner should find the Instance Group and create an initial number of idle VMs.
When a job is picked up the configured instance runner, it will consume a running VM and replace it via AWS service calls in the
[[runners]] name = "local-taskrunner" url = "https://gitlab.com/" token = "REDACTED" executor = "instance" shell = "bash" [runners.autoscaler] max_use_count = 1 max_instances = 20 plugin = "fleeting-plugin-aws" # Fleeting plugin name as built above . [runners.autoscaler.plugin_config] credentials_file = "/Users/josephburnett/.aws/credentials". # Credentials which can scale an Autoscaling Group (ASG) . name = "jburnett-taskrunner-asg" # ASG name. project = "jburnett-ad8e5d54" # ASG project. region = "us-east-2" # ASG region. [runners.autoscaler.connector_config] username = "ubuntu" # ASG instance template username for login. [[runners.autoscaler.policy]] idle_count = 5 idle_time = 0 scale_factor = 0.0 scale_factor_limit = 0
If you terminate GitLab Runner with SIGTERM you may see some of these processes hanging around. Instead terminate with SIGQUIT.
Note that ASGs should have autoscaling disabled. GitLab Runner takes care of autoscaling via the Taskscaler library.
GitLab Runner test suite consists of “core” tests and tests for executors. Tests for executors require certain binaries to be installed on your local machine. Some of these binaries cannot be installed on all operating systems. If a binary is not installed tests requiring this binary will be skipped.
These are the binaries that you can install:
- VirtualBox and Vagrant; the Vagrant Parallels plugin is also required
- kubectl with minikube
- Parallels Pro or Business edition
After installing the binaries run:
To execute the tests run:
To run correctly, some Kubernetes integration tests require specific configuration or runtime arguments of the Kubernetes cluster they run against. These tests will be skipped if the cluster configuration is incorrect. Below is a sample configuration for Kubernetes clusters that would commonly be used on a developer workstation:
minikube delete minikube config set container-runtime containerd minikube config set feature-gates "ProcMountType=true" minikube start
k3s server --tls-san=k3s --kube-apiserver-arg=feature-gates=ProcMountType=true
If you are developing functionality inside a helper, you’ll most likely want to run tests with the version of the Docker image that contains the newest changes.
If you run tests without passing
-ldflags, the default version in
This means that the runner defaults to pulling a helper image
make targets inject
-ldflags automatically. You can run all tests by using:
make targets also inject
parallel_test_execute, which is most commonly used by the CI/CD jobs.
In case you want a more customized
go test command, you can use
go test -ldflags "$(make print_ldflags)" -run TestDockerCommandBuildCancel -v ./executors/docker/...
Currently, GoLand doesn’t support dynamic Go tool arguments, so you’ll need to run
make print_ldflags first
and then paste it in the configuration.
Build the newest version of the helper image with:
Then you’ll have the image ready for use:
REPOSITORY TAG IMAGE ID CREATED SIZE gitlab/gitlab-runner-helper x86_64-a6bc0800 f10d9b5bbb41 32 seconds ago 57.2MB
If you are running a local Kubernetes cluster make sure to reuse the cluster’s Docker daemon to build images. For example, with minikube:
eval $(minikube docker-env)
golangci-lint, used for the
vale, used for the
Installation instructions will pop up when running a Makefile target if a tool is missing.
You can start hacking
If you need an IDE to edit and debug code, there are a few free suggestions you can use:
- JetBrains GoLand IDE.
- Visual Studio Code using the
workspace recommended extensions,
GitLab Runner uses Go Modules to manage its dependencies.
Don’t add dependency from upstream default branch when version tags are available.
Unit test files have a suffix of
_test.goand contain the following build directive in the header:
// go:build !integration
Integration test files have a suffix of
_integration_test.goand contain the following build directive in the header:
// go:build integration
They can be run by adding
To test the state of the build directives in test files,
make check_test_directives can be used.
The following are required:
Which virtual machine to use depends on your use case:
- The Windows Server machine has Docker pre-installed and should always be used when you are developing on GitLab Runner for Windows.
- The Windows 10 machine is there for you to have a windows environment with a GUI which sometimes can help you debugging some Windows features. Note that you cannot have Docker running inside of Windows 10 because nested virtualization is not supported.
vagrant up windows_10 will start the Windows 10 machine for
- SSH inside of the Windows 10 machine, run
vagrant ssh windows_10.
- Access the GUI for the Windows 10, you can connect via
RDP by running
vagrant rdp windows_10, which will connect to the machine using a locally installed RDP program.
For both machines, the GitLab Runner source code is synced
bi-directionally so that you can edit from your machine with your
favorite editor. The source code can be found under the
environment variable. We have a
RUNNER_SRC environment variable which
you can use to find out the full path so when using PowerShell,
you can use