Configure flow execution

Flows use agents to execute tasks.

• Flows executed from the GitLab UI use CI/CD.
• Flows executed in an IDE run locally.

You can configure the environment where flows use CI/CD to execute. You can also use your own runners, and specify variables in your jobs.

Executor architecture

When a flow runs in CI/CD, the runner:

1. Downloads the GitLab Duo CLI binary for its operating system and architecture from the GitLab package registry. Node.js and npm are not needed.
2. Runs the GitLab Duo CLI, which uses WebSocket to connect to the GitLab Duo Workflow Service.
3. Executes tools (file operations, Git commands) as directed by the AI model.

The executor version is managed by GitLab and updated as part of regular releases.

Configure CI/CD execution

To customize how flows are executed in CI/CD, create an agent configuration file in your project.

For a list of supported keys and their types, see the agent-config.yml reference.

Create the agent configuration file

You can generate the agent configuration file with GitLab Duo, or create it manually.

The configuration is applied when flows run in CI/CD for your project.

For an example of a complete agent-config.yml file, see the agent-config.yml reference.

Configure setup scripts

You can define setup scripts that run before your flow executes. This is useful for installing dependencies, configuring environments, or initialization.

To add setup scripts, in the agent-config.yml file, add the following commands:

setup_script:
  - apt-get update && apt-get install -y curl
  - pip install -r requirements.txt
  - echo "Setup complete"

These commands complete the following actions:

• Run before the main workflow commands.
• Execute in the order specified.
• Can be a single command or an array of commands.

The user context for setup_script depends on the Docker image. The default GitLab image runs as root. Custom images run as the user defined in the image’s USER directive. If your setup_script requires root access (for example, to install system packages), ensure your custom image is configured accordingly.

Configure caching

To configure caching to speed up subsequent flow runs, configure the agent-config.yml file to preserve files and directories between executions. Caching can be useful for dependency folders like node_modules or Python virtual environments.

Basic cache configuration

To cache specific paths, add the following to your agent-config.yml file:

cache:
  paths:
    - node_modules/
    - .npm/

Cache with keys

You can use cache keys to create different caches for different scenarios. Cache keys help ensure that the cache is based on your project’s state.

Use a string key

cache:
  key: my-project-cache
  paths:
    - vendor/
    - .bundle/

Use file-based cache keys

Create dynamic cache keys based on file contents (like lock files). When these files change, a new cache is created. This generates a SHA checksum of the specified files:

cache:
  key:
    files:
      - package-lock.json
      - yarn.lock
  paths:
    - node_modules/

Use a prefix with file-based keys

Combine a prefix with the SHA computed for the cache key files:

cache:
  key:
    files:
      - package-lock.json
    prefix: $CI_JOB_NAME
  paths:
    - node_modules/
    - .npm/

In this example, if the job name is test and the SHA checksum is abc123, the cache key becomes test-abc123.

Cache limitations

• You can specify up to 10 files for cache key generation. If more files are specified, only the first 10 are used.
• The cache paths field is required. A cache configuration without paths has no effect.
• Cache keys support CI/CD variables in the prefix field.

Start sessions from a reference repository

The runner clones your whole repository at the start of every flow. For a large repository, this clone can take most of the time before the flow starts.

To make sessions start faster, build an image that contains a bare copy of your repository, and point the agent configuration file at it. Each session then fetches only the commits made after the image was built, and reads older objects from the copy in the image. For example, on the gitlab-org/gitlab project, the clone dropped from about 4 minutes to 14 seconds.

The image is larger than one without the repository, so it takes longer to pull. Use a reference repository when the clone takes longer than the extra pull time. The longer the image goes without a rebuild, the more commits each session fetches.

Some foundational flows always use the GitLab image. These flows ignore git_strategy.

Prerequisites:

• The runner that executes flows must be GitLab Runner 17.3 or later.
• The Maintainer or Owner role for the project.
• An image for your flows that includes Git 2.31 or later.
• The bare repository in the image must be at an absolute path outside /builds. The runner mounts the build directory over /builds.
• The bare repository must have at least one branch. A session sends the branches of the reference repository to GitLab to determine which commits it already has. Without a branch, every session downloads the whole history.

To start sessions from a reference repository:

1. In your repository, create a .gitlab/duo/session-image.Dockerfile file. Use the image your flows use now as the base image:

   dockerfile

   FROM <your-current-image>
   
   COPY ref.git /opt/duo/ref.git

2. In your .gitlab-ci.yml file, add a job that copies the default branch into a bare repository, then builds and pushes the image:

   yaml

   build-duo-session-image:
     image: docker:27-cli
     services:
       - docker:27-dind
     variables:
       DOCKER_TLS_CERTDIR: "/certs"
       GIT_STRATEGY: clone
       GIT_DEPTH: 0
       GIT_FETCH_EXTRA_FLAGS: "--prune --quiet"
     rules:
       - if: $CI_PIPELINE_SOURCE == "schedule" && $SCHEDULE_TYPE == "duo-session-image"
     script:
       - apk add --no-cache git
       - git init --quiet --bare duo-image/ref.git
       - git -C duo-image/ref.git fetch --quiet --no-tags "$CI_PROJECT_DIR" "$CI_COMMIT_SHA:refs/heads/$CI_DEFAULT_BRANCH"
       - echo "$CI_REGISTRY_PASSWORD" | docker login --username "$CI_REGISTRY_USER" --password-stdin "$CI_REGISTRY"
       - docker build --file .gitlab/duo/session-image.Dockerfile --tag "$CI_REGISTRY_IMAGE/duo-session:latest" duo-image
       - docker push "$CI_REGISTRY_IMAGE/duo-session:latest"

   The GIT_* variables make the runner clone the full, unfiltered history, so the bare repository is complete. The job copies the repository it already cloned, so the image build needs no credentials. The job uses Docker-in-Docker, which needs a runner with privileged mode. For more information, see use Docker to build Docker images. If other jobs in your pipeline run in scheduled pipelines, add rules so they don’t run in this one.

3. Create a pipeline schedule for the default branch, for example weekly, with the variable SCHEDULE_TYPE set to duo-session-image.

4. Run the schedule once to build the first image.

5. In the agent-config.yml file, set the image and the path to the bare repository:

   yaml

   image: registry.gitlab.com/my-group/my-project/duo-session:latest
   git_strategy:
     reference_repository_path: /opt/duo/ref.git

   If the flow cannot pull the image from the container registry, see configure flows to pull the image from the GitLab container registry.

Sessions now start from the reference repository.

Verify that sessions use the reference repository

To check that a flow uses the reference repository, start a flow and open the job log. The log includes the following line:

Skipping Git repository setup and creating an empty build directory

If the path does not exist in the image the flow runs in, or is not a bare repository, the job clones the repository without the reference repository, and the log also includes:

Reference repository not found at /opt/duo/ref.git, cloning without it

The flow still runs, but without the faster start. Check the path, and that the flow uses the image from the agent configuration file.

Configure ID tokens

To authenticate with third-party services from a flow, configure ID tokens.

ID tokens are JSON web tokens (JWT) that GitLab CI/CD generates and injects into the job that runs the flow for keyless OpenID Connect (OIDC) authentication without storing long-lived credentials. For example, you can use ID tokens to retrieve secrets from a secrets manager or sign binaries and Git commits.

To configure ID tokens, in the agent-config.yml file, add an id_tokens block. Each token requires an aud (audience) claim:

id_tokens:
  VAULT_ID_TOKEN:
    aud: https://vault.example.com

network_policy:
  allowed_domains:
    - vault.example.com

The aud claim can be a single string or a list of strings:

id_tokens:
  MY_ID_TOKEN:
    aud:
      - https://first.service.example.com
      - https://second.service.example.com

network_policy:
  allowed_domains:
    - first.service.example.com
    - second.service.example.com

Each token is available in the flow job as an environment variable that uses the name of the token. For the previous examples, the flow can use $VAULT_ID_TOKEN and $MY_ID_TOKEN.

If a token name matches a variable name declared elsewhere in your configuration, the ID token takes precedence.

For more information about the token payload and how to configure trust with third-party services, see OpenID Connect (OIDC) authentication using ID tokens.

Configure runners to execute flows

Flows that use CI/CD run on runners.

On GitLab.com, flows can use hosted runners, which GitLab provides. These are enabled by default.

You also have the option to configure your own runner for flows.

Prerequisites:

• Your runner must meet the resource requirements.

To configure your own runner for flows:

1. Create an instance runner or a group runner assigned to the top-level group. If you want flows to use project runners or group runners assigned to a subgroup, turn off the duo_runner_restrictions feature flag (GitLab Self-Managed only).

2. Add the gitlab--duo tag to the runner so that it picks up jobs for flows. If the runner does not have this tag, jobs with flows remain queued indefinitely. Use any of the following methods:

   • When you create the runner, in the Tags field, enter gitlab--duo.

   • For an existing runner, edit the jobs the runner can run and enter gitlab--duo in the Tags field.

   • If you configure runners with a config.toml file, add the tag to the [[runners]] section:

     toml

     [[runners]]
       executor = "docker"
       tags = ["gitlab--duo"]

3. Configure the runner to use an executor that supports Docker images, like docker, docker-autoscaler, or kubernetes. The shell executor is not supported.

4. If your top-level group has turned on IP address restrictions, add the runner’s IP address to your group’s IP allowlist so the runner can access the group.

5. GitLab Self-Managed only. Ensure the runner can reach the services that flows require:

   • Allow outbound connections from the GitLab instance to the Agent Platform.
   • Allow outbound connections from the runner to the Agent Platform.
   • For instances with self-signed certificates in the certificate chain, complete the additional GitLab Duo CLI configuration.

Runner resource requirements

The GitLab Duo CLI runs in headless mode in each CI/CD job that executes a flow. Each job needs the following resources for the CLI:

GitLab-hosted runners meet these requirements.

These values cover only the GitLab Duo CLI. Each job also needs resources for the following:

• Your project’s repository
• Commands the flow runs, like setup scripts, builds, and tests
• Large images that the flow reads

A PNG or JPEG file larger than 2 MiB, or wider or taller than 2000 pixels, can use up to 1 GiB of additional memory.

Large repositories and long-running flows use more memory. For these flows, use at least the recommended values. If a job runs out of memory, it fails with exit code 137. On Kubernetes, the container status is OOMKilled.

Flow duration depends mostly on AI model response time, so additional CPU has little effect.

For more information about how these values are measured, see GitLab Duo CLI system requirements.

Verify the runner configuration

To verify that the runner is configured correctly for flows:

1. In the top bar, select Search or go to and find your project.
2. In the left sidebar, select Settings > General.
3. Expand GitLab Duo.
4. In the CI/CD runner row, check the status.

The check verifies that the runner is online and has the correct tag, executor, and scope.

To rerun the check after you register or fix a runner, select Check again.

Use the execution environment sandbox to secure flows

For network and file system isolation, use the execution environment sandbox to secure flows executed on runners.

To use the sandbox, you must use one of the following images:

• Default Docker base image for the Agent Platform
• Hardened UBI 9 Minimal image
• A custom image with SRT installed

To configure runners to use the sandbox, set privileged = true in your runner configuration.

For example:

[[runners]]
  executor = "docker"
  tags = ["gitlab--duo"]
  [runners.docker]
    privileged = true

You cannot use the sandbox with custom images that do not have SRT installed.

Sandbox requirements without privileged mode

Privileged mode is one way to meet the sandbox requirements, but it is not sufficient on its own and is not always necessary.

The sandbox’s requirement is that the job can create user and mount namespaces. At runtime, the sandbox probes for unshare -m first, then falls back to unshare -rm:

• unshare -m requires CAP_SYS_ADMIN, which a container running as root has when the runner is configured with privileged = true.
• unshare -rm creates an unprivileged user namespace and works for images that run as a non-root user, such as the hardened UBI 9 Minimal image. unshare -rm maps the container user to root inside the new namespace only. It grants no privilege on the runner host.

A privileged = true setting does not guarantee the sandbox initializes. It still fails when:

• The runner host has user.max_user_namespaces set to 0.
• A seccomp or AppArmor profile blocks the CLONE_NEWUSER flag.
• Privileged mode is set on the runner but does not reach the job container. This can happen with some docker-autoscaler and kubernetes executor configurations.

If no namespace mode is available, the flow runs without the sandbox and the job log contains the following warning:

Warning: SRT found but can't create sandbox (insufficient privileges), running command directly without sandbox