# Configure flow execution

{{< details >}}

- Tier: [Free](../../../../subscriptions/gitlab_credits.md#for-the-free-tier), Premium, Ultimate
- Offering: GitLab.com, GitLab Self-Managed, GitLab Dedicated

{{< /details >}}

{{< history >}}

- [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/477166) in GitLab 18.3.

{{< /history >}}

Flows use agents to execute tasks.

- Flows executed from the GitLab UI use CI/CD.
- Flows executed in an IDE run locally.

You can configure the environment where flows use CI/CD to execute.
You can also [use your own runners](#configure-runners-to-execute-flows), and
[specify variables in your jobs](execution-variables.md).

## Executor architecture

{{< history >}}

- [Changed](https://gitlab.com/gitlab-org/gitlab/-/issues/600436) to a precompiled binary instead of an `npm` package in GitLab 19.4.

{{< /history >}}

When a flow runs in CI/CD, the runner:

1. Downloads the GitLab Duo CLI binary for its operating system and architecture from the
   GitLab package registry. Node.js and `npm` are not needed.
1. Runs the GitLab Duo CLI, which uses WebSocket to connect to the GitLab Duo Workflow Service.
1. Executes tools (file operations, Git commands) as directed by the AI model.

The executor version is managed by GitLab and updated as part of regular releases.

## Configure CI/CD execution

To customize how flows are executed in CI/CD, create an agent configuration file in your project.

For a list of supported keys and their types, see the [`agent-config.yml` reference](agent-config-yaml.md).

> [!note]
> You cannot use predefined CI/CD variables to configure the `agent-config.yml`.
> You must use [variables](execution-variables.md#available-variables) for jobs that execute your flows.

### Create the agent configuration file

{{< history >}}

- Generating the agent configuration file with GitLab Duo [introduced](https://gitlab.com/groups/gitlab-org/-/work_items/22591) in GitLab 19.5 [with a feature flag](../../../../administration/feature_flags/_index.md) named `duo_agent_readiness_settings`. Enabled by default.
- Generating the agent configuration file with GitLab Duo [generally available](https://gitlab.com/gitlab-org/gitlab/-/work_items/607567) in GitLab 19.6. Feature flag `duo_agent_readiness_settings` removed.

{{< /history >}}

You can generate the agent configuration file with GitLab Duo, or create it manually.

{{< tabs >}}

{{< tab title="Generate with GitLab Duo" >}}

Prerequisites:

- The Maintainer or Owner role for the project.
- Flow execution turned on for the project.

To generate the file:

1. In the top bar, select **Search or go to** and find your project.
1. In the left sidebar, select **Settings** > **General**.
1. Expand **GitLab Duo**.
1. In the **Agent configuration file** row, select **Generate**.
   GitLab Duo generates the file and opens a merge request to add it to your project.
1. Review the merge request and merge it when ready.

If the file is already on the default branch, or a merge request that adds it is open,
the **Agent configuration file** row includes a link to it instead.

{{< /tab >}}

{{< tab title="Create manually" >}}

To create the file manually:

1. In your project's repository, create a `.gitlab/duo/` folder.
1. In the folder, create a configuration file named `agent-config.yml`.
1. Add your required configuration options.
1. Commit and push the file to your default branch.

{{< /tab >}}

{{< /tabs >}}

The configuration is applied when flows run in CI/CD for your project.

For an example of a complete `agent-config.yml` file, see the [`agent-config.yml` reference](agent-config-yaml.md#complete-example).

> [!note]
> The configuration file is read-only from the project's default branch.
> Files committed to other branches are ignored, even when a flow runs from those branches.

### Configure setup scripts

You can define setup scripts that run before your flow executes. This is useful for installing dependencies, configuring environments, or initialization.

To add setup scripts, in the `agent-config.yml` file, add the following commands:

```yaml
setup_script:
  - apt-get update && apt-get install -y curl
  - pip install -r requirements.txt
  - echo "Setup complete"
```

These commands complete the following actions:

- Run before the main workflow commands.
- Execute in the order specified.
- Can be a single command or an array of commands.

The user context for `setup_script` depends on the Docker image. The default
GitLab image runs as `root`. Custom images run as the user defined in the
image's `USER` directive. If your `setup_script` requires root access (for
example, to install system packages), ensure your custom image is configured
accordingly.

> [!warning]
> `setup_script` commands run before SRT is applied and execute outside it.
> These commands have access to all environment variables in the flow, including
> the triggering user's OAuth token, service token, and identity details.
> For the security model and recommended protections, see
> [security implications of `agent-config.yml`](security-considerations.md#security-implications-of-agent-configyml).

### Configure caching

To configure caching to speed up subsequent flow runs, configure the `agent-config.yml` file to
preserve files and directories between executions. Caching can be useful for dependency folders like `node_modules` or Python virtual environments.

#### Basic cache configuration

To cache specific paths, add the following to your `agent-config.yml` file:

```yaml
cache:
  paths:
    - node_modules/
    - .npm/
```

#### Cache with keys

You can use cache keys to create different caches for different scenarios. Cache keys help ensure that the cache is based on your project's state.

##### Use a string key

```yaml
cache:
  key: my-project-cache
  paths:
    - vendor/
    - .bundle/
```

##### Use file-based cache keys

Create dynamic cache keys based on file contents (like lock files). When these files change, a new cache is created. This generates a SHA checksum of the specified files:

```yaml
cache:
  key:
    files:
      - package-lock.json
      - yarn.lock
  paths:
    - node_modules/
```

##### Use a prefix with file-based keys

Combine a prefix with the SHA computed for the cache key files:

```yaml
cache:
  key:
    files:
      - package-lock.json
    prefix: $CI_JOB_NAME
  paths:
    - node_modules/
    - .npm/
```

In this example, if the job name is `test` and the SHA checksum is `abc123`, the cache key becomes `test-abc123`.

#### Cache limitations

{{< history >}}

- File limit for cache key generation [changed](https://gitlab.com/gitlab-org/gitlab/-/work_items/631954) to 10 in GitLab 19.5 [with a feature flag](../../../../administration/feature_flags/_index.md) named `increase_ci_cache_key_files_limit`. Disabled by default.

{{< /history >}}

> [!flag]
> The availability of this feature is controlled by a feature flag.
> For more information, see the history.

- You can specify up to 10 files for cache key generation. If more files are specified, only the first 10 are used.
- The cache `paths` field is required. A cache configuration without paths has no effect.
- Cache keys support CI/CD variables in the `prefix` field.

### Start sessions from a reference repository

{{< details >}}

- Status: Beta

{{< /details >}}

{{< history >}}

- [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/262094) as a [beta](../../../../policy/development_stages_support.md#beta) in GitLab 19.6 [with a feature flag](../../../../administration/feature_flags/_index.md) named `dap_reference_repository`. Disabled by default.

{{< /history >}}

> [!flag]
> The availability of this feature is controlled by a feature flag.
> For more information, see the history.

The runner clones your whole repository at the start of every flow.
For a large repository, this clone can take most of the time before the flow starts.

To make sessions start faster, build an image that contains a bare copy of your repository,
and point the agent configuration file at it. Each session then fetches only the commits made
after the image was built, and reads older objects from the copy in the image.
For example, on the `gitlab-org/gitlab` project, the clone dropped from about 4 minutes to 14 seconds.

The image is larger than one without the repository, so it takes longer to pull.
Use a reference repository when the clone takes longer than the extra pull time.
The longer the image goes without a rebuild, the more commits each session fetches.

Some foundational flows always use the GitLab image.
These flows ignore `git_strategy`.

Prerequisites:

- The runner that executes flows must be GitLab Runner 17.3 or later.
- The Maintainer or Owner role for the project.
- An image for your flows that includes Git 2.31 or later.
- The bare repository in the image must be at an absolute path outside `/builds`.
  The runner mounts the build directory over `/builds`.
- The bare repository must have at least one branch. A session sends the branches of the
  reference repository to GitLab to determine which commits it already has. Without a branch,
  every session downloads the whole history.

To start sessions from a reference repository:

1. In your repository, create a `.gitlab/duo/session-image.Dockerfile` file.
   Use the image your flows use now as the base image:

   ```dockerfile
   FROM <your-current-image>

   COPY ref.git /opt/duo/ref.git
   ```

1. In your `.gitlab-ci.yml` file, add a job that copies the default branch into a
   bare repository, then builds and pushes the image:

   ```yaml
   build-duo-session-image:
     image: docker:27-cli
     services:
       - docker:27-dind
     variables:
       DOCKER_TLS_CERTDIR: "/certs"
       GIT_STRATEGY: clone
       GIT_DEPTH: 0
       GIT_FETCH_EXTRA_FLAGS: "--prune --quiet"
     rules:
       - if: $CI_PIPELINE_SOURCE == "schedule" && $SCHEDULE_TYPE == "duo-session-image"
     script:
       - apk add --no-cache git
       - git init --quiet --bare duo-image/ref.git
       - git -C duo-image/ref.git fetch --quiet --no-tags "$CI_PROJECT_DIR" "$CI_COMMIT_SHA:refs/heads/$CI_DEFAULT_BRANCH"
       - echo "$CI_REGISTRY_PASSWORD" | docker login --username "$CI_REGISTRY_USER" --password-stdin "$CI_REGISTRY"
       - docker build --file .gitlab/duo/session-image.Dockerfile --tag "$CI_REGISTRY_IMAGE/duo-session:latest" duo-image
       - docker push "$CI_REGISTRY_IMAGE/duo-session:latest"
   ```

   The `GIT_*` variables make the runner clone the full, unfiltered history, so the bare
   repository is complete. The job copies the repository it already cloned, so the image
   build needs no credentials. The job uses Docker-in-Docker, which needs a runner with
   privileged mode. For more information, see
   [use Docker to build Docker images](../../../../ci/docker/using_docker_build.md).
   If other jobs in your pipeline run in scheduled pipelines, add rules so they don't run in this one.

1. [Create a pipeline schedule](../../../../ci/pipelines/schedules.md#create-a-pipeline-schedule)
   for the default branch, for example weekly, with the variable `SCHEDULE_TYPE` set to `duo-session-image`.
1. Run the schedule once to build the first image.
1. In the `agent-config.yml` file, set the image and the path to the bare repository:

   ```yaml
   image: registry.gitlab.com/my-group/my-project/duo-session:latest
   git_strategy:
     reference_repository_path: /opt/duo/ref.git
   ```

   If the flow cannot pull the image from the container registry, see
   [configure flows to pull the image from the GitLab container registry](images.md#configure-flows-to-pull-the-image-from-the-gitlab-container-registry).

Sessions now start from the reference repository.

#### Verify that sessions use the reference repository

To check that a flow uses the reference repository, start a flow and open the job log.
The log includes the following line:

```plaintext
Skipping Git repository setup and creating an empty build directory
```

If the path does not exist in the image the flow runs in, or is not a bare repository,
the job clones the repository without the reference repository, and the log also includes:

```plaintext
Reference repository not found at /opt/duo/ref.git, cloning without it
```

The flow still runs, but without the faster start. Check the path, and that the flow uses
the image from the agent configuration file.

### Configure ID tokens

{{< history >}}

- [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/224940) in GitLab 19.2.

{{< /history >}}

To authenticate with third-party services from a flow, configure
[ID tokens](../../../../ci/secrets/id_token_authentication.md).

ID tokens are JSON web tokens (JWT) that GitLab CI/CD generates and injects into the job that runs the flow
for keyless OpenID Connect (OIDC) authentication without storing long-lived credentials.
For example, you can use ID tokens to retrieve secrets from a secrets manager or sign binaries and Git commits.

To configure ID tokens, in the `agent-config.yml` file, add an `id_tokens` block.
Each token requires an `aud` (audience) claim:

```yaml
id_tokens:
  VAULT_ID_TOKEN:
    aud: https://vault.example.com

network_policy:
  allowed_domains:
    - vault.example.com
```

The `aud` claim can be a single string or a list of strings:

```yaml
id_tokens:
  MY_ID_TOKEN:
    aud:
      - https://first.service.example.com
      - https://second.service.example.com

network_policy:
  allowed_domains:
    - first.service.example.com
    - second.service.example.com
```

Each token is available in the flow job as an environment variable that uses the name of the token.
For the previous examples, the flow can use `$VAULT_ID_TOKEN` and `$MY_ID_TOKEN`.

If a token name matches a variable name declared elsewhere in your configuration, the ID token
takes precedence.

> [!warning]
> An ID token is a credential that grants access to any service that trusts its `aud` claim.
> Set the narrowest possible `aud` value for each token so that a compromised token can
> authenticate with as few services as possible. Because the configuration file is read from
> the default branch, apply the [recommended protections](security-considerations.md#recommended-protections) to control
> who can change which tokens a flow can request.

For more information about the token payload and how to configure trust with third-party services,
see [OpenID Connect (OIDC) authentication using ID tokens](../../../../ci/secrets/id_token_authentication.md).

## Configure runners to execute flows

Flows that use CI/CD run on runners.

On GitLab.com, flows can use [hosted runners](../../../../ci/runners/hosted_runners/_index.md), which GitLab provides. These are enabled by default.

You also have the option to configure your own runner for flows.

> [!note]
> If your top-level group has [IP address restrictions](../../../group/access_and_permissions.md#restrict-group-access-by-ip-address)
> enabled, hosted runners cannot be used for flows. Hosted runners use dynamic IP addresses
> from cloud provider pools that cannot be added to your group's IP allowlist. Instead,
> configure your own group runner at the top-level group.

Prerequisites:

- Your runner must meet the [resource requirements](#runner-resource-requirements).

To configure your own runner for flows:

1. Create an [instance runner](../../../../ci/runners/runners_scope.md) or a group runner assigned to the top-level group. If you want flows to use project runners or group runners assigned to a subgroup, turn off the `duo_runner_restrictions` feature flag (GitLab Self-Managed only).
1. Add the `gitlab--duo` tag to the runner so that it picks up jobs for flows. If the runner does not have this tag, jobs with flows remain queued indefinitely.
   Use any of the following methods:
   - When you create the runner, in the **Tags** field, enter `gitlab--duo`.
   - For an existing runner, [edit the jobs the runner can run](../../../../ci/runners/configure_runners.md#control-jobs-that-a-runner-can-run)
     and enter `gitlab--duo` in the **Tags** field.
   - If you configure runners with a `config.toml` file, add the tag to the `[[runners]]` section:

     ```toml
     [[runners]]
       executor = "docker"
       tags = ["gitlab--duo"]
     ```

1. Configure the runner to use an [executor](https://docs.gitlab.com/runner/executors/) that
   supports Docker images, like `docker`, `docker-autoscaler`, or `kubernetes`.
   The `shell` executor is not supported.
1. If your top-level group has turned on [IP address restrictions](../../../group/access_and_permissions.md#restrict-group-access-by-ip-address),
   add the runner's IP address to your group's IP allowlist so the runner can access the group.
1. GitLab Self-Managed only. Ensure the runner can reach the services that flows require:
   - [Allow outbound connections from the GitLab instance](../../../../administration/gitlab_duo/configure/_index.md#allow-outbound-connections-from-the-gitlab-instance-to-gitlab-duo) to the Agent Platform.
   - [Allow outbound connections from the runner](../../../../administration/gitlab_duo/configure/_index.md#allow-connections-from-the-runner) to the Agent Platform.
   - For instances with self-signed certificates in the certificate chain, complete the
     [additional GitLab Duo CLI configuration](../../../gitlab_duo_cli/use.md#certificate-errors).

### Runner resource requirements

The GitLab Duo CLI runs in headless mode in each CI/CD job that executes a flow.
Each job needs the following resources for the CLI:

| Resource | Minimum      | Recommended  |
|----------|--------------|--------------|
| CPU      | 1 vCPU       | 2 vCPU       |
| Memory   | 512 MiB      | 1 GiB        |
| Disk     | 500 MiB free | 500 MiB free |

GitLab-hosted runners meet these requirements.

These values cover only the GitLab Duo CLI.
Each job also needs resources for the following:

- Your project's repository
- Commands the flow runs, like setup scripts, builds, and tests
- Large images that the flow reads

A PNG or JPEG file larger than 2 MiB, or wider or taller than 2000 pixels, can use up to 1 GiB of additional memory.

Large repositories and long-running flows use more memory.
For these flows, use at least the recommended values.
If a job runs out of memory, it fails with exit code `137`.
On Kubernetes, the container status is `OOMKilled`.

Flow duration depends mostly on AI model response time, so additional CPU has little effect.

For more information about how these values are measured, see
[GitLab Duo CLI system requirements](https://gitlab.com/gitlab-org/editor-extensions/gitlab-lsp/-/blob/main/packages/cli/app/docs/system-requirements.md).

### Verify the runner configuration

{{< history >}}

- **CI/CD runner** check under project settings [introduced](https://gitlab.com/groups/gitlab-org/-/work_items/22591) in GitLab 19.5 [with a feature flag](../../../../administration/feature_flags/_index.md) named `duo_agent_readiness_settings`. Enabled by default.
- **CI/CD runner** check under project settings [generally available](https://gitlab.com/gitlab-org/gitlab/-/work_items/607567) in GitLab 19.6. Feature flag `duo_agent_readiness_settings` removed.

{{< /history >}}

To verify that the runner is configured correctly for flows:

1. In the top bar, select **Search or go to** and find your project.
1. In the left sidebar, select **Settings** > **General**.
1. Expand **GitLab Duo**.
1. In the **CI/CD runner** row, check the status.

The check verifies that the runner is online and has the correct tag, executor, and scope.

To rerun the check after you register or fix a runner, select **Check again**.

### Use the execution environment sandbox to secure flows

For network and file system isolation, use the [execution environment sandbox](../../environment_sandbox.md)
to secure flows executed on runners.

To use the sandbox, you must use one of the following images:

- Default Docker base image for the Agent Platform
- [Hardened UBI 9 Minimal image](images.md#use-a-red-hat-universal-base-image-9-minimal)
- A [custom image with SRT installed](../../environment_sandbox.md#install-anthropic-sandbox-runtime-srt-on-a-custom-image)

To configure runners to use the sandbox, set `privileged = true` in your [runner configuration](https://docs.gitlab.com/runner/configuration/advanced-configuration/).

For example:

```toml
[[runners]]
  executor = "docker"
  tags = ["gitlab--duo"]
  [runners.docker]
    privileged = true
```

You cannot use the sandbox with custom images that do not have SRT installed.

### Sandbox requirements without privileged mode

Privileged mode is one way to meet the sandbox requirements, but it is not sufficient
on its own and is not always necessary.

The sandbox's requirement is that the job can create user and mount namespaces.
At runtime, the sandbox probes for `unshare -m` first, then falls back to `unshare -rm`:

- `unshare -m` requires `CAP_SYS_ADMIN`, which a container running as root has when the
  runner is configured with `privileged = true`.
- `unshare -rm` creates an unprivileged user namespace and works for images that run as a
  non-root user, such as the hardened UBI 9 Minimal image. `unshare -rm` maps the container
  user to root inside the new namespace only. It grants no privilege on the runner host.

A `privileged = true` setting does not guarantee the sandbox initializes. It still fails when:

- The runner host has `user.max_user_namespaces` set to `0`.
- A seccomp or AppArmor profile blocks the `CLONE_NEWUSER` flag.
- Privileged mode is set on the runner but does not reach the job container. This can happen
  with some `docker-autoscaler` and `kubernetes` executor configurations.

If no namespace mode is available, the flow runs without the sandbox and the job log contains
the following warning:

```plaintext
Warning: SRT found but can't create sandbox (insufficient privileges), running command directly without sandbox
```

