Managing foundational agents
This guide covers foundational agents. For foundational flows, see the foundational flows guide. To understand the difference between agents and flows, see the glossary.
Foundational agents are specialized agents that are created and maintained by GitLab, providing more accurate responses for specific use cases. These agents are available by default on any place chat and GitLab Duo chat are available, including groups, and are supported on GitLab Duo Self-Hosted.
Create a foundational agent
There are two ways of creating a foundational agent, using the AI Catalog or GitLab Duo Workflow Service. AI Catalog provides a user-friendly interface, and it is the preferred approach, but writing a definition on GitLab Duo Workflow Service provides more flexibility for complex cases.
Using the AI catalog
Create your agent on the AI Catalog, and note its ID. Make sure the agent is set to public. Example: Planner Agent has ID 348.
Agents created on the AI Catalog need to be bundled into GitLab Duo Workflow Service, so they can be available to self-hosted setups that do not have access to our SaaS. To achieve this, open an MR to GitLab Duo Workflow Service adding the ID of the agent:
# https://gitlab.com/gitlab-org/modelops/applied-ml/code-suggestions/ai-assist/-/blob/main/Dockerfile - RUN poetry run fetch-foundational-agents "https://gitlab.com" "$GITLAB_TOKEN" "348" \ + RUN poetry run fetch-foundational-agents "https://gitlab.com" "$GITLAB_TOKEN" "duo_planner:348,<agent-reference>:<agent-catalog-id>" \The command above can also be executed locally for testing purposes. Agent reference must be lowercase without spaces (example: ’test_agent’).
To make the agent be selectable, add it to the
FoundationalChatAgentsDefinitions.rb. Use the reference used in the Dockerfile:{ id: 3, reference: '<agent-reference>', version: 'experimental', name: 'Test Agent', description: "An agent for testing" }Update user facing documentation.
Using GitLab Duo Workflow Service
Create a flow configuration file in
/duo_workflow_service/agent_platform/v1/flows/configs/(located either on your GDK underPATH-TO-YOUR-GDK/gdk/gitlab-ai-gatewayor on the ai-assist repository):File:
/duo_workflow_service/agent_platform/v1/flows/configs/foundational_pirate_agent/1.0.0.ymlversion: "v1" environment: chat-partial components: - name: "foundational_pirate_agent" type: AgentComponent prompt_id: "foundational_pirate_agent_prompt" inputs: - from: "context:goal" as: "goal" - from: "context:project_id" as: "project_id" toolset: [] ui_log_events: [] prompts: - name: Foundational Pirate Agent prompt_id: "foundational_pirate_agent_prompt" model: params: model_class_provider: anthropic max_tokens: 2_000 prompt_template: system: | You are a seasoned pirate from the Golden Age of Piracy. You speak exclusively in pirate dialect, using nautical terms, pirate slang, and colorful seafaring expressions. Transform any input into authentic pirate speak while maintaining the original meaning. Use terms like 'ahoy', 'matey', 'ye', 'aye', 'landlubber', 'scallywag', 'doubloons', 'plunder', etc. Add pirate exclamations like 'Arrr!', 'Shiver me timbers!', and 'Yo ho ho!' where appropriate. Refer to yourself in the first person as a pirate would. user: | {{goal}} placeholder: history routers: [] flow: entry_point: "foundational_pirate_agent"Add your agent definition to FoundationalChatAgentsDefinitions.rb:
# frozen_string_literal: true module Ai module FoundationalChatAgentsDefinitions extend ActiveSupport::Concern ITEMS = [ { id: 1, reference: 'chat', version: '', name: 'GitLab Duo Agent', description: "GitLab Duo is your general development assistant" }, { id: 2, reference: 'foundational_pirate_agent', version: 'v1', name: 'Foundational Pirate Agent', description: "A most important agent that speaks like a pirate" } ].freeze end endUpdate user facing documentation.
Tips:
- You can use the AI Catalog to test foundational agents, even before you add them to the codebase. Create a new private agent in the AI Catalog with the same prompt and same tools, and enable it on your test project. Once results reach desired levels, add to GitLab Duo Workflow Service.
- Add prompts to the GitLab Duo Workflow Service to enable testing the agent in your local GDK.
- When using AI catalog, the version field of an agent in
FoundationalChatAgentsDefinitions.rbshould beexperimental. When creating the definition in GitLab Duo Workflow Service, the version should bev1.
Secret-safety requirements for agent prompts
If your foundational agent’s scope includes any of the following, you must include secret-safety guidance in its system prompt:
- Generating or modifying files (for example,
.gitlab-ci.yml, configuration files, scripts). - CI/CD pipeline configuration or conversion.
- Handling credentials, API keys, tokens, or connection strings.
Required prompt guidance
Add the following instructions verbatim to the agent’s system prompt:
Never write literal secret values (API keys, tokens, passwords, connection strings, or any credentials)
into files or repository content. Always substitute secrets with CI/CD variable references
(for example, $API_KEY, $DB_PASSWORD, $DEPLOY_TOKEN). If a user provides a secret value directly,
do not echo it into any file — instead, recommend storing it in Settings > CI/CD > Variables and
reference it as a variable. When converting pipelines from other CI systems (for example, Jenkins,
GitHub Actions, CircleCI) that contain hardcoded secrets, replace those values with variable
references and flag to the user that the original pipeline contained hardcoded secrets.Checklist
Before merging a new foundational agent, confirm:
- Agent prompt reviewed for file-writing, CI/CD configuration, or credential-handling scope.
- If in scope: secret-safety guidance added verbatim to the system prompt.
- If not in scope: documented in the MR description with reasoning.
Use feature flags for releasing chat agents
Control the release of new foundational agents with feature flags:
# ee/app/graphql/resolvers/ai/foundational_chat_agents_resolver.rb
def resolve(*, project_id: nil, namespace_id: nil)
project = GitlabSchema.find_by_gid(project_id)
filtered_agents = []
filtered_agents << 'foundational_pirate_agent' if Feature.disabled?(:my_feature_flag, project)
# filtered_agents << 'foundational_pirate_agent' if Feature.disabled?(:my_feature_flag, current_user)
::Ai::FoundationalChatAgent
.select {|agent| filtered_agents.exclude?(agent.reference) }
.sort_by(&:id)
endThis also allows making a foundational agent available to a specific tier.
Scoping
Not every agent is useful in every area. For example, some agents operate in projects, while others are more useful or have more capabilities in groups. Scoping is not supported. See issue 577395.
Triggers
Triggers are not supported for foundational chat agents. However, if they are defined on AI Catalog, users can still add it to their project at which point they can be used through triggers.
Versioning
Pin a foundational agent to a specific flow version with the flow_version attribute in
FoundationalChatAgentsDefinitions.rb.
The value is a semantic version constraint that GitLab Duo Workflow Service uses to select a
flow config from the bundled versions. Use version pinning to iterate on foundational agents
without breaking existing GitLab Self-Managed and GitLab Dedicated customers.
{
id: 2,
reference: 'foundational_pirate_agent',
version: 'v1',
flow_version: '^1.0.0',
name: 'Foundational Pirate Agent',
description: "A most important agent that speaks like a pirate"
}When flow_version is set, the monolith sends flow_config_id, flow_config_schema_version, and
flow_version to GitLab Duo Workflow Service when starting a workflow. The service then resolves
the matching flow config from its bundled versions.
Use a constraint that matches the compatibility you need:
^1.0.0accepts any1.x.yrelease (recommended for most agents).~1.2.0accepts any1.2.xrelease.1.2.3pins to an exact version.
Choose the version increment based on the change:
- Increment the major version for breaking changes (for example, expecting a new required input).
- Increment the minor version for backwards-compatible additions (for example, a new optional parameter).
- Increment the patch version for bug fixes.
Version pinning is only available for agents defined in GitLab Duo Workflow Service.
Agents backed by an AI Catalog item resolve their version from the catalog item and
ignore flow_version.
Without flow_version, GitLab Duo Workflow Service falls back to its default resolution.
Consider potential breaking changes to older GitLab versions before changing an agent.
Context variables
Context variables let you inject runtime information into a Duo Workflow Service agent’s system prompt. Use them to make prompt sections conditional or pass more information — for example, customizing the prompt based on the user location, or pass data from a form.
Context variables are only supported for agents defined in GitLab Duo Workflow Service. Agents created in the AI Catalog cannot use context variables except orbit_enabled
The end-to-end flow is:
- The GitLab monolith builds an
additional_contextpayload and includes it in thestartWorkflowrequest. - GitLab Duo Workflow Service reads the
additional_context, resolves the values to Jinja2 variables using the componentinputsmapping, and pre-renders the system prompt. - Jinja2 variables in the prompt (for example,
{% if my_variable %}...{% endif %}) are evaluated with the resolved values. Unknown variables like{{ goal }}are preserved and resolved in a later pass.
Define context variables in the flow config
In your flow YAML
(/duo_workflow_service/agent_platform/v1/flows/configs/<agent_name>/<version>.yml), declare:
- A
flow.inputsentry that describes the variable’s category and schema. - A component
inputsentry that maps the variable using thecontext:inputs.<category>.<field>path.
components:
- name: "my_agent"
type: AgentComponent
prompt_id: "my_agent_prompt"
inputs:
- from: "context:goal"
as: "goal"
- from: "context:inputs.my_context.my_first_var"
as: "my_first_var"
optional: true # optional: true prevents an error when the variable is absent
- from: "context:inputs.my_context.my_second_var"
as: "my_second_var"
optional: true
toolset: []
ui_log_events: []
flow:
inputs:
- category: my_context
input_schema:
my_var:
type: boolean
description: Whether the feature is available for this user
entry_point: "my_agent"Use context variables in the prompt template
Use Jinja2 {% if %} blocks in the prompt_template.system field to conditionally include prompt sections:
prompts:
- name: My Agent
prompt_id: "my_agent_prompt"
prompt_template:
system: |
You are a helpful agent.
Additional information {{ my_first_var }}
{% if my_second_var %}
<my_feature_integration>
You also have access to the my feature API. Use it when the user asks about...
</my_feature_integration>
{% endif %}
user: |
{{goal}}Wire context variables from the GitLab monolith
Context variables reach the Duo Workflow Service through the additionalContext field of the
startWorkflow call. Each entry has a category, a content field (a JSON string), and metadata.
In duo_agentic_chat_state_manager.vue, inject the context envelope only when a foundational agent
is active:
const mergedAdditionalContext =
this.selectedFoundationalAgent && goal
? [
{
category: 'my_context',
content: JSON.stringify({ my_first_var: 1, my_second_var: this.myFeatureEnabled }),
metadata: '{}',
},
...(additionalContext || []).filter((c) => c.category !== 'my_context'),
]
: additionalContext || [];Filter internal categories from display and GraphQL
Internal context categories (for example, my_context) must not be shown in the UI or serialized
through GraphQL. Custom category names are not registered in the AiAdditionalContextCategory enum,
and serialization causes errors.
Filter them in ee/app/assets/javascripts/ai/duo_agentic_chat/utils/workflow_utils.js:
const INTERNAL_CATEGORIES = new Set(['my_context']);
msg.extras = {
contextItems: msg.additional_context.filter((c) => !INTERNAL_CATEGORIES.has(c.category)),
};Strip them in ee/app/presenters/ai/duo_workflows/workflow_checkpoint_event_presenter.rb before
building DuoMessage objects:
INTERNAL_CONTEXT_CATEGORIES = %w[my_context].freeze
if msg['additional_context'].is_a?(Array)
msg['additional_context'] = msg['additional_context'].reject do |ctx|
INTERNAL_CONTEXT_CATEGORIES.include?(ctx['category'])
end
endDeveloping foundational agents locally
For AI catalog created agents, you need to sync the agents locally. To do so, either create the agent in the local AI Catalog or on GitLab.com AI Catalog.
Fetch agents from GitLab.com
On
$GDK/gitlab-ai-gateway, run the following command:poetry run fetch-foundational-agents "http://gdk.test:3000 or https://gitlab.com" "<token-to-your-local-gdk>" \ "<agent-reference>:<agent-id-in-local-catalog>" --flow-registry-version v1An example pulling
duo_plannerandsecurity_analyst_agentfrom GitLab.com would look like this:poetry run fetch-foundational-agents "https://gitlab.com" "<token-to-your-local-gdk>" \ "duo_planner:348,security_analyst_agent:356" --flow-registry-version v1Where:
348is the GitLab Duo Planner catalog ID on GitLab.com356is the Security Analyst Agent catalog ID on GitLab.com
After fetching the configurations, restart the service:
gdk restart duo-workflow-serviceVerify the setup
Foundational agents are saved in
$GDK/gitlab-ai-gateway/duo_workflow_service/agent_platform/v1/flows/configs/as.ymlfiles.For example if you used the above
poetrycommand to pullduo_plannerandsecurity_analyst_agent, you can run the following:ls duo_workflow_service/agent_platform/v1/flows/configs/ | grep -e "duo_planner" -e "security_analyst"You then should see the following output:
duo_planner.yml security_analyst_agent.ymlAlternatively to check in the GDK UI:
- With the changes to
FoundationalChatAgentsDefinitions.rb, you can now select your foundational agent in the web chat locally. - Verify that you can see and interact with the foundational agents
- Test sending a message to confirm the agents respond correctly
- With the changes to
Troubleshooting
- Agents don’t appear in chat: Verify the configuration files were created in your GitLab-ai-gateway directory and the service restarted successfully
- Permission errors: Ensure your GitLab.com API token has the API scope
- Flow registry version errors: Confirm you’re using
--flow-registry-version v1
Testing foundational agent synchronization pipeline
This section describes how to test the pipeline used to sync foundational agents in your local GDK. For developing the foundational flows or pulling the latest flows refer to the Developing foundational agents locally section above.
Prerequisites
- A running GDK instance
- A GitLab API token with
apiscope ($GDK_PAT_WITH_API_SCOPE) - Access to the
gitlab-ai-gatewayrepository in your GDK
Step 1: Check existing foundational agents
First, identify which foundational agents are defined in the monolith but missing from your local AI Catalog:
Check the foundational agents definitions:
# In your GDK's gitlab directory cat ee/lib/ai/foundational_chat_agents_definitions.rbList existing agents in your local AI Catalog:
curl --header "Authorization: Bearer $GDK_PAT_WITH_API_SCOPE" \ --header "Content-Type: application/json" \ "http://gdk.test:3000/api/graphql" \ --data '{"query": "query { aiCatalogItems { nodes { id name description } } }"}'Compare the results to identify missing foundational agents (typically
duo_plannerandsecurity_analyst_agent).
Step 2: Create missing foundational agents
If foundational agents are missing from your local AI Catalog, create them programmatically:
Get a project ID for hosting the agents:
If you’ve run the duo setup script with
bundle exec rake gitlab:duo:setup, you can use the project with ID1000000as the foundational agents owning project. If not, you can pick any Premium or Ultimate project in your GDK and use that project’s ID.# If you haven't run the duo setup script, get any project ID curl --header "Authorization: Bearer $GDK_PAT_WITH_API_SCOPE" \ "http://gdk.test:3000/api/v4/projects" | jq '.[0].id'Create the Planner agent:
curl --header "Authorization: Bearer $GDK_PAT_WITH_API_SCOPE" \ --header "Content-Type: application/json" \ "http://gdk.test:3000/api/graphql" \ --data '{ "query": "mutation { aiCatalogAgentCreate(input: { projectId: \"gid://gitlab/Project/YOUR_PROJECT_ID\", name: \"Planner\", description: \"Get help with planning and workflow management. Organize, edit, create, and track work more effectively in GitLab.\", public: true, systemPrompt: \"You are a helpful planning assistant that helps users organize, edit, create, and track work more effectively in GitLab.\" }) { item { id name } errors } }" }'Create the Security Analyst agent:
curl --header "Authorization: Bearer $GDK_PAT_WITH_API_SCOPE" \ --header "Content-Type: application/json" \ "http://gdk.test:3000/api/graphql" \ --data '{ "query": "mutation { aiCatalogAgentCreate(input: { projectId: \"gid://gitlab/Project/YOUR_PROJECT_ID\", name: \"Security Analyst\", description: \"Automate vulnerability management and security workflows. The Security Analyst Agent acts as an AI team member that can autonomously analyze, triage, and remediate security vulnerabilities.\", public: true, systemPrompt: \"You are a security analyst AI that helps with vulnerability management and security workflows. You can analyze, triage, and help remediate security vulnerabilities.\" }) { item { id name } errors } }" }'Replace
YOUR_PROJECT_IDwith the actual project ID from step 1.
Step 3: Get the local agent IDs
After creating the agents, get their local catalog IDs:
# Get Planner agent ID
curl --header "Authorization: Bearer $GDK_PAT_WITH_API_SCOPE" \
--header "Content-Type: application/json" \
"http://gdk.test:3000/api/graphql" \
--data '{"query": "query { aiCatalogItems(search: \"Planner\") { nodes { id name } } }"}'
# Get Security Analyst agent ID
curl --header "Authorization: Bearer $GDK_PAT_WITH_API_SCOPE" \
--header "Content-Type: application/json" \
"http://gdk.test:3000/api/graphql" \
--data '{"query": "query { aiCatalogItems(search: \"Security Analyst\") { nodes { id name } } }"}'Note the numeric IDs from the responses (for example, 10 and 11).
Step 4: Fetch foundational agent configurations
In your gitlab-ai-gateway directory, fetch the agent configurations using the local IDs:
# For v1 flow registry (recommended)
poetry run fetch-foundational-agents "http://gdk.test:3000" "$GDK_PAT_WITH_API_SCOPE" \
"duo_planner:10,security_analyst_agent:11" \
--flow-registry-version v1
# For experimental flow registry (alternative)
poetry run fetch-foundational-agents "http://gdk.test:3000" "$GDK_PAT_WITH_API_SCOPE" \
"duo_planner:10,security_analyst_agent:11" \
--flow-registry-version experimental \
--output-path duo_workflow_service/agent_platform/experimental/flows/configsReplace 10 and 11 with the actual agent IDs from step 3.
Step 5: Restart GitLab Duo Workflow Service
gdk restart duo-workflow-serviceStep 6: Verify the setup
With the changes to FoundationalChatAgentsDefinitions.rb and the fetched configurations, you can now select your foundational agents in the web chat locally.
Troubleshooting
Missing agents: If agents don’t appear in chat after following these steps, verify:
- The agents exist in your local AI Catalog (check with the GraphQL query from Step 3)
- The flow configuration files were created in the correct directory after running
fetch-foundational-agents - The GitLab Duo Workflow Service was restarted successfully
Flow registry version: Use
v1for production-like behavior,experimentalfor testing new featuresPermission errors: Ensure your API token has the
apiscope and sufficient project permissionsGraphQL errors: Check the exact mutation parameters using GraphQL introspection:
curl --header "Authorization: Bearer $GDK_PAT_WITH_API_SCOPE" \ --header "Content-Type: application/json" \ "http://gdk.test:3000/api/graphql" \ --data '{"query": "query { __type(name: \"AiCatalogAgentCreatePayload\") { fields { name type { name } } } }"}'
Integration testing foundational agents
Foundational agents have an integration test harness that runs the full agent loop end-to-end using real LLM calls. Use it to verify that an agent correctly selects tools, passes the right arguments, and produces valid responses — without needing a live GitLab instance or real tool backends.
The tests live in the ai-assist repository and run as CI jobs (for example, the Data Analyst agent tests run on changes to the agent prompt, otherwise manual).
Key concepts
- Real LLM calls: Both the agent execution and response validation use actual model calls, so tests catch regressions in prompt behavior, not just string matching.
- Mockable tools: Tool responses can be stubbed so tests are deterministic and fast.
- Fluent assertion API: Chain assertions like
assert_called_toolandassert_llm_validatesto express expected behavior clearly.
Example test
@pytest.mark.asyncio
async def test_how_many_open_issues(
analytics_agent,
initial_state,
mock_gitlab_client,
):
"""Agent must call run_glql_query and report the count."""
mock_glql_response(mock_gitlab_client, glql_response(SAMPLE_ISSUES, count=42))
result = await ask_agent(
analytics_agent,
initial_state,
"How many open issues are there in the gitlab-org group?",
)
(result.assert_has_tool_calls().assert_called_tool("run_glql_query"))
await result.assert_llm_validates(
[
"Response says 42 open issues",
]
)ask_agent runs the agent loop with the given prompt and returns a result object.
assert_called_tool verifies the agent invoked the expected tool.
assert_llm_validates asks an LLM to check the response against plain-language criteria,
so you don’t need brittle substring matches.
CI configuration
Integration test jobs are defined in .gitlab/ci/test.gitlab-ci.yml. Available configuration variables:
EXECUTION_MODEL: The model used to run the agent during the test.VALIDATION_MODEL: The model used byassert_llm_validatesto judge responses.
Getting started
To reuse or extend this harness for your agent, see these merge requests for reference:
Architecture design
Foundational Chat Agents are developed by GitLab and must be available to all GitLab deployments (GitLab.com, Self-Managed, and Dedicated).
The architecture of how Foundational Agents are made available avoids connecting to AI Catalog to fetch definitions at runtime and allows GitLab engineering teams full control over when they are released.
This design could also be extended to support Foundational flows.
Foundational Agents in Monolith
Defining foundational agents in the monolith serves two purposes: backwards compatibility support and release control.
With FoundationalChatAgentsDefinitions
The FoundationalChatAgentsDefinitions
module manages agent versioning based on the GitLab instance version.
affecting older GitLab versions, similar to prompt versioning.
Additionally, on FoundationalChatAgentsResolver,
teams are able to select which conditions can make a foundational chat agent available, for situations like:
- does the user have Ultimate,
- is the feature flag enabled,
- is the agent SaaS exclusive
If we relied exclusively on AI Catalog or Duo Workflow Service, such flexibility wouldn’t be possible
Version resolution
Agent versions are resolved based on the version field in FoundationalChatAgentsDefinitions.rb,
which maps to a
folder in GitLab Duo Workflow Service (for example, v1, experimental).
In the future, version resolution will be based on semantic versioning. This will allow:
- Patch and minor updates (bug fixes, performance improvements, prompt refinements) to be shipped to existing GitLab versions without requiring a GitLab instance update
- Major version releases for breaking changes that require new GitLab features (such as new tools, API changes, or schema modifications) to be shipped only to compatible GitLab versions
This approach ensures backward compatibility while enabling continuous improvement of foundational agents.
Bundling into GitLab Duo Workflow Service
Bundling agents into GitLab Duo Workflow Service makes agents defined in AI Catalog available on all deployments, including self-hosted setups. With semantic versioning support, the latest version of each major release will be bundled, along with specific pinned versions of each foundational agent.
The alternative to this would be to ship the YAML definitions themselves as part of GitLab monolith, but that comes with the downside of not being able to quickly ship fixes to cloud-connected self-managed instances.
Eventually, if labels are implemented on AI Catalog, teams wouldn’t need to add their entries to the Dockerfile, versions could be fetched by the correct labels.
Creation flow
%%{init: {"sequence": {"actorMargin": 50}}}%%
sequenceDiagram
accTitle: Foundational agent creation flow
accDescr: Sequence diagram showing the process of creating a foundational agent from AI Catalog through to GitLab monolith
participant Team
participant AI Catalog
participant DWS Repo as DWS Repository
participant CI
participant Monolith
Team->>AI Catalog: Create foundational agent
Team->>DWS Repo: Add agent ID to Dockerfile
DWS Repo->>CI: Trigger build
CI->>AI Catalog: Pull agent definitions
AI Catalog->>CI: Returns all required versions
CI->>CI: Store definitions in DWS image
CI->>CI: Ships images with definitions
Team->>Monolith: Add agent to FoundationalChatAgentsDefinitions.rb
Usage flow
%%{init: {"sequence": {"actorMargin": 50}}}%%
sequenceDiagram
accTitle: Foundational agent usage flow
accDescr: Sequence diagram showing how users interact with foundational agents through GitLab monolith and Duo Workflow Service
participant User
participant Monolith
participant DWS as GitLab Duo Workflow Service
User->>Monolith: Request to foundational agent
Monolith->>DWS: Request specific agent version
DWS->>DWS: Resolve agent version
DWS->>DWS: Process request
DWS->>Monolith: Return response
Monolith->>User: Return response
The execution flows are the same whether the user is using a local monolith, GitLab.com, the cloud-connected DWS or a local installation of DWS.