Configure GitLab to access GitLab Duo Self-Hosted

Prerequisites:

• Upgrade GitLab to version 17.9 or later.

To configure your GitLab instance to access the available self-hosted models in your infrastructure:

1. Confirm that a fully self-hosted configuration is appropriate for your use case.
2. Configure your GitLab instance to access the AI gateway.
3. Configure the self-hosted model.
4. Configure the GitLab Duo features to use your self-hosted model.

Configure your GitLab instance to access the AI gateway

1. On the left sidebar, at the bottom, select Admin.
2. Select GitLab Duo.
3. In the GitLab Duo section, select Change configuration.
4. Under Local AI Gateway URL, enter your AI Gateway URL.
5. Select Save changes.

Configure the self-hosted model

Prerequisites:

• You must be an administrator.
• You must have an Ultimate license.
• You must have a Duo Enterprise license add-on.

To configure a self-hosted model:

1. On the left sidebar, at the bottom, select Admin.

2. Select GitLab Duo Self-Hosted.

   • If the GitLab Duo Self-Hosted menu item is not available, synchronize your subscription after purchase:
     1. On the left sidebar, select Subscription.
     2. In Subscription details, to the right of Last sync, select synchronize subscription ( ).

3. Select Add self-hosted model.

4. Complete the fields:

   • Deployment name: Enter a name to uniquely identify the model deployment, for example, Mixtral-8x7B-it-v0.1 on GCP.

   • Model family: Select the model family the deployment belongs to. Only GitLab-approved models are in this list.

   • Endpoint: Enter the URL where the model is hosted.

     • For more information about configuring the endpoint for models deployed through vLLM, see the vLLM documentation.

   • API key: Optional. Add an API key if you need one to access the model.

   • Model identifier: This is a required field. The value of this field is based on your deployment method, and should match the following structure:

     Deployment method	Format	Example
     vLLM	custom_openai/<name of the model served through vLLM>	custom_openai/Mixtral-8x7B-Instruct-v0.1
     Bedrock	bedrock/<model ID of the model>	bedrock/mistral.mixtral-8x7b-instruct-v0:1
     Azure OpenAI	azure/<model ID of the model>	azure/gpt-35-turbo

     For more information about configuring the model identifier for models deployed through vLLM, see the vLLM documentation.

5. Select Create self-hosted model.

Configure self-hosted beta models and features

Prerequisites:

• You must be an administrator.
• You must have an Ultimate license.
• You must have a Duo Enterprise license add-on.

To enable self-hosted beta models and features:

1. On the left sidebar, at the bottom, select Admin.
2. Select GitLab Duo.
3. In the GitLab Duo section, select Change configuration.
4. Under Self-hosted beta models and features, select the Use beta models and features in GitLab Duo Self-Hosted checkbox.
5. Select Save changes.

For more information, see the list of available beta models under evaluation.

Configure GitLab Duo features to use self-hosted models

Prerequisites:

• You must be an administrator.
• You must have an Ultimate license.
• You must have a Duo Enterprise license add-on.

View configured features

1. On the left sidebar, at the bottom, select Admin.
2. Select GitLab Duo Self-Hosted.
   • If the GitLab Duo Self-Hosted menu item is not available, synchronize your subscription after purchase:
     1. On the left sidebar, select Subscription.
     2. In Subscription details, to the right of Last sync, select synchronize subscription ( ).
3. Select the AI-powered features tab.

Configure the feature to use a self-hosted model

Configure the GitLab Duo feature and sub-feature to send queries to the configured self-hosted model:

1. On the left sidebar, at the bottom, select Admin.

2. Select GitLab Duo Self-Hosted.

3. Select the AI-powered features tab.

4. For the feature and sub-feature you want to configure, from the dropdown list, choose the self-hosted model you want to use.

   For example, for the code generation sub-feature under GitLab Duo Code Suggestions, you can select claude sonnet on bedrock (Claude 3).

   

GitLab Duo Chat sub-feature fall back configuration

When configuring GitLab Duo Chat sub-features, if you do not select a specific self-hosted model for a sub-feature, that sub-feature automatically falls back to using the self-hosted model configured for General Chat. This ensures all Chat functionality works even if you have not explicitly configured each sub-feature with its own self-hosted model.

Disabling GitLab Duo features

To disable a feature, you must explicitly select Disabled when configuring a feature or sub-feature.

• Not choosing a model for a sub-feature is insufficient.
• For Chat sub-features, not selecting a model causes that sub-feature to fall back to using the model configured for General Chat.

To disable a GitLab Duo feature or sub-feature:

1. On the left sidebar, at the bottom, select Admin.

2. Select GitLab Duo Self-Hosted.

3. Select the AI-powered features tab.

4. For the feature or sub-feature you want to disable, from the dropdown list, select Disabled.

   For example, to specifically disable the Write Test and Refactor Code features, select Disabled: