Configure GitLab to access self-hosted models

Tier: For a limited time, Premium and Ultimate. In the future, GitLab Duo Enterprise. Offering: Self-managed Status: Experiment
History
The availability of this feature is controlled by a feature flag. For more information, see the history.
caution
This feature is considered experimental and is not intended for customer usage outside of initial design partners. We expect major changes to this feature.
This page contains information related to upcoming products, features, and functionality. It is important to note that the information presented is for informational purposes only. Please do not rely on this information for purchasing or planning purposes. The development, release, and timing of any products, features, or functionality may be subject to change or delay and remain at the sole discretion of GitLab Inc.

To configure your GitLab instance to access the available self-hosted large language models (LLMs) in your infrastructure:

  1. Configure the self-hosted model.
  2. Configure the GitLab Duo AI-powered features to use your self-hosted models.

Configure the self-hosted model

Prerequisites:

  • You must be an administrator.

To configure a self-hosted model:

  1. On the left sidebar, at the bottom, select Admin.
  2. Select AI-powered features.
    • If the AI-powered features menu item is not available, synchronize your subscription after purchase:
      1. On the left sidebar, select Subscription.
      2. In Subscription details, to the right of Last sync, select synchronize subscription ().
  3. Select Models.
  4. Set your model details by selecting New self-hosted model.
  5. Complete the fields:
    • Enter the model name, for example, Mistral.
    • From the Model dropdown list, select the model. Only GitLab-approved models are listed here.
    • For Endpoint, select the self-hosted model endpoint, for example, the server hosting the model.
    • Optional. For API token, add an API key if you need one to access the model.
  6. Select Create model.

Configure the features to your models

Prerequisites:

  • You must be an administrator.

To configure the AI-powered features to use your model:

  1. On the left sidebar, at the bottom, select Admin.
  2. Select AI-powered features.
    • If the AI-powered features menu item is not available, synchronize your subscription after purchase:
      1. On the left sidebar, select Subscription.
      2. In Subscription details, to the right of Last sync, select synchronize subscription ().
  3. Select Features.
  4. For the feature you want to set, select Edit. For example, Code Generation.
  5. Select the model provider for the feature:
    • From the list, select Self-Hosted Model.
    • Choose the self-hosted model you would like to use, for example, Mistral.
  6. Select Save Changes.