GitLab Duo Self-Hosted configuration and authentication

Tier: Ultimate with GitLab Duo Enterprise - Start a trial
Offering: GitLab Self-Managed
Status: Beta
History

There are two configuration options for self-managed customers:

  • GitLab.com AI gateway: Use the GitLab-managed AI gateway with default external large language model (LLM) providers (for example, Google Vertex or Anthropic).
  • Self-hosted AI gateway: Deploy and manage your own AI gateway and language models in your infrastructure, without depending on GitLab-provided external language providers.

GitLab.com AI gateway

In this configuration, your GitLab instance depends on and sends requests to the external GitLab AI gateway, which communicates with external AI vendors such as Google Vertex or Anthropic. The response is then forwarded back to your GitLab instance.

GitLab AI Vendor (External)GitLab AI gateway (External)Self-hosted GitLab (Your Instance)GitLab AI Vendor (External)GitLab AI gateway (External)Self-hosted GitLab (Your Instance)UserSend requestCheck if self-hosted model is configuredForward request for AI processingCreate prompt and send request to AI model serverRespond to the promptForward AI responseForward AI responseUser

Self-hosted AI gateway

In this configuration, the entire system is isolated within the enterprise, ensuring a fully self-hosted environment that safeguards data privacy.

Self-hosted modelSelf-hosted AI gatewaySelf-hosted GitLabSelf-hosted modelSelf-hosted AI gatewaySelf-hosted GitLabUserSend requestCheck if self-hosted model is configuredCreate prompt and perform request to AI model serverForward request for AI processingRespond to the promptForward AI responseForward AI responseUser

For more information, see the self-hosted model deployment blueprint.

Authentication for GitLab Duo Self-Hosted

The authentication process for GitLab Duo Self-Hosted is secure, efficient, and made up of the following key components:

  • Self-issued tokens: In this architecture, access credentials are not synchronized with cloud.gitlab.com. Instead, tokens are self-issued dynamically, similar to the functionality on GitLab.com. This method provides users with immediate access while maintaining a high level of security.

  • Offline environments: In offline setups, there are no connections to cloud.gitlab.com. All requests are routed exclusively to the self-hosted AI gateway.

  • Token minting and verification: The instance mints the token, which is then verified by the AI gateway against the GitLab instance.

  • Model configuration and security: When an administrator configures a model, they can incorporate an API key to authenticate requests. Additionally, you can enhance security by specifying connection IP addresses within your network, ensuring that only trusted IPs can interact with the model.

As illustrated in the following diagram:

  1. The authentication flow begins when the user configures the model through the GitLab instance and submits a request to access the GitLab Duo feature.
  2. The GitLab instance mints an access token, which the user forwards to GitLab and then to the AI gateway for verification.
  3. Upon confirming the token’s validity, the AI gateway sends a request to the AI model, which uses the API key to authenticate the request and process it.
  4. The results are then relayed back to the GitLab instance, completing the flow by sending the response to the user, which is designed to be secure and efficient.
AI ModelAI gatewayGitLab InstanceUserAI ModelAI gatewayGitLab InstanceUserConfigure ModelRequest AccessMint TokenSend TokenForward Minted TokenVerify TokenToken ValidatedSend Request to ModelSend Request to ModelAuthenticate using API KeyProcess RequestSend Result to GitLabSend Response