GitLab-managed Kubernetes resources

Tier: Premium, Ultimate
Offering: GitLab.com, GitLab Self-Managed, GitLab Dedicated

Use GitLab-managed Kubernetes resources to provision Kubernetes resources with environment templates. An environment template can:

Create namespaces and service accounts automatically for new environments
Manage access permissions through role bindings
Configure other required Kubernetes resources

When developers deploy applications, GitLab creates the resources based on the environment template.

Configure GitLab-managed Kubernetes resources

Prerequisites:

You must have a configured GitLab agent for Kubernetes.
You have authorized the agent to access relevant projects or groups.
(Optional) You have configured agent impersonation to prevent privilege escalations. The default environment template assumes you have configured ci_job impersonation.

Turn on Kubernetes resource management

In your agent configuration file

To turn on resource management, modify the agent configuration file to include the required permissions:

ci_access:
  projects:
    - id: <your_group/your_project>
      access_as:
        ci_job: {}
      resource_management:
        enabled: true
  groups:
    - id: <your_other_group>
      access_as:
        ci_job: {}
      resource_management:
        enabled: true

In your CI/CD jobs

To have the agent manage resources for an environment, specify the agent in your deployment job. For example:

deploy_review:
  stage: deploy
  script:
    - echo "Deploy a review app"
  environment:
    name: review/$CI_COMMIT_REF_SLUG
    kubernetes:
      agent: path/to/agent/project:agent-name

CI/CD variables can be used in the agent path. For more information, see Where variables can be used.

Create environment templates

Environment templates define what Kubernetes resources are created, updated, or removed.

The default environment template creates a Namespace and configures a RoleBinding for the CI/CD job.

To overwrite the default template, add a template configuration file called default.yaml in the agent directory:

.gitlab/agents/<agent-name>/environment_templates/default.yaml

Supported Kubernetes resources

The following Kubernetes resources (kind) are supported:

Namespace
ServiceAccount
RoleBinding
FluxCD Source Controller objects:
- GitRepository
- HelmRepository
- HelmChart
- Bucket
- OCIRepository
FluxCD Kustomize Controller objects:
- Kustomization
FluxCD Helm Controller objects:
- HelmRelease
FluxCD Notification Controller objects:
- Alert
- Provider
- Receiver

Example environment template

The following example creates a namespace and grants a group administrator access to a cluster.

objects:
  - apiVersion: v1
    kind: Namespace
    metadata:
      name: '{{ .environment.slug }}-{{ .project.id }}-{{ .agent.id }}'
  - apiVersion: rbac.authorization.k8s.io/v1
    kind: RoleBinding
    metadata:
      name: bind-{{ .environment.slug }}-{{ .project.id }}-{{ .agent.id }}
      namespace: '{{ .environment.slug }}-{{ .project.id }}-{{ .agent.id }}'
    subjects:
      - kind: Group
        apiGroup: rbac.authorization.k8s.io
        name: gitlab:project_env:{{ .project.id }}:{{ .environment.slug }}
    roleRef:
      apiGroup: rbac.authorization.k8s.io
      kind: ClusterRole
      name: admin

# Resource lifecycle configuration
apply_resources: on_start    # Resources are applied when environment is started/restarted
delete_resources: on_stop    # Resources are removed when environment is stopped

Template variables

Environment templates support limited variable substitution. The following variables are available:

Category	Variable	Description	Type	Default value when not set
Agent	`{{ .agent.id }}`	The agent ID.	Integer	N/A
Agent	`{{ .agent.name }}`	The agent name.	String	N/A
Agent	`{{ .agent.url }}`	The agent URL.	String	N/A
Environment	`{{ .environment.id }}`	The environment ID.	Integer	N/A
Environment	`{{ .environment.name }}`	The environment name.	String	N/A
Environment	`{{ .environment.slug }}`	The environment slug.	String	N/A
Environment	`{{ .environment.url }}`	The environment URL.	String	Empty string
Environment	`{{ .environment.page_url }}`	The environment page URL.	String	N/A
Environment	`{{ .environment.tier }}`	The environment tier.	String	N/A
Project	`{{ .project.id }}`	The project ID.	Integer	N/A
Project	`{{ .project.slug }}`	The project slug.	String	N/A
Project	`{{ .project.path }}`	The project path.	String	N/A
Project	`{{ .project.url }}`	The project URL.	String	N/A
CI/CD Pipeline	`{{ .ci_pipeline.id }}`	The pipeline ID.	Integer	Zero
CI/CD Job	`{{ .ci_job.id }}`	The CI/CD job ID.	Integer	Zero
User	`{{ .user.id }}`	The user ID.	Integer	N/A
User	`{{ .user.username }}`	The username.	String	N/A

All variables should be referenced using the double curly brace syntax, for example: {{ .project.id }}. See text/template documentation for more information on the templating system used.

Resource lifecycle management

Use the following settings to configure when Kubernetes resources should be removed:

# Never delete resources
delete_resources: never

# Delete resources when environment is stopped
delete_resources: on_stop

The default value is on_stop, which is specified in the default environment template.

Managed resource labels and annotations

The resources created by GitLab use a series of labels and annotations for tracking and troubleshooting purposes.

The following labels are defined on every resource created by GitLab. The values are intentionally left empty:

agent.gitlab.com/id-<agent_id>: ""
agent.gitlab.com/project_id-<project_id>: ""
agent.gitlab.com/env-<gitlab_environment_slug>-<project_id>-<agent_id>: ""
agent.gitlab.com/environment_slug-<gitlab_environment_slug>: ""

On every resource created by GitLab, an agent.gitlab.com/env-<gitlab_environment_slug>-<project_id>-<agent_id> annotation is defined. The value of the annotation is a JSON object with the following keys:

Key	Description
`environment_id`	The GitLab environment ID.
`environment_name`	The GitLab environment name.
`environment_slug`	The GitLab environment slug.
`environment_url`	The link to the environment. Optional.
`environment_page_url`	The link to the GitLab environment page.
`environment_tier`	The GitLab environment deployment tier.
`agent_id`	The agent ID.
`agent_name`	The agent name.
`agent_url`	The agent URL in the agent registration project.
`project_id`	The GitLab project ID.
`project_slug`	The GitLab project slug.
`project_path`	The full GitLab project path.
`project_url`	The link to the GitLab project.
`template_name`	The name of the template used.

Disable GitLab-managed Kubernetes resources

You can disable GitLab-managed Kubernetes resources for specific environments while still using other Kubernetes features like the dashboard. Disabling managed resources helps when working with global agents that have managed resources enabled by default, but you need to opt out specific projects or environments.

To disable managed resources for an environment, add the managed_resources.enabled: false configuration:

deploy_review:
  stage: deploy
  script:
    - echo "Deploy a review app"
  environment:
    name: review/$CI_COMMIT_REF_SLUG
    kubernetes:
      agent: path/to/agent/project:agent-name
      managed_resources:
        enabled: false

Troubleshooting

Any errors related to managed Kubernetes resources can be found on:

The environment page in your GitLab project
The CI/CD job logs when using the feature in pipelines