• Summary
• Motivation
  • Goals
  • Non-Goals
• Proposal
  • Overview
  • Existing components and relationships
  • Issues
  • Example
  • Further details
    • Multi-Project Deployment Pipelines
• Design and implementation details
  • Associate Environment with Agent
  • Fetch resources through user_access
  • Dependency graph
  • Health status of resources

View and manage resources deployed by GitLab Agent For Kuberenetes

Summary

As part of the GitLab Kubernetes Dashboard epic, users want to view and manage their resources deployed by GitLab Agent For Kuberenetes. Users should be able to interact with the resources through GitLab UI, such as Environment Index/Details page.

This blueprint describes how the association is established and how these domain models interact with each other.

Motivation

Goals

• The proposed architecture can be used in GitLab Kubernetes Dashboard.
• The proposed architecture can be used in Organization-level Environment dashboard.
• The cluster resources and events can be visualized per GitLab Environment. An environment-specific view scoped to the resources managed either directly or indirectly by a deployment commit.
• Support both GitOps mode and CI Access mode.

Non-Goals

• The design details of GitLab Kubernetes Dashboard and Organization-level Environment dashboard.
• Support Environment/Deployment features that rely on GitLab CI/CD pipelines, such as Protected Environments, Deployment Approvals, Deployment safety, and Environment rollback. These features are already available in CI Access mode, however, it’s not available in GitOps mode.

Proposal

Overview

• GitLab Environment and GitLab Agent For Kubernetes have 1-to-1 relationship.
• GitLab Environment tracks all resources produced by the connected agent. This includes not only resources written in manifest files but also subsequently generated resources (for example, Pods created by Deployment manifest file).
• GitLab Environment renders dependency graph, such as Deployment => ReplicaSet => Pod. This is for providing ArgoCD-style resource view.
• GitLab Environment has the Resource Health status that represents a summary of resource statuses, such as Healthy, Progressing or Degraded.

Existing components and relationships

• GitLab Project and GitLab Environment have 1-to-many relationship.
• GitLab Project and Agent have 1-to-many direct relationship. Only one project can own a specific agent.
• GitOps mode
  • GitLab Project and Agent do NOT have many-to-many indirect relationship yet. This will be supported in Manifest projects outside of the Agent configuration project.
• CI Access mode
  • GitLab Project and Agent have many-to-many indirect relationship. The project owning the agent can share the access with the other proejcts. (NOTE: Technically, only running jobs inside the project are allowed to access the cluster due to job-token authentication.)

Issues

• GitLab Environment should have ID of GitLab Agent For Kubernetes as the foreign key.
• GitLab Environment should have parameters how to group resources in the associated cluster, for example, namespace, lable and inventory-id (GitOps mode only) can passed as parameters.
• GitLab Environment should be able to fetch all relevant resources, including both default resource kinds and other Custom Resources.
• GitLab Environment should be aware of dependency graph.
• GitLab Environment should be able to compute Resource Health status from the associated resources.

Example

This is an example of how the architecture works in push-based deployment. The feature is documented here as CI access mode.

Further details

Multi-Project Deployment Pipelines

The microservice project setup can be improved by Multi-Project Deployment Pipelines:

• Deployment Project can behave as the shared deployment engine for any upstream application projects and environments.
• Environments can be created within the application projects. It gives more visibility of environments for developers.
• Deployment Project can be managed under Operator group. More segregation of duties.
• Users don’t need to setup RBAC to restrict CI/CD jobs.
• This is especitially helpful for dynamic environments, such as Review Apps.

Design and implementation details

Associate Environment with Agent

Users can explicitly set a GitLab Agent For Kubernetes to a GitLab Environment in setting UI. Frontend will use this associated agent for authenticating/authorizing the user access, which is described in a latter section.

We need to adjust the read_cluster_agent permission in DeclarivePolicy for supporting agents shared by an external project (also known as the Agent management project).

Fetch resources through user_access

When user visits an environment page, GitLab frontend fetches an environment via GraphQL. Frontend additionally fetches the associated agent-ID and namespace.

Here is an example of GraphQL query:

{
  project(fullPath: "group/project") {
    id
    environment(name: "<environment-name>") {
      slug
      kubernetesNamespace
      clusterAgent {
        id
        name
        project {
          name
        }
      }
    }
  }
}

GitLab frontend authenticate/authorize the user access with browser cookie. If the access is forbidden, frontend shows an error message that You don't have access to an agent that deployed to this environment. Please contact agent administrator if you are allowed in "user_access" in agent config file. See <troubleshooting-doc-link>.

After the user gained access to the agent, GitLab frontend fetches specific Resource kinds (for example, Deployment, Pod) in the Kubernetes with the following parameters:

• namespace … #{environment.kubernetesNamespace}

If no resources are found, this is likely that the users have not embedded these lables into their resources. In this case, frontend shows an warning message There are no resources found for the environment. Do resources have GitLab preserved labels? See <troubleshooting-doc-link>.

Dependency graph

• GitLab frontend uses Owner References to idenfity the dependencies between resources. These are embedded in resources as metadata.ownerReferences field.
• For the resoruces that don’t have owner references, we can use Well-Known Labels, Annotations and Taints as complement. for example, EndpointSlice doesn’t have metadata.ownerReferences, but has kubernetes.io/service-name as a reference to the parent Service resource.

Health status of resources

• GitLab frontend computes the status summary from the fetched resources. Something similar to ArgoCD’s Resource Health for example, Healthy, Progressing, Degraded and Suspended. The formula is TBD.