Set up local development environment

Set up your local development environment to work on the Remote Development Workspaces features. You can choose between two setup modes depending on your development needs:

• Agent for workspace (agentw):

  • Simpler setup.
  • Uses direct agent communication.
  • Recommended for most development scenarios.

• GitLab Workspaces Proxy:

  • More complex setup.
  • Uses proxy for workspace communication.
  • Required for testing proxy-specific features.

Set up Kubernetes

1. Install Rancher Desktop 1.20.0.

2. In Rancher Desktop, select the Preferences icon.

3. Configure the virtual machine:

   • Go to Virtual Machine > Hardware and set minimum values of 4 CPUs and 8 GB RAM.

   • For macOS only:

     1. Go to Virtual Machine > Emulation
     2. Select VZ as the Virtual Machine Type.
     3. Select Enable Rosetta support.

4. Go to Container Engine, select containerd.

5. Go to Kubernetes:

   1. Select Kubernetes version v1.33.4.
   2. Clear the Enable Traefik checkbox.

Set up GDK

1. Install GDK.

2. Set the GDK_ROOT environment variable:

   echo 'export GDK_ROOT="/path/to/your/gdk"' >> ~/.zshrc

   Replace /path/to/your/gdk with your actual GDK directory path.

3. Set up an EE license.

4. Configure GDK to run on a local private IP address by following the local network binding documentation.

   For this setup, we assume the private IP address is 172.16.123.1. If you use a different IP address, substitute the correct value in later steps.

5. Configure NGINX for GDK:

   1. Add this configuration to your gdk.yml file:

      hostname: gdk.test
      nginx:
        enabled: true
        http:
            enabled: true

   2. Install NGINX:

      brew install nginx

6. Optional. Check out the desired branches on GitLab:

   cd "${GDK_ROOT}/gitlab"
   git checkout my_branch

   To prevent changes from being lost when you run gdk update, add this to your gdk.yml:

   gdk:
     auto_rebase_projects: true

7. Restart your GDK:

   cd "${GDK_ROOT}"
   gdk restart

Set up GitLab Agent Server (KAS) in GDK

1. Enable agent for Kubernetes in your GDK by adding this configuration to gdk.yml:

   gitlab_k8s_agent:
    enabled: true
    agent_listen_address: gdk.test:8150
    k8s_api_listen_address: gdk.test:8154

2. Reconfigure and restart GDK:

    cd "${GDK_ROOT}"
    gdk reconfigure
    gdk restart

3. Optional. Check out the desired branches on agent for Kubernetes:

         cd "${GDK_ROOT}/gitlab-k8s-agent"
         git checkout my_branch

4. Optional. To manually run kas, for example, to run in a different cloned directory or debug in an IDE:

   • See running kas and agentk locally.

   • For debugging with JetBrains GoLand IDE:

     1. Set up a kas “Run Configuration” “Run Kind: Directory'
     2. Point to /path/to/cmd/kas.
     3. Check Run after build.
     4. Pass all the same ENV vars and options as you do to Bazel.

Verify agent for Kubernetes installation

To ensure agent for Kubernetes is properly installed:

1. Install Xcode from the Apple App Store and accept the license:

    sudo xcodebuild -license accept

2. Clone the agent repository:

    git clone https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent

3. Test the installation:

    cd gitlab-agent
    make test

4. If you encounter errors, clean and retry:

    bazel clean --expunge
    make test

   You might also need to do this in the <GDK_ROOT>/gitlab-k8s-agent, which is used by the GDK.

When make test passes, agent for Kubernetes is ready to use.

Set up agent for Kubernetes (agentk) for Workspaces

1. Create an agent configuration file:

   1. Go to http://gdk.test:3000/gitlab-org.
   2. Create a private project named gitlab-agent-configurations with a README.
   3. In the project, create a file at .gitlab/agents/remotedev/config.yaml with your agentk configuration.
   When you create or change this file, you must start or restart agentk described in step 4.

2. Register agentk with GitLab:

   1. In the gitlab-agent-configurations project, go to Operate > Kubernetes clusters.
   2. Select Connect a cluster.
   3. Enter remotedev in the input field.
   4. Select Register.
   5. Copy and save the generated token. It is required for the AGENT_TOKEN environment variable.

3. Map the agent to the gitlab-org group:

   1. Go to http://gdk.test:3000/gitlab-org.
   2. Go to Settings > Workspaces.
   3. Select the All agents tab.
   4. Find remotedev in the list and select Allow.
   5. Verify the agent appears in the Allowed agents tab.

4. Start the Agent (agentk):

    export AGENT_TOKEN='your_copied_token'
    cd "${GDK_ROOT}/gitlab-k8s-agent"
    echo -n "$AGENT_TOKEN" > "$HOME/.gitlab-agentk-token.txt"
    export POD_NAMESPACE=default
    export POD_NAME=remotedev
    bazel run //cmd/agentk -- --kas-address=grpc://gdk.test:8150 --token-file=$HOME/.gitlab-agentk-token.txt

5. Optional. To manually run agentk, for example, to run in a different cloned dir or debug in an IDE:

   • See running kas and agentk locally.

   • For debugging with JetBrains GoLand IDE:

     1. Set up a kas “Run Configuration” “Run Kind: Directory'
     2. Point to /path/to/cmd/kas.
     3. Check Run after build.
     4. Pass all the same ENV vars and options as you do to Bazel.

Go to Operate > Kubernetes clusters. The remotedev agent should now display as Connected.

Configure Workspaces in GDK

For agent for workspace mode

Use this configuration in the .gitlab/agents/remotedev/config.yaml file:

remote_development:
  enabled: true
  network_policy:
    enabled: true
    egress:
    - allow: '0.0.0.0/0'
      except:
      - '10.0.0.0/8'
      - '172.16.0.0/12'
      - '192.168.0.0/16'
    - allow: '172.16.123.1/32'
  gitlab_workspaces_proxy:
    http_enabled: false
    ssh_enabled: false

observability:
  logging:
    level: debug
    grpc_level: warn

For GitLab Workspaces Proxy mode

Use this configuration in the .gitlab/agents/remotedev/config.yaml file:

remote_development:
  enabled: true
  dns_zone: workspaces.localtest.me
  network_policy:
    enabled: true
    egress:
    - allow: '0.0.0.0/0'
      except:
      - '10.0.0.0/8'
      - '172.16.0.0/12'
      - '192.168.0.0/16'
    - allow: '172.16.123.1/32'

observability:
  logging:
    level: debug
    grpc_level: warn

Set up GitLab Workspaces Proxy (proxy mode only)

If you chose the GitLab Workspaces Proxy mode, complete these additional steps:

1. Create an OAuth application:

   1. Go to http://gdk.test:3000/admin.

   2. Go to Applications and select Add new application.

   3. Set the name to GitLab Workspaces Proxy.

   4. Set the redirect URI to https://workspaces.localtest.me/auth/callback.

   5. Set the scopes to api, read_user, openid, and profile.

   6. Export the client credentials:

      export CLIENT_ID="your_client_id"
      export CLIENT_SECRET="your_client_secret"

2. Export GITLAB_URL to point to your GDK:

    export GITLAB_URL="http://gdk.test:3000"

3. Set up Ingress Controller and GitLab Workspaces Proxy:

    brew install mkcert
    cd "${GDK_ROOT}/gitlab"
    ./scripts/remote_development/workspaces_kubernetes_setup.sh

Create a workspace

1. Optional. Define a devfile for testing:

   1. If you don’t need to test specific configurations, skip this step in favor of using the default Devfile in the next step.

   2. Go to the gitlab-org/gitlab-shell project at http://gdk.test:3000/gitlab-org/gitlab-shell.

   3. Create a file at .devfile.yaml:

        schemaVersion: 2.2.0
        components:
          - name: tooling-container
            attributes:
              gl/inject-editor: true
            container:
                image: "registry.gitlab.com/gitlab-org/workspaces/gitlab-workspaces-docs/ubuntu:04@sha256:07590ca30ebde8a5339c3479404953e43ee70e7e9e0c2ede2770684010ddf7fe"

      The SHA256 is required to ensure the pulled container is the AMD64 architecture container. The GitLab VS Code fork for Workspaces does not support other architectures yet. To track this, see issue 392693.

2. Create a new workspace:

   1. Go to Your Work > Workspaces at http://gdk.test:3000/-/remote_development/workspaces.

   2. Select New Workspace.

   3. Choose the project with your devfile (or search for shell to find GitLab Shell).

      If you get an error about a missing agent configuration, check your agentk debug logs to ensure that your agentk successfully connects and reads your agent configuration file.

   4. Choose your cluster agent.

   5. If you skipped the devfile step, select Use GitLab default devfile.

   6. Select Create Workspace.

   7. Wait for the workspace to reach the Running state.

   8. Select Open Workspace.

3. To enable Extensions Marketplace for Web IDE in a workspace, see manage extensions.

   By default, the GitLab VS Code fork for Workspaces server uses Open VSX Extensions Marketplace. These settings are configured during a workspace startup in the product.json file. This file is located in the ${GL_EDITOR_VOLUME_DIR}/code-server directory.

   To customize the Extensions Marketplace configuration, these are the relevant properties in the product.json file:

   {
       "extensionsGallery": {
           "serviceUrl": "",
           "itemUrl": "",
           "resourceUrlTemplate": ""
       }
   }

Optional. Set up AI features

To enable AI features in workspaces:

1. Follow the instructions in Set up GitLab Team Member License for GDK.

   This page also lists other set up AI features options for local development. To provision a GitLab Self-Managed Ultimate Subscription with Duo Pro add-on license yourself, follow the cloud license with CustomersDot approach.

2. Configure your instance to use the staging AI gateway (https://cloud.staging.gitlab.com/ai).

For workspaces, you must enable Duo Chat features. They are only available with a Duo Enterprise license. You cannot provision this license for yourself through the staging Customers Portal. To upgrade your subscription from Duo Pro to Duo Enterprise, submit your request in the #g_provision Slack channel.

If configured correctly, in the Admin > GitLab Duo Pro settings, the message, No health problems detected. is displayed.

IDE Setup

RubyMine

Scopes

Use these scopes to:

• Search code in the scope.
• Inspect Code (cmd-shift-A -> Inspect Code) to run static analysis on the scope.

remote_dev

file[gitlab]:ee/lib/remote_development//*||file[gitlab]:ee/spec/factories/remote_development//*||file[gitlab]:ee/app/services/remote_development//*||file[gitlab]:app/models/remote_development//*||file[gitlab]:ee/app/graphql/mutations/remote_development//*||file[gitlab]:ee/app/graphql/resolvers/remote_development//*||file[gitlab]:ee/app/graphql/types/remote_development//*||file[gitlab]:ee/app/models/remote_development//*||file[gitlab]:ee/spec/graphql/types/remote_development//*||file[gitlab]:ee/spec/models/remote_development//*||file[gitlab]:ee/spec/services/remote_development//*||file[gitlab]:ee/app/finders/remote_development//*||file[gitlab]:ee/spec/features/remote_development//*||file[gitlab]:ee/spec/support/shared_contexts/remote_development//*||file[gitlab]:ee/app/graphql/ee/types/user_interface.rb||file[gitlab]:ee/app/graphql/resolvers/concerns/remote_development//*||file[gitlab]:ee/app/graphql/resolvers/projects/workspaces_resolver.rb||file[gitlab]:ee/app/graphql/resolvers/users/workspaces_resolver.rb||file[gitlab]:ee/spec/requests/api/graphql/mutations/remote_development//*||file[gitlab]:ee/spec/requests/api/graphql/remote_development//*||file[gitlab]:ee/spec/finders/remote_development//*||file[gitlab]:ee/app/assets/javascripts/remote_development//*||file[gitlab]:ee/spec/frontend/remote_development//*||file[gitlab]:ee/spec/graphql/api/workspace_spec.rb||file[gitlab]:ee/spec/fixtures/remote_development//*||file[gitlab]:ee/spec/lib/remote_development//*

remote_dev services & lib

Use this scope to set up restricted YARD inspections and to have safety warnings for:

• Editor -> Inspections -> YARD -> Missing @param tag in method signature, add scope as Warning
• Editor -> Inspections -> YARD -> Missing @return tag in method, add scope as Warning

file[gitlab]:ee/app/services/remote_development//*||file[gitlab]:ee/lib/remote_development//*

Testing

Run test suite

To run a subset of specs related to the Workspaces feature for a pre-commit “Smoke Test”, use the following script:

scripts/remote_development/run-smoke-test-suite.sh

Run E2E spec locally

• There is an end to end test that verifies the creation of a new running workspace.
• The test works by running UI actions on a running installation of a test GitLab instance, using GDK, KAS, and agentk.

To run the test:

1. Ensure that the test GitLab instance is up and running with the default KAS or agentk stopped.

2. Ensure KAS with remote development code is up and running.

3. By default, the E2E test assumes the existence of an agent with name test-agent under the group gitlab-org in the GDK GitLab instance:

   1. To work with the defaults, create an agent with the name test-agent in a project in gitlab-org group. The gitlab-shell project in the gitlab-org group is a candidate for where to create this agent.
   2. Alternatively, to use a custom group/agent, override the group and agent name with the environment variables AGENTK_GROUP and AGENTK_NAME.

4. Change the current working directory to {GDK_ROOT}/gitlab.

5. Run the test with scripts/remote_development/run-e2e-spec.sh.

   • To override the defaults, use AGENTK_GROUP=some-org GITLAB_PASSWORD=example scripts/remote_development/run-e2e-spec.sh

   The complete list of environment variables are in scripts/remote_development/run-e2e-spec.sh.

Verify behavior with example projects

Use the example projects to test GitLab Workspaces. These projects include devfiles and work out-of-the-box.

1. Clone the example projects to your development machine and push them to your local GitLab installation as public projects under the gitlab-org group.
2. The projects appear in the project list when you create a workspace.
3. Follow the README file in each project for specific usage instructions.

Repositories

These repositories are used to develop Workspaces:

These dependencies are external repositories that Workspaces relies on:

Debugging

IDE setup for debugging

For tips on debugging under Ruby Mine, see Using RubyMine debugger for GitLab running under GDK.

KAS and agentk debugging also works under GoLand. If you need help to set up Run Configurations, reach out to one of the developers or engineers.

Rails

log/remote_development.log

log/remote_development.log contains specific remote development logs in JSON format. You might need to install jq.

tail -f log/remote_development.log | jq

log/development.log

For other details or exceptions that are not in log/remote_development.log, see the standard Rails log/development.log:

tail -f log/development.log

Delete orphaned workspace in Rails

You might get orphaned workspace records on Rails, or you might want to start with a clean slate. To do this:

1. Go to the gitlab repository in your GDK.

2. Open the Rails console:

   bin/rails c

3. Delete all the workspace records:

   RemoteDevelopment::Workspace.delete_all

Kubernetes

Context and namespace management

Resource inspection

Logs and debugging

Get logs from a pod:

kubectl -n NAMESPACE logs -f POD_NAME

Get logs from a specific container:

kubectl -n NAMESPACE logs -f POD_NAME -c CONTAINER_NAME

Workspace-specific commands

Get all workspace objects:

kubectl get serviceaccount,pvc,networkpolicy,resourcequota,deployment,service,secret,configmap -l "agent.gitlab.com/id"

Get the gitlab-workspaces-proxy-config secret:

kubectl -n gitlab-workspaces get secret gitlab-workspaces-proxy-config -o go-template='{{range $k,$v := .data}}{{printf "%s: " $k}}{{if not $v}}{{$v}}{{else}}{{$v | base64decode}}{{end}}{{"\n"}}{{end}}'

Enter workspace main container shell:

PODNAME=$(kubectl get po -o name | cut -d/ -f2) && CONTAINER_NAME=$(kubectl get pod $PODNAME -o jsonpath='{range .spec.containers[*]}{.name}{"\t"}{range .env[*]}{.name}{","}{end}{"\n"}{end}' | grep GL_TOOLS_DIR | cut -f 1) && kubectl exec $PODNAME -c $CONTAINER_NAME -it -- /bin/bash

Run commands in the workspace container (example with log tailing):

PODNAME=$(kubectl get po -o name | cut -d/ -f2) && CONTAINER_NAME=$(kubectl get pod $PODNAME -o jsonpath='{range .spec.containers[*]}{.name}{"\t"}{range .env[*]}{.name}{","}{end}{"\n"}{end}' | grep GL_TOOLS_DIR | cut -f 1) && kubectl exec $PODNAME -c $CONTAINER_NAME -it -- /bin/bash -c "tail -n 100 -f /tmp/*.log"

Cleanup operations

Delete a namespace:

kubectl delete namespace NAMESPACE

Delete a pod:

kubectl -n NAMESPACE delete pods POD_NAME

Delete all workspace namespaces:

kubectl get namespace | grep gl- | cut -f1 -d" " | xargs -I {} kubectl delete namespace {}

Additional resources

For information about how localhost traffic reaches Kubernetes when using GitLab Workspaces Proxy, see this comment.