Development of GitLab Duo Workflow

This guide describes how to set up the local development environment for the various projects that make up GitLab Duo Workflow.

Prerequisites

  • Vertex API access
    • You need access to the ai-enablement-dev-69497ba7 project in GCP. This should by available to all engineers at GitLab.
  • Docker
    • See which Docker tooling is approved for GitLab team members in the handbook.

Set up local development for GitLab Duo Workflow

Duo Workflow consists of four separate services:

  1. GitLab instance
  2. Duo Workflow Service
  3. Duo Workflow Executor
  4. Duo Workflow UI within the Language server

GDK Setup

We recommend setting up Duo Workflow directly with the GitLab Development Kit (GDK) for setting up local versions of GitLab, the Duo Workflow Service and Executor.

This setup can be used with the publicly available version of the VS Code Extension. Follow these instructions to see Duo Workflow UI local build in VS Code if you want to actively need to develop it or use an unreleased version.

Manual Setup

Duo Workflow UI in Visual Studio Code (VS Code)

There is no need for the GDK, Duo Workflow service or Duo Workflow executor local build to test Duo Workflow UI. Only set these up if you are making changes to one of these packages and need to test their integration with the Duo Workflow UI in VS Code. Please refer to the Duo Workflow README.md file in the Language Server project to get started with local development of Duo Workflow UI.

Set up your local GitLab instance

  1. Configure the Duo Workflow Service URL in your local GitLab instance by updating the config/gitlab.yml file:

    development:
      duo_workflow:
        service_url: 0.0.0.0:50052
        secure: false
  2. Restart the GitLab instance.

    gdk restart rails
    
  3. In your local GitLab instance, enable the duo_workflow feature flag from the Rails console:

    Feature.enable(:duo_workflow)
    
  4. Set up GitLab Runner with GDK so you can create CI jobs locally to test Workflow.
  5. Create a personal access token in your local GitLab instance with the api scope. Save this value and use it in the next step.
  6. Run GDK with an Ultimate license.
  7. If you’re running GitLab in SaaS mode, you’ll need to turn on the beta and experimental features functionality, as they are turned off by default. In the group settings for the project you’ll run workflow against, ensure that the Use experiment and beta Duo features checkbox is checked.
  8. Manually create a Workflow using the following curl request; the output will be a workflow ID that is referred to as $WORKFLOW_ID throughout the rest of these docs:

    curl POST --verbose \
      --header "Authorization: Bearer $YOUR_GITLAB_PAT" \
      --header 'Content-Type: application/json' \
      --data '{
         "project_id": "$PROJECT_ID_FOR_RUNNING_WORKFLOW_AGAINST"
      }' \
      $YOUR_GDK_ROOT_URL/api/v4/ai/duo_workflows/workflows
    

Set up the Duo Workflow Service and Executor

Refer to the readme of Duo Workflow Service and Duo Workflow Executor to set them up individually.

Troubleshooting

Issues connecting to 50052 port

JAMF may be listening on the 50052 port which will conflict with Duo Workflow Service.

$ sudo lsof -i -P | grep LISTEN | grep :50052
jamfRemot  <redacted>           root   11u  IPv4 <redacted>      0t0    TCP localhost:50052 (LISTEN)

To work around this,run the serveron 50053 with:

PORT=50053 poetry run duo-workflow-service