Workhorse handlers

Long HTTP requests are hard to handle efficiently in Rails. The requests are either memory-inefficient (file uploads) or impossible at all due to shorter timeouts (for example, Puma server has 60-second timeout). Workhorse can efficiently handle a large number of long HTTP requests. Workhorse acts as a proxy that intercepts all HTTP requests and either propagates them without changing or handles them itself by performing additional logic.

Injectors

%%{init: { "fontFamily": "GitLab Sans" }}%%
sequenceDiagram
    participant Client
    participant Workhorse
    participant Rails

    Client->>+Workhorse: Request
    Workhorse->>+Rails: Propagate the request as-is
    Rails-->>-Workhorse: Respond with a special header that contains instructions for proceeding with the request
    Workhorse-->>Client: Response

Example: Send a Git blob

%%{init: { "fontFamily": "GitLab Sans" }}%%
sequenceDiagram
    participant Client
    participant Workhorse
    participant Rails
    participant Gitaly

    Client->>+Workhorse: HTTP Request for a blob
    Workhorse->>+Rails: Propagate the request as-is
    Rails-->>-Workhorse: Respond with a git-blob:{encoded_data} header
    Workhorse->>+Gitaly: BlobService.GetBlob gRPC request
    Gitaly-->>-Workhorse: BlobService.GetBlob gRPC request
    Workhorse-->>Client: Stream the data

How GitLab Rails processes the request

How Workhorse processes the header

Example: Send a file

%%{init: { "fontFamily": "GitLab Sans" }}%%
sequenceDiagram
    participant Client
    participant Workhorse
    participant Rails
    participant Object Storage

    Client->>+Workhorse: HTTP Request for a file
    Workhorse->>+Rails: Propagate the request as-is
    Rails-->>-Workhorse: Respond with a send-url:{encoded_data} header
    Workhorse->>+Object Storage: Request for a file
    Object Storage-->>-Workhorse: Stream the data
    Workhorse-->>Client: Stream the data

Pre-authorized requests

%%{init: { "fontFamily": "GitLab Sans" }}%%
sequenceDiagram
    participant Client
    participant Workhorse
    participant Rails
    participant Object Storage

    Client->>+Workhorse: PUT /artifacts/uploads
    Note right of Rails: Append `/authorize` to the original URL and call Rails for an Auth check
    Workhorse->>+Rails: GET /artifacts/uploads/authorize
    Rails-->>-Workhorse: Authorized successfully

    Client->>+Workhorse: Stream the file content
    Workhorse->>+Object Storage: Upload the file
    Object Storage-->>-Workhorse: Success

    Workhorse->>+Rails: Finalize the request
    Note right of Rails: Workhorse calls the original URL to create a database record
    Rails-->>-Workhorse: Finalized successfully
    Workhorse-->>Client: Uploaded successfully

Git over HTTP(S)

Workhorse accelerates Git over HTTP(S) by handling Git HTTP protocol requests. For example, Git push/pull may require serving large amounts of data. To avoid transferring it through GitLab Rails, Workhorse only performs authorization checks against GitLab Rails, then performs a Gitaly gRPC request directly, and streams the data from Gitaly to the Git client.

Git pull

%%{init: { "fontFamily": "GitLab Sans" }}%%
sequenceDiagram
participant Git on client
participant Workhorse
participant Rails
participant Gitaly

Note left of Git on client: git clone/fetch
Git on client->>+Workhorse: GET /foo/bar.git/info/refs/?service=git-upload-pack
Workhorse->>+Rails: GET Repositories::GitHttpController#info_refs
Note right of Rails: Access check/Log activity
Rails-->>Workhorse: 200 OK, Gitlab::Workhorse.git_http_ok
Workhorse->>+Gitaly: SmartHTTPService.InfoRefsUploadPack gRPC request
Gitaly -->>-Workhorse: SmartHTTPService.InfoRefsUploadPack gRPC response
Workhorse-->>-Git on client: send info-refs response
Git on client->>+Workhorse: GET /foo/bar.git/info/refs/?service=git-upload-pack
Workhorse->>+Rails: GET Repositories::GitHttpController#git_receive_pack
Note right of Rails: Access check/Update statistics
Rails-->>Workhorse: 200 OK, Gitlab::Workhorse.git_http_ok
Workhorse->>+Gitaly: SmartHTTPService.PostUploadPackWithSidechannel gRPC request
Gitaly -->>-Workhorse: SmartHTTPService.PostUploadPackWithSidechannel gRPC response
Workhorse-->>-Git on client: send response

Git push

%%{init: { "fontFamily": "GitLab Sans" }}%%
sequenceDiagram
participant Git on client
participant Workhorse
participant Rails
participant Gitaly

Note left of Git on client: git push
Git on client->>+Workhorse: GET /foo/bar.git/info/refs/?service=git-receive-pack
Workhorse->>+Rails: GET Repositories::GitHttpController#info_refs
Note right of Rails: Access check/Log activity
Rails-->>Workhorse: 200 OK, Gitlab::Workhorse.git_http_ok
Workhorse->>+Gitaly: SmartHTTPService.InfoRefsReceivePack gRPC request
Gitaly -->>-Workhorse: SmartHTTPService.InfoRefsReceivePack gRPC response
Workhorse-->>-Git on client: send info-refs response
Git on client->>+Workhorse: GET /foo/bar.git/info/refs/?service=git-receive-pack
Workhorse->>+Rails: GET Repositories::GitHttpController#git_receive_pack
Note right of Rails: Access check/Update statistics
Rails-->>Workhorse: 200 OK, Gitlab::Workhorse.git_http_ok
Workhorse->>+Gitaly: SmartHTTPService.PostReceivePackWithSidechannel gRPC request
Gitaly -->>-Workhorse: SmartHTTPService.PostReceivePackWithSidechannel gRPC response
Workhorse-->>-Git on client: send response