Workhorse handlers

Long HTTP requests are hard to handle efficiently in Rails. The requests are either memory-inefficient (file uploads) or impossible at all due to shorter timeouts (for example, Puma server has 60-second timeout). Workhorse can efficiently handle a large number of long HTTP requests. Workhorse acts as a proxy that intercepts all HTTP requests and either propagates them without changing or handles them itself by performing additional logic.

Injectors

%%{init: { "fontFamily": "GitLab Sans" }}%% sequenceDiagram participant Client participant Workhorse participant Rails Client->>+Workhorse: Request Workhorse->>+Rails: Propagate the request as-is Rails-->>-Workhorse: Respond with a special header that contains instructions for proceeding with the request Workhorse-->>Client: Response

Example: Send a Git blob

%%{init: { "fontFamily": "GitLab Sans" }}%% sequenceDiagram participant Client participant Workhorse participant Rails participant Gitaly Client->>+Workhorse: HTTP Request for a blob Workhorse->>+Rails: Propagate the request as-is Rails-->>-Workhorse: Respond with a git-blob:{encoded_data} header Workhorse->>+Gitaly: BlobService.GetBlob gRPC request Gitaly-->>-Workhorse: BlobService.GetBlob gRPC request Workhorse-->>Client: Stream the data

How GitLab Rails processes the request

How Workhorse processes the header

Example: Send a file

%%{init: { "fontFamily": "GitLab Sans" }}%% sequenceDiagram participant Client participant Workhorse participant Rails participant Object Storage Client->>+Workhorse: HTTP Request for a file Workhorse->>+Rails: Propagate the request as-is Rails-->>-Workhorse: Respond with a send-url:{encoded_data} header Workhorse->>+Object Storage: Request for a file Object Storage-->>-Workhorse: Stream the data Workhorse-->>Client: Stream the data

Pre-authorized requests

%%{init: { "fontFamily": "GitLab Sans" }}%% sequenceDiagram participant Client participant Workhorse participant Rails participant Object Storage Client->>+Workhorse: PUT /artifacts/uploads Note right of Rails: Append `/authorize` to the original URL and call Rails for an Auth check Workhorse->>+Rails: GET /artifacts/uploads/authorize Rails-->>-Workhorse: Authorized successfully Client->>+Workhorse: Stream the file content Workhorse->>+Object Storage: Upload the file Object Storage-->>-Workhorse: Success Workhorse->>+Rails: Finalize the request Note right of Rails: Workhorse calls the original URL to create a database record Rails-->>-Workhorse: Finalized successfully Workhorse-->>Client: Uploaded successfully

Git over HTTP(S)

Workhorse accelerates Git over HTTP(S) by handling Git HTTP protocol requests. For example, Git push/pull may require serving large amounts of data and in order to avoid transferring it through GitLab Rails, Workhorse only performs authorization checks against GitLab Rails, then performs Gitaly gRPC request directly and streams the data from Gitaly to the Git client.

Git pull

%%{init: { "fontFamily": "GitLab Sans" }}%% sequenceDiagram participant Git on client participant Workhorse participant Rails participant Gitaly Note left of Git on client: git clone/fetch Git on client->>+Workhorse: GET /foo/bar.git/info/refs/?service=git-upload-pack Workhorse->>+Rails: GET Repositories::GitHttpController#info_refs Note right of Rails: Access check/Log activity Rails-->>Workhorse: 200 OK, Gitlab::Workhorse.git_http_ok Workhorse->>+Gitaly: SmartHTTPService.InfoRefsUploadPack gRPC request Gitaly -->>-Workhorse: SmartHTTPService.InfoRefsUploadPack gRPC response Workhorse-->>-Git on client: send info-refs response Git on client->>+Workhorse: GET /foo/bar.git/info/refs/?service=git-upload-pack Workhorse->>+Rails: GET Repositories::GitHttpController#git_receive_pack Note right of Rails: Access check/Update statistics Rails-->>Workhorse: 200 OK, Gitlab::Workhorse.git_http_ok Workhorse->>+Gitaly: SmartHTTPService.PostUploadPackWithSidechannel gRPC request Gitaly -->>-Workhorse: SmartHTTPService.PostUploadPackWithSidechannel gRPC response Workhorse-->>-Git on client: send response

Git push

%%{init: { "fontFamily": "GitLab Sans" }}%% sequenceDiagram participant Git on client participant Workhorse participant Rails participant Gitaly Note left of Git on client: git push Git on client->>+Workhorse: GET /foo/bar.git/info/refs/?service=git-receive-pack Workhorse->>+Rails: GET Repositories::GitHttpController#info_refs Note right of Rails: Access check/Log activity Rails-->>Workhorse: 200 OK, Gitlab::Workhorse.git_http_ok Workhorse->>+Gitaly: SmartHTTPService.InfoRefsReceivePack gRPC request Gitaly -->>-Workhorse: SmartHTTPService.InfoRefsReceivePack gRPC response Workhorse-->>-Git on client: send info-refs response Git on client->>+Workhorse: GET /foo/bar.git/info/refs/?service=git-receive-pack Workhorse->>+Rails: GET Repositories::GitHttpController#git_receive_pack Note right of Rails: Access check/Update statistics Rails-->>Workhorse: 200 OK, Gitlab::Workhorse.git_http_ok Workhorse->>+Gitaly: SmartHTTPService.PostReceivePackWithSidechannel gRPC request Gitaly -->>-Workhorse: SmartHTTPService.PostReceivePackWithSidechannel gRPC response Workhorse-->>-Git on client: send response