Export to CSV
This document lists the different implementations of CSV export in GitLab codebase.
| Export type | Implementation | Advantages | Disadvantages | Existing examples |
|---|---|---|---|---|
| Streaming | - Query and yield data in batches to a response stream. - Download starts immediately. | - Report available immediately. | - No progress indicator. - Requires a reliable connection. | Export audit event log |
| Downloading | - Query and write data in batches to a temporary file. - Loads the file into memory. - Sends the file to the client. | - Report available immediately. | - Large amount of data might cause request timeout. - Memory intensive. - Request expires when the user goes to a different page. | - Export Chain of Custody Report - Export License Usage File |
| As email attachment | - Asynchronously process the query with background job. - Email uses the export as an attachment. | - Asynchronous processing. | - Requires users use a different app (email) to download the CSV. - Email providers may limit attachment size. | - Export issues - Export merge requests |
| As downloadable link in email (*) | - Asynchronously process the query with background job. - Email uses an export link. | - Asynchronous processing. - Bypasses email provider attachment size limit. | - Requires users use a different app (email). - Requires additional storage and cleanup. | Export User Permissions |
| Polling (non-persistent state) | - Asynchronously processes the query with the background job. - Frontend(FE) polls every few seconds to check if CSV file is ready. | - Asynchronous processing. - Automatically downloads to local machine on completion. - In-app solution. | - Non-persistable request - request expires when the user goes to a different page. - API is processed for each polling request. | Export Vulnerabilities |
| Polling (persistent state) (*) | - Asynchronously processes the query with background job. - Backend (BE) maintains the export state - FE polls every few seconds to check status. - FE shows ‘Download link’ when export is ready. - User can download or regenerate a new report. | - Asynchronous processing. - No database calls made during the polling requests (HTTP 304 status is returned until export status changes). - Does not require user to stay on page until export is complete. - In-app solution. - Can be expanded into a generic CSV feature (such as dashboard / CSV API). | - Requires to maintain export states in DB. - Does not automatically download the CSV export to local machine, requires users to select ‘Download’. | Export Merge Commits Report |
Export types marked as * are currently work in progress.