GitLab tests in the Continuous Integration (CI) context
Test suite parallelization on the CI
Our current CI parallelization setup is as follows:
- The
retrieve-tests-metadata
job in theprepare
stage ensures we have aknapsack/report-master.json
file:- The
knapsack/report-master.json
file is fetched from S3, if it’s not here we initialize the file with{}
.
- The
- Each
[rspec|rspec-ee] [unit|integration|system|geo] n m
job are run withknapsack rspec
and should have an evenly distributed share of tests:- It works because the jobs have access to the
knapsack/report-master.json
since the “artifacts from all previous stages are passed by default”. - the jobs set their own report path to
"knapsack/${TEST_TOOL}_${TEST_LEVEL}_${DATABASE}_${CI_NODE_INDEX}_${CI_NODE_TOTAL}_report.json"
. - if knapsack is doing its job, test files that are run should be listed under
Report specs
, not underLeftover specs
.
- It works because the jobs have access to the
- The
update-tests-metadata
job (which only runs on scheduled pipelines for the canonical project takes all theknapsack/rspec*_pg_*.json
files and merge them all together into a singleknapsack/report-master.json
file that is then uploaded to S3.
After that, the next pipeline will use the up-to-date knapsack/report-master.json
file.
Monitoring
The GitLab test suite is monitored for the master
branch, and any branch
that includes rspec-profile
in their name.
A public dashboard is available for everyone to see. Feel free to look at the slowest test files and try to improve them.
CI setup
- Rails logging to
log/test.log
is disabled by default in CI for performance reasons. To override this setting, provide theRAILS_ENABLE_TEST_LOG
environment variable.
Help and feedback
If there's something you don't like about this feature
To propose functionality that GitLab does not yet offer
To further help GitLab in shaping new features
If you didn't find what you were looking for
If you want help with something very specific to your use case, and can use some community support
POST ON GITLAB FORUM
If you have problems setting up or using this feature (depending on your GitLab subscription)
REQUEST SUPPORT
To view all GitLab tiers and features or to upgrade
If you want to try all features available in GitLab.com
If you want to try all features available in GitLab self-managed
If you spot an error or a need for improvement and would like to fix it yourself in a merge request
EDIT THIS PAGE
If you would like to suggest an improvement to this doc