- How Load Performance Testing works
- Configure the Load Performance Testing job
With Load Performance Testing, you can test the impact of any pending code changes to your application’s backend in GitLab CI/CD.
GitLab uses k6, a free and open source tool, for measuring the system performance of applications under load.
Unlike Browser Performance Testing, which is used to measure how web sites perform in client browsers, Load Performance Testing can be used to perform various types of load tests against application endpoints such as APIs, Web Controllers, and so on. This can be used to test how the backend or the server performs at scale.
For example, you can use Load Performance Testing to perform many concurrent GET calls to a popular API endpoint in your application to see how it performs.
First, define a job in your
.gitlab-ci.yml file that generates the
Load Performance report artifact.
GitLab checks this report, compares key load performance metrics
between the source and target branches, and then shows the information in a merge request widget:
Next, you need to configure the test environment and write the k6 test.
The key performance metrics that the merge request widget shows after the test completes are:
- Checks: The percentage pass rate of the checks configured in the k6 test.
- TTFB P90: The 90th percentile of how long it took to start receiving responses, aka the Time to First Byte (TTFB).
- TTFB P95: The 95th percentile for TTFB.
- RPS: The average requests per second (RPS) rate the test was able to achieve.
.gitlab-ci.ymlfor the very first time, the Load Performance report widget won’t show. It must have run at least once on the target branch (
master, for example), before it will display in a merge request targeting that branch.
Configuring your Load Performance Testing job can be broken down into several distinct parts:
- Determine the test parameters such as throughput, and so on.
- Set up the target test environment for load performance testing.
- Design and write the k6 test.
The first thing you need to do is determine the type of load test you want to run, and how it will run (for example, the number of users, throughput, and so on).
A large part of the effort around load performance testing is to prepare the target test environment for high loads. You should ensure it’s able to handle the throughput it will be tested with.
It’s also typically required to have representative test data in the target environment for the load performance test to use.
We strongly recommend not running these tests against a production environment.
After the environment is prepared, you can write the k6 test itself. k6 is a flexible tool and can be used to run many kinds of performance tests. Refer to the k6 documentation for detailed information on how to write tests.
When your k6 test is ready, the next step is to configure the load performance
testing job in GitLab CI/CD. The easiest way to do this is to use the
template that is included with GitLab.
This template runs the k6 Docker container in the job and provides several ways to customize the job.
An example configuration workflow:
- Set up a GitLab Runner that can run Docker containers, such as a Runner using the Docker-in-Docker workflow.
Configure the default Load Performance Testing CI job in your
.gitlab-ci.ymlfile. You need to include the template and configure it with variables:
include: template: Verify/Load-Performance-Testing.gitlab-ci.yml load_performance: variables: K6_TEST_FILE: <PATH TO K6 TEST FILE IN PROJECT>
The above example creates a
load_performance job in your CI/CD pipeline that runs
the k6 test.
k6 has various options to configure how it will run tests, such as what throughput (RPS) to run with,
how long the test should run, and so on. Almost all options can be configured in the test itself, but as
you can also pass command line options via the
For example, you can override the duration of the test with a CLI option:
include: template: Verify/Load-Performance-Testing.gitlab-ci.yml load_performance: variables: K6_TEST_FILE: <PATH TO K6 TEST FILE IN PROJECT> K6_OPTIONS: '--duration 30s'
GitLab only displays the key performance metrics in the MR widget if k6’s results are saved via summary export as a Load Performance report artifact. The latest Load Performance artifact available is always used.
If GitLab Pages is enabled, you can view the report directly in your browser.
The best approach is to capture the dynamic URL into a custom environment variable that
is then inherited
load_performance job. The k6 test script to be run should then be configured to
use that environment URL, such as:
- In the
- Capture the dynamic URL and save it into a
echo "ENVIRONMENT_URL=$CI_ENVIRONMENT_URL" >> review.env.
- Set the
.envfile to be an
- Capture the dynamic URL and save it into a
- Set the
load_performancejob to depend on the review job, so it inherits the environment variable.
- Configure the k6 test script to use the environment variable in it’s steps.
.gitlab-ci.yml file might be similar to:
stages: - deploy - performance include: template: Verify/Load-Performance-Testing.gitlab-ci.yml review: stage: deploy environment: name: review/$CI_COMMIT_REF_NAME url: http://$CI_ENVIRONMENT_SLUG.example.com script: - run_deploy_script - echo "ENVIRONMENT_URL=$CI_ENVIRONMENT_URL" >> review.env artifacts: reports: dotenv: review.env rules: - if: '$CI_COMMIT_BRANCH' # Modify to match your pipeline rules, or use `only/except` if needed. load_performance: dependencies: - review rules: - if: '$CI_COMMIT_BRANCH' # Modify to match your pipeline rules, or use `only/except` if needed.