- Branch and tag protection
- Mirroring between repositories
- Types of pipelines
- CI Jobs
- Release jobs
- Housekeeping Jobs
Repositories, branches, and CI pipelines
omnibus-gitlab
CI pipeline is a moderately complex one with pipelines split
between all the mirrors of the project:
- Development repository for normal feature development.
- Release mirror for building release artifacts.
- Security mirror for security development.
- QA mirror for developers to run a package build and QA for development purposes.
Branch and tag protection
Protected branches
- Development repository
-
master
: Maintainers, Delivery team -
*-stable
: Delivery team, Release Managers -
*-stable-ee
: Delivery team, Release Managers -
*-auto-deploy-*
: Maintainers,delivery
group,managers
group
-
- Release mirror:
-
master
: Maintainers -
*-stable
: Maintainers -
*-stable-ee
: Maintainers -
*-auto-deploy-*
: Maintainers
-
- Security mirror:
-
master
: GitLab Release Tools Bot, GitLab Bot, Delivery team, Release Managers -
*-stable
: GitLab Release Tools Bot, GitLab Bot, Delivery team, Release Managers -
*-stable-ee
: GitLab Release Tools Bot, GitLab Bot, Delivery team, Release Managers -
*-auto-deploy-*
: GitLab Release Tools Bot, GitLab Bot, Delivery team, Release Managers
-
- QA mirror:
-
master
: Developers (Merge only), Maintainers
-
master
branch in QA mirror because
that’s required to run a triggered pipeline against the branch. There is an
open issue to change this
situation.Protected Tags
- Development repository:
-
*
: Maintainers, Delivery team, Release Managers
-
- Release mirror:
-
*
: Maintainers
-
-
Security mirror: Nil
- QA mirror: Nil
Mirroring between repositories
Most of the development happens in the Development repository and security related changes go to Security mirror. These changes then gets mirrored to Release mirror and QA mirror. The following diagram describes what all gets mirrored between each of these repositories.
Types of pipelines
Branch pipelines
omnibus-gitlab
doesn’t make use of detached merge request pipelines yet. So,
pipelines caused by branch pushes are the common pipelines encountered in this
project. These pipelines are used in Development repository, Security mirror,
and Release mirror.
Tag pipelines
Tag pushes to Release mirror starts a pipeline with the release jobs. Tag pushes to Development repository and Security mirror behaves as regular branch pushes (except that they don’t have an option to start a package-and-qa pipeline) and run basic stylistic checks and unit tests.
Scheduled pipelines
In Development repository, there are two scheduled pipelines.
-
Dependencies Update
- Pipeline to check for outdated dependencies using thedependency_update
job. -
Generate license pages
- Pipeline to populate License collection webpage with license information from the S3 bucket.
In Release mirror, there are two scheduled pipelines
-
CE nightly
- Pipeline to build nightly packages and Docker images for GitLab CE -
EE nightly
- Pipeline to build nightly packages and Docker images for GitLab EE
Other mirrors doesn’t have any scheduled pipelines
Auto-deploy pipelines
GitLab uses Auto deploy branches and tags for its release process. These
branches are named <MAJOR>-<MINOR>-auto-deploy-<TIMESTAMP>
and tags have the
format <MAJOR>.<MINOR>.<TIMESTAMP>+<gitlab sha>.<omnibus-gitlab sha>
.
Only certain jobs - that are needed to deploy the changes to GitLab environments - will be part of this pipeline.
Triggered pipelines
We use triggered pipelines to run a “package-and-qa” pipeline in the QA mirror. These can be triggered either from a pipeline in the Development repository or GitLab project.
This pipeline is intended to give the developers a package and an image to test their changes in addition to automatically doing a QA run against these artifacts. It also provides an option to do a QA run against an HA instance spun up using these artifacts.
CI Jobs
Development Jobs
danger-review
This job runs a check against the merge request to ensure it satisfies some basic requirements using the Danger tool.
This job is run only on Development repository and Security mirror on branch and tag pipelines.
rubocop
Check the source code files to satisfy certain stylistic requirements.
This job is run only on Development repository and Security mirror on branch and tag pipelines.
docs-lint
Check the docs files to satisfy certain stylistic requirements.
This job is run only on Development repository and Security mirror on branch and tag pipelines.
review-docs-deploy
A manual job that will trigger a docs build in gitlab-docs
, which will deploy a
review app for the GitLab Docs with changes from the current commit.
This job is run only on Development repository on branch pipelines.
review-docs-cleanup
A manual job to stop the environment created by review-docs-deploy
.
Automatically ran on merging a merge request.
This job is run only on Development repository on branch pipelines.
<OS_NAME> knapsack
We have tests covering our Chef recipes and libraries using RSpec
and
Chefspec
. They are parallelized with the help of
knapsack.
This job pulls in the knapsack reports of previous pipeline runs from the cache and gets them in place for the current spec runs.
These jobs are run only on Development repository and Security mirror on branch and tag pipelines.
<OS_NAME> specs
Jobs that actually run rspec
via knapsack
. Parallelized to a count of 6
using the parallel
keyword
These jobs are run only on Development repository and Security mirror on branch and tag pipelines.
update-knapsack
This job combines the reports from each individual parallel runs of unit tests and prepares a final JSON report. This report is used by the MR widget to display spec status, and is also uploaded to the cache for the next pipeline run to use.
This job is run only on Development repository and Security mirror on branch and tag pipelines.
Trigger:ce-package
This is a manual job which when played triggers a pipeline in the QA mirror to run a package build and QA for development purposes. This can be used by developers to get a package or Docker image for testing, or to run a full QA suite against their MR’s changes.
This job is run only on Development repository and Security mirror on branch pipelines.
Trigger:ee-package
Same as Trigger:ce-package
, but builds an EE package.