Amazon Machine Images (AMIs) and Marketplace Listings
GitLab caters to the AWS ecosystem via the following methods
- Community AMIs
- AWS Marketplace listing
Building the AMIs
AMIs are built as part of regular release process in the tag pipelines that run
in the Build mirror, and use the
Ubuntu 22.04 packages under the hood. They are built using packer
with their Amazon EBS
builder. Each Community AMI listed above has a corresponding packer
configuration file to specify the build and AMI attributes and an update script
to install the GitLab package and configure the AMI’s startup behaviors. The
update script downloads the Ubuntu 22.04 package and installs it on the VM. It
also installs a cloud-init
script which automatically configures (with a gitlab-ctl reconfigure run) the
GitLab instance to work with the VM’s external IP address on startup.
In addition to these public Community AMIs, two private AMIs are also built -
for GitLab EE Premium and Ultimate tiers, which ships 5-seat licenses of the
respective GitLab tier. This license file gets used as part of the initial
gitlab-ctl reconfigure run on VM startup. These AMIs are backing our AWS
Marketplace listings.
Releasing to AWS Marketplace
In addition to building the AMIs during the release process, Omnibus GitLab tag pipeline also publishes the new version of the respective AWS Marketplace listing. The private AMIs mentioned above are used to back these listings. As part of release pipeline, we submit a changeset to publish the new version. Unlike AMI creation, this process is not immediate and we need to manually check the status of the changeset periodically to ensure it got applied to the listings, preferably after 24 hours.
Common release blocker events
The following events has happen often in the past, and has caused the release pipeline to fail, and needs immediate attention:
-
Exhausting quota on Public AMIs - When builds fail due to quota exhaustion, as an immediate fix, request a quota increase. Then discuss with Alliances/Product on de-registering or making private AMIs of some of the older versions. Also check issue discussing retention policy of AMIs.
-
AWS Marketplace version limit - AWS Marketplace has a 100 versions limit for each product, exhausting which we can’t publish newer versions. However, they usually inform us (via email to a specific email account which is forwarded to selected Distribution Build team and Alliance team members) which is when we are nearing that limit, and we work with Alliances/Product to unlist some of the older versions.
-
AWS Marketplace listing blocked by a pending changeset - When this happens, the changeset needs to be manually cancelled, and the Marketplace release job in the release pipeline needs to be retried. This requires someone with Maintainer level access to the Build mirror of
omnibus-gitlabproject.