Deploying Python repositories

Python Libraries and Utilities

We deploy libraries and utilities to pypi with the gitlab user using poetry. Configure the deployment in the pyproject.toml file:

[tool.poetry]
name = "gitlab-<your package name>"
version = "0.1.0"
description = "<Description of your library/utility>"
authors = ["gitlab"]
readme = "README.md"
packages = [{ include = "<your module>" }]
homepage = ""https://gitlab.com/gitlab/<path/to/repository>"
repository = "https://gitlab.com/gitlab/<path/to/repository>"

Refer to poetry’s documentation for additional configuration options.

To configure deployment of the PyPI package:

1. Authenticate to PyPI using the “PyPI GitLab” credentials found in 1Password (PyPI does not support organizations as of now).

2. Create a token under Account Settings > Add API Tokens.

3. For the initial publish, select Entire account (all projects) scope. If the project already exists, scope the token to the specific project.

4. Configure credentials:

   Locally:

   poetry config pypi-token.pypi <your-api-token>

   To configure deployment with CI, set the POETRY_PYPI_TOKEN_PYPI to the token created. Alternatively, define a trusted publisher for the project, in which case no token is needed.

5. Use Poetry to publish your package:

   poetry publish

Python Services

Runway deployment for .com

Services for GitLab.com, GitLab Dedicated and self-hosted customers using CloudConnect are deployed using Runway. Please refer to the project documentation on how to add or manage Runway services.

Deploying in self-hosted environments

Deploying services to self-hosted environments poses challenges as services are not part of the monolith. Currently, Runway does not support self-hosted instances, and Omnibus does not support Python services, so deployment is only possible by users pulling the service image.

Image guidelines

1. Use a different user than the root user
2. Configure poetry variables correctly to avoid runtime issues
3. Use multi-stage Docker builds images to create lighter images

Example of image

Versioning

Self-hosted customers need to know which version of the service is compatible with their GitLab installation. Python services do not make use of managed versioning, so each service needs to handle its versioning and release cuts.

Per convention, once GitLab creates a new release, it can tag the service repo with a new tag named self-hosted-<gitlab-version>. An image with that tag is created, as seen on AI Gateway. It’s important that we have a version tag that matches GitLab versions, making it easier for users to deploy the full environment.

Publishing images

Images must be published in the container registry of the project.

It’s also recommend to publish the images on DockerHub. To create an image repository on Docker Hub, create an account with your GitLab handle and create an Access Request to be added to the GitLab organization. Once the image repository is created, make sure the user gitlabcibuild has read and write access to the repository.

Linux package deployment

To be added.

Deployment on GitLab Dedicated

Deployment of Python services on GitLab Dedicated is not currently supported