Python development guidelines

This document describes conventions and practices we adopt at GitLab when developing Python code. While GitLab is built primarily on Ruby on Rails, we use Python when needed to leverage the ecosystem.

Some examples of Python in our codebase:

Design principles

  • Tooling should help contributors achieve their goals, both on short and long term.
  • A developer familiar with a Python codebase in GitLab should feel familiar with any other Python codebase at GitLab.
  • This documentation should support all contributors, regardless of their goals and incentives: from Python experts to one-off contributors.
  • We strive to follow external guidelines, but if needed we will choose conventions that better support GitLab contributors.

When should I consider Python for development

Ruby should always be the first choice for development at GitLab, as we have a larger community, better support, and easier deployment. However, there are occasions where using Python is worth breaking the pattern. For example, when working with AI and ML, most of the open source uses Python, and using Ruby would require building and maintaining large codebases.

Creating a new Python application

Scaffolding, libraries and pipelines for a new codebase.

Conventions Style Guidelines

Writing consistent codebases.

Contributing to a Python codebase

Resources to get started, examples and tips.

Code review and Maintainership guidelines

How to create a merge request that minimizes review time and things to pay attention to when reviewing code.

Deploying a Python codebase

Deploying libraries, utilities and services.

Python as part of the Monorepo

GitLab requires Python as a dependency for reStructuredText markup rendering. It requires Python 3.

Installation

There are several ways of installing Python on your system. To be able to use the same version we use in production, we suggest you use pyenv. It works and behaves similarly to its counterpart in the Ruby world: rbenv.

macOS

To install pyenv on macOS, you can use Homebrew with: 

brew install pyenv

Windows

pyenv does not officially support Windows and does not work in Windows outside the Windows Subsystem for Linux. If you are a Windows user, you can use pyenv-win.

To install pyenv-win on Windows, run the following PowerShell command: 

Invoke-WebRequest -UseBasicParsing -Uri "https://raw.githubusercontent.com/pyenv-win/pyenv-win/master/pyenv-win/install-pyenv-win.ps1" -OutFile "./install-pyenv-win.ps1"; &"./install-pyenv-win.ps1"

Learn more about pyenv-win.

Linux

To install pyenv on Linux, you can run the command below: 

curl "https://pyenv.run" | bash

Alternatively, you may find pyenv available as a system package via your distribution’s package manager.

You can read more about it in the pyenv prerequisites.

Shell integration

Pyenv installation adds required changes to Bash. If you use a different shell, check for any additional steps required for it.

For Fish, you can install a plugin for Fisher: 

fisher add fisherman/pyenv

Or for Oh My Fish: 

omf install pyenv

Dependency management

While GitLab doesn’t directly contain any Python scripts, because we depend on Python to render reStructuredText markup, we need to keep track on dependencies on the main project level, so we can run that on our development machines.

Recently, an equivalent to the Gemfile and the Bundler project has been introduced to Python: Pipfile and Pipenv.

A Pipfile with the dependencies now exists in the root folder. To install them, run: 

pipenv install

Running this command installs both the required Python version as well as required pip dependencies.

Use instructions

To run any Python code under the Pipenv environment, you need to first start a virtualenv based on the dependencies of the application. With Pipenv, this is a simple as running: 

pipenv shell

After running that command, you can run GitLab on the same shell and it uses the Python and dependencies installed from the pipenv install command.

Learning resources

If you are new to Python or looking to refresh your knowledge, this section provides variours materials for learning the language.

  1. Python Cheatsheet A comprehensive reference covering essential Python syntax, built-in functions, and useful libraries. This is ideal for both beginners and experienced users who want a quick, organized summary of Python’s key features.

  2. A Whirlwind Tour of Python (Jupyter Notebook) A fast-paced introduction to Python fundamentals, tailored especially for data science practitioners but works well for everyone who wants to get just the basic understanding of the language. This is a Jupiter Notebook which makes this guide an interactive resource as well as a good introduction to Jupiter Notebook itself.

  3. 100-page Python Intro Brief guide provides a straightforward introduction to Python, covering all the essentials needed to start programming effectively. It’s a beginner-friendly option that covers everything from syntax to debugging and testing.

  4. Learn X in Y Minutes: Python A very brief, high-level introduction cuts directly to the core syntax and features of Python, making it a valuable quick start for developers transitioning to Python.

  5. Exercism Python Track Use Exercism’s Python track as a foundation for learning Python concepts and best practices. Exercism provides hands-on practice with mentoring support, making it an excellent resource for mastering Python through coding exercises and feedback.

When building Python APIs, we use FastAPI and Pydantic. To get started with building and reviewing these technologies, refer to the following resources:

  1. FastAPI Documentation FastAPI is a modern web framework for building APIs with Python. This resource will help you learn how to create fast and efficient web applications and APIs. FastAPI is especially useful for building Python applications with high performance and scalability.

  2. Pydantic Documentation Pydantic is a Python library for data validation and settings management using Python type annotations. Learn how to integrate Pydantic into your Python projects for easier data validation and management, particularly when working with FastAPI.

We use pytest for testing Python code. To learn more about writing and running tests with pytest, refer to the following resources:

  1. pytest Documentation pytest is a popular testing framework for Python that makes it easy to write simple and scalable tests. This resource provides comprehensive documentation on how to write and run tests using pytest, including fixtures, plugins, and test discovery.

  2. Python Testing with pytest (Book) This book is a comprehensive guide to testing Python code with pytest. It covers everything from the basics of writing tests to advanced topics like fixtures, plugins, and test organization.