Using Git submodules with GitLab CI/CD

Use Git submodules to keep a Git repository as a subdirectory of another Git repository. You can clone another repository into your project and keep your commits separate.

Configure the .gitmodules file

When you use Git submodules, your project should have a file named .gitmodules. You have multiple options to configure it to work in a GitLab CI/CD job.

Using absolute URLs

For example, your generated .gitmodules configuration might look like the following if:

• Your project is located at https://gitlab.com/secret-group/my-project.
• Your project depends on https://gitlab.com/group/project, which you want to include as a submodule.
• You check out your sources with an SSH address like git@gitlab.com:secret-group/my-project.git.

[submodule "project"]
  path = project
  url = git@gitlab.com:group/project.git

In this case, use the GIT_SUBMODULE_FORCE_HTTPS variable to instruct GitLab Runner to convert the URL to HTTPS before it clones the submodules.

Alternatively, if you also use HTTPS locally, you can configure an HTTPS URL:

[submodule "project"]
  path = project
  url = https://gitlab.com/group/project.git

You do not need to configure additional variables in this case, but you need to use a personal access token to clone it locally.

Using relative URLs

When your submodule is on the same GitLab server, you can also use relative URLs in your .gitmodules file:

[submodule "project"]
  path = project
  url = ../../project.git

The previous configuration instructs Git to automatically deduce the URL to use when cloning sources. You can clone with HTTPS in all your CI/CD jobs, and you can continue to use SSH to clone locally.

For submodules not located on the same GitLab server, always use the full URL:

[submodule "project-x"]
  path = project-x
  url = https://gitserver.com/group/project-x.git

Use Git submodules in CI/CD jobs

Prerequisites:

• If you use the CI_JOB_TOKEN to clone a submodule in a pipeline job, you must have at least the Reporter role for the submodule repository to pull the code.
• CI/CD job token access must be properly configured in the upstream submodule project.

To make submodules work correctly in CI/CD jobs:

1. You can set the GIT_SUBMODULE_STRATEGY variable to either normal or recursive to tell the runner to fetch your submodules before the job:

   variables:
     GIT_SUBMODULE_STRATEGY: recursive

2. For submodules located on the same GitLab server and configured with a Git or SSH URL, make sure you set the GIT_SUBMODULE_FORCE_HTTPS variable.

3. Use GIT_SUBMODULE_DEPTH to configure the cloning depth of submodules independently of the GIT_DEPTH variable:

   variables:
     GIT_SUBMODULE_DEPTH: 1

4. You can filter or exclude specific submodules to control which submodules are synchronized using GIT_SUBMODULE_PATHS.

   variables:
     GIT_SUBMODULE_PATHS: submoduleA submoduleB

5. You can provide additional flags to control advanced checkout behavior using GIT_SUBMODULE_UPDATE_FLAGS.

   variables:
     GIT_SUBMODULE_STRATEGY: recursive
     GIT_SUBMODULE_UPDATE_FLAGS: --jobs 4

Check out nested submodules

Nested submodules are submodules that contain their own submodules. You might need to check out only specific nested submodules rather than all submodules in your repository.

GitLab Runner 18.6 and later externalizes Git configuration (including credentials) to a separate file to avoid tainting the build directory. When you navigate into a submodule directory and run Git commands, the main repository’s configuration is automatically inherited for all submodules depending on GIT_SUBMODULE_STRATEGY:

• If GIT_SUBMODULE_STRATEGY: normal is used, then the top-level submodules are initialized.
• If GIT_SUBMODULE_STRATEGY: recursive is used, then all the nested submodules are initialized.

To check out a subset of nested submodules:

1. Set the GIT_SUBMODULE_STRATEGY to normal:

      variables:
        GIT_SUBMODULE_STRATEGY: normal

2. In your job, explicitly pass the externalized configuration:

      my-job:
        script:
          - git submodule sync
          - git submodule update --init
          - cd path/to/submodule-with-nested-submodule
          - git -c "include.path=$(git -C $CI_PROJECT_DIR config include.path)" submodule update --init nested-submodule

The git -C $CI_PROJECT_DIR config include.path command retrieves the path to the externalized configuration file from the main repository. This ensures that credentials and other settings are available when you check out the nested submodule.

Use submodules from another GitLab instance

When your submodule is hosted on a different GitLab instance than your main project, the CI_JOB_TOKEN from your current instance cannot authenticate to the external instance. You must use a token created on the external instance to authenticate.

You have two main approaches for authenticating with external GitLab instances:

• URL rewriting: Modifies Git URLs to include authentication credentials.
• Git credential helper: Stores credentials that Git automatically uses when needed.

The authentication method you choose depends on your GitLab Runner executor type:

• Containerized executors (Docker or Kubernetes): Each job runs in an isolated container, so global Git configuration changes only affect the current job and are automatically cleaned up when the container is destroyed.

• Shell executors: Jobs run directly on the runner host system, so global Git configuration changes persist between jobs. This can cause authentication conflicts if different jobs use different credentials.

You can use one of the following token types:

• Personal access token
• Deploy token
• Project access token

Configure authentication with URL rewriting

To configure authentication with URL rewriting:

1. In your .gitmodules file, use an absolute HTTPS URL for the submodule:

   [submodule "external-project"]
     path = external-project
     url = https://other-gitlab.example.com/group/project.git

2. On the external GitLab instance, create a token with the read_repository scope.

3. In your main project, add the token as a masked CI/CD variable. For example, name it EXTERNAL_GITLAB_TOKEN.

4. In your .gitlab-ci.yml file, configure authentication based on your executor type:

   For containerized executors (Docker or Kubernetes):

   variables:
     GIT_SUBMODULE_STRATEGY: recursive
   
   my-job:
     before_script:
       - git config --global url."https://<username>:${EXTERNAL_GITLAB_TOKEN}@other-gitlab.example.com/".insteadOf "https://other-gitlab.example.com/"
     script:
       - echo "Submodules are fetched with authentication"
       - ls -la external-project/

   For shell executors:

   variables:
     GIT_SUBMODULE_STRATEGY: none
   
   my-job:
     before_script:
       - parent_include_path=$(git -C $CI_PROJECT_DIR config include.path)
       - git -c "include.path=${parent_include_path}" -c "url.https://<username>:${EXTERNAL_GITLAB_TOKEN}@other-gitlab.example.com/.insteadOf=https://other-gitlab.example.com/" submodule update --init --recursive --force
     script:
       - echo "Submodules are fetched with authentication"
       - ls -la external-project/

   Replace <username> with the GitLab username associated with the token.

   To configure authentication globally for all jobs in containerized executors only:

   hooks:
     pre_get_sources_script:
       - git config --global url."https://<username>:${EXTERNAL_GITLAB_TOKEN}@other-gitlab.example.com/".insteadOf "https://other-gitlab.example.com/"

Configure authentication with Git credential helper

To configure authentication with Git credential helper:

1. On the external GitLab instance, create a token with the read_repository scope.

2. In your main project, add the token as a masked CI/CD variable. For example, name it EXTERNAL_GITLAB_TOKEN.

3. In your .gitlab-ci.yml file, configure the credential helper based on your executor type:

   For containerized executors (Docker or Kubernetes):

   my-job:
     before_script:
       - git config --global credential.helper store
       - echo "https://<username>:${EXTERNAL_GITLAB_TOKEN}@other-gitlab.example.com" >> ~/.git-credentials
     script:
       - echo "Submodules are fetched with authentication"
       - ls -la external-project/

   For shell executors:

   my-job:
     before_script:
       - TEMP_CREDS=$(mktemp)
       - echo "https://<username>:${EXTERNAL_GITLAB_TOKEN}@other-gitlab.example.com" > "$TEMP_CREDS"
       - git config credential.helper "store --file=$TEMP_CREDS"
       - trap "rm -f $TEMP_CREDS" EXIT
     script:
       - echo "Submodules are fetched with authentication"
       - ls -la external-project/

   Replace <username> with the GitLab username associated with the token.

Troubleshooting

Can’t find the .gitmodules file

The .gitmodules file might be hard to find because it is usually a hidden file. You can check documentation for your specific OS to learn how to find and display hidden files.

If there is no .gitmodules file, it’s possible the submodule settings are in a git config file.

Error: fatal: run_command returned non-zero status

This error can happen in a job when working with submodules and the GIT_STRATEGY is set to fetch.

Setting the GIT_STRATEGY to clone should resolve the issue.

Error: fatal: could not read Username for 'https://gitlab.com': No such device or address

You might encounter this error when your CI/CD job attempts to clone, fetch, or perform other Git operations with submodules. This issue occurs when:

• Running Git commands (like git fetch) from within a submodule directory, because the externalized Git configuration may not be automatically inherited for all Git operations.
• Working with nested submodules, because GitLab Runner 18.6 and later externalizes Git configuration, which may not be automatically inherited by submodules.
• Using GitLab-hosted runners with submodules that reference https://gitlab.com, because the CI_SERVER_FQDN differs from gitlab.com. GitLab Runner automatically performs Git URL substitution during initial checkout, but this may not apply to subsequent Git operations within submodule directories.

To resolve this issue:

• For nested submodules, see check out nested submodules.

• For Git operations within submodule directories, explicitly pass the externalized configuration:

    my-job:
      script:
        - cd path/to/submodule
        - git -c "include.path=$(git -C $CI_PROJECT_DIR config include.path)" fetch origin

• For GitLab-hosted runners or jobs with multiple Git operations within submodules, configure URL substitution with CI_JOB_TOKEN:

  my-job:
    script:
      - cd path/to/submodule
      - git -c "include.path=$(git -C $CI_PROJECT_DIR config include.path)" -c "url.https://gitlab-ci-token:${CI_JOB_TOKEN}@${CI_SERVER_FQDN}/.insteadOf=https://gitlab.com/" fetch origin

  For executor-specific configuration options, see use submodules from another GitLab instance.