Clean up Rake tasks

GitLab provides Rake tasks for cleaning up GitLab instances.

Remove unreferenced LFS files

When you remove LFS files from a repository’s history, they become orphaned and continue to consume disk space. With this Rake task, you can remove invalid references from the database, which allows garbage collection of LFS files. For example:

sudo gitlab-rake gitlab:cleanup:orphan_lfs_file_references PROJECT_PATH="gitlab-org/gitlab-foss"

bundle exec rake gitlab:cleanup:orphan_lfs_file_references RAILS_ENV=production PROJECT_PATH="gitlab-org/gitlab-foss"

You can also specify the project with PROJECT_ID instead of PROJECT_PATH.

For example:

$ sudo gitlab-rake gitlab:cleanup:orphan_lfs_file_references PROJECT_ID="13083"

I, [2019-12-13T16:35:31.764962 #82356]  INFO -- :  Looking for orphan LFS files for project GitLab Org / GitLab Foss
I, [2019-12-13T16:35:31.923659 #82356]  INFO -- :  Removed invalid references: 12

By default, this task does not delete anything but shows how many file references it can delete. Run the command with DRY_RUN=false if you actually want to delete the references. You can also use LIMIT={number} parameter to limit the number of deleted references.

This Rake task only removes the references to LFS files. Unreferenced LFS files are garbage-collected later (once a day). If you need to garbage collect them immediately, run rake gitlab:cleanup:orphan_lfs_files described below.

Remove unreferenced LFS files immediately

Unreferenced LFS files are removed on a daily basis but you can remove them immediately if you need to. To remove unreferenced LFS files immediately:

sudo gitlab-rake gitlab:cleanup:orphan_lfs_files

bundle exec rake gitlab:cleanup:orphan_lfs_files

Example output:

$ sudo gitlab-rake gitlab:cleanup:orphan_lfs_files
I, [2020-01-08T20:51:17.148765 #43765]  INFO -- : Removed unreferenced LFS files: 12

Clean up project upload files

Clean up project upload files if they don’t exist in GitLab database.

Clean up project upload files from file system

Clean up local project upload files if they don’t exist in GitLab database. The task attempts to fix the file if it can find its project, otherwise it moves the file to a lost and found directory. To clean up project upload files from file system:

sudo gitlab-rake gitlab:cleanup:project_uploads

bundle exec rake gitlab:cleanup:project_uploads RAILS_ENV=production

Example output:

$ sudo gitlab-rake gitlab:cleanup:project_uploads

I, [2018-07-27T12:08:27.671559 #89817]  INFO -- : Looking for orphaned project uploads to clean up. Dry run...
D, [2018-07-27T12:08:28.293568 #89817] DEBUG -- : Processing batch of 500 project upload file paths, starting with /opt/gitlab/embedded/service/gitlab-rails/public/uploads/test.out
I, [2018-07-27T12:08:28.689869 #89817]  INFO -- : Can move to lost and found /opt/gitlab/embedded/service/gitlab-rails/public/uploads/test.out -> /opt/gitlab/embedded/service/gitlab-rails/public/uploads/-/project-lost-found/test.out
I, [2018-07-27T12:08:28.755624 #89817]  INFO -- : Can fix /opt/gitlab/embedded/service/gitlab-rails/public/uploads/foo/bar/89a0f7b0b97008a4a18cedccfdcd93fb/foo.txt -> /opt/gitlab/embedded/service/gitlab-rails/public/uploads/qux/foo/bar/89a0f7b0b97008a4a18cedccfdcd93fb/foo.txt
I, [2018-07-27T12:08:28.760257 #89817]  INFO -- : Can move to lost and found /opt/gitlab/embedded/service/gitlab-rails/public/uploads/foo/bar/1dd6f0f7eefd2acc4c2233f89a0f7b0b/image.png -> /opt/gitlab/embedded/service/gitlab-rails/public/uploads/-/project-lost-found/foo/bar/1dd6f0f7eefd2acc4c2233f89a0f7b0b/image.png
I, [2018-07-27T12:08:28.764470 #89817]  INFO -- : To cleanup these files run this command with DRY_RUN=false

$ sudo gitlab-rake gitlab:cleanup:project_uploads DRY_RUN=false
I, [2018-07-27T12:08:32.944414 #89936]  INFO -- : Looking for orphaned project uploads to clean up...
D, [2018-07-27T12:08:33.293568 #89817] DEBUG -- : Processing batch of 500 project upload file paths, starting with /opt/gitlab/embedded/service/gitlab-rails/public/uploads/test.out
I, [2018-07-27T12:08:33.689869 #89817]  INFO -- : Did move to lost and found /opt/gitlab/embedded/service/gitlab-rails/public/uploads/test.out -> /opt/gitlab/embedded/service/gitlab-rails/public/uploads/-/project-lost-found/test.out
I, [2018-07-27T12:08:33.755624 #89817]  INFO -- : Did fix /opt/gitlab/embedded/service/gitlab-rails/public/uploads/foo/bar/89a0f7b0b97008a4a18cedccfdcd93fb/foo.txt -> /opt/gitlab/embedded/service/gitlab-rails/public/uploads/qux/foo/bar/89a0f7b0b97008a4a18cedccfdcd93fb/foo.txt
I, [2018-07-27T12:08:33.760257 #89817]  INFO -- : Did move to lost and found /opt/gitlab/embedded/service/gitlab-rails/public/uploads/foo/bar/1dd6f0f7eefd2acc4c2233f89a0f7b0b/image.png -> /opt/gitlab/embedded/service/gitlab-rails/public/uploads/-/project-lost-found/foo/bar/1dd6f0f7eefd2acc4c2233f89a0f7b0b/image.png

If using object storage, run the All-in-one Rake task to ensure all uploads are migrated to object storage and there are no files on disk in the uploads folder.

Clean up project upload files from object storage

Move object store upload files to a lost and found directory if they don’t exist in GitLab database. To clean up project upload files from object storage:

sudo gitlab-rake gitlab:cleanup:remote_upload_files

bundle exec rake gitlab:cleanup:remote_upload_files RAILS_ENV=production

Example output:

$ sudo gitlab-rake gitlab:cleanup:remote_upload_files

I, [2018-08-02T10:26:13.995978 #45011]  INFO -- : Looking for orphaned remote uploads to remove. Dry run...
I, [2018-08-02T10:26:14.120400 #45011]  INFO -- : Can be moved to lost and found: @hashed/6b/DSC_6152.JPG
I, [2018-08-02T10:26:14.120482 #45011]  INFO -- : Can be moved to lost and found: @hashed/79/02/7902699be42c8a8e46fbbb4501726517e86b22c56a189f7625a6da49081b2451/711491b29d3eb08837798c4909e2aa4d/DSC00314.jpg
I, [2018-08-02T10:26:14.120634 #45011]  INFO -- : To cleanup these files run this command with DRY_RUN=false

$ sudo gitlab-rake gitlab:cleanup:remote_upload_files DRY_RUN=false

I, [2018-08-02T10:26:47.598424 #45087]  INFO -- : Looking for orphaned remote uploads to remove...
I, [2018-08-02T10:26:47.753131 #45087]  INFO -- : Moved to lost and found: @hashed/6b/DSC_6152.JPG -> lost_and_found/@hashed/6b/DSC_6152.JPG
I, [2018-08-02T10:26:47.764356 #45087]  INFO -- : Moved to lost and found: @hashed/79/02/7902699be42c8a8e46fbbb4501726517e86b22c56a189f7625a6da49081b2451/711491b29d3eb08837798c4909e2aa4d/DSC00314.jpg -> lost_and_found/@hashed/79/02/7902699be42c8a8e46fbbb4501726517e86b22c56a189f7625a6da49081b2451/711491b29d3eb08837798c4909e2aa4d/DSC00314.jpg

Remove orphan artifact files

When you notice there are more job artifacts files and/or directories on disk than there should be, you can run:

sudo gitlab-rake gitlab:cleanup:orphan_job_artifact_files

This command:

• Scans through the entire artifacts folder.
• Checks which files still have a record in the database.
• If no database record is found, the file and directory are deleted from disk.

By default, this task does not delete anything but shows what it can delete. Run the command with DRY_RUN=false if you actually want to delete the files:

sudo gitlab-rake gitlab:cleanup:orphan_job_artifact_files DRY_RUN=false

You can also limit the number of files to delete with LIMIT (default 100):

sudo gitlab-rake gitlab:cleanup:orphan_job_artifact_files LIMIT=100

This deletes only up to 100 files from disk. You can use this to delete a small set for testing purposes.

Providing DEBUG=1 displays the full path of every file that is detected as being an orphan.

If ionice is installed, the tasks uses it to ensure the command is not causing too much load on the disk. You can configure the niceness level with NICENESS. Below are the valid levels, but consult man 1 ionice to be sure.

• 0 or None
• 1 or Realtime
• 2 or Best-effort (default)
• 3 or Idle

Clean up orphaned package files

When you delete a project, its package files can remain on disk in the package registry local storage path. To find and move these orphaned package files:

sudo gitlab-rake gitlab:cleanup:orphan_package_files

bundle exec rake gitlab:cleanup:orphan_package_files RAILS_ENV=production

This task:

• Scans the top-level hashed directories of the packages storage path.
• Checks each directory against the projects in the database, and the groups that have Debian distributions.
• Lists each directory without a matching project or group, with its size and the total size.
• Skips a directory with a file changed in the last 24 hours. A later run handles it.

By default, the task doesn’t change anything. To move the orphaned directories to the lost and found directory, run the task with DRY_RUN=false:

sudo gitlab-rake gitlab:cleanup:orphan_package_files DRY_RUN=false

The task moves directories to <packages storage path>/-/package-lost-found/ and keeps their relative paths. The task never deletes files. If the task cannot measure or move a directory, it logs an error for that directory and ends with the count of failed directories. After you confirm the package registry works as expected, delete the lost and found directory. To restore a directory, move it back to the same relative path under the packages storage path.

Example output:

$ sudo gitlab-rake gitlab:cleanup:orphan_package_files

I, [2026-10-07T10:15:02.123456 #4321]  INFO -- : Looking for orphaned package files to clean up. Dry run...
I, [2026-10-07T10:15:03.234567 #4321]  INFO -- : Can move to lost and found /var/opt/gitlab/gitlab-rails/shared/packages/4f/c8/4fc82b26aecb47d2868c4efbe3581732a3e7cbcc6c2efb32062c08170a05eeb8 -> /var/opt/gitlab/gitlab-rails/shared/packages/-/package-lost-found/4f/c8/4fc82b26aecb47d2868c4efbe3581732a3e7cbcc6c2efb32062c08170a05eeb8 (12.5 MB)
I, [2026-10-07T10:15:03.345678 #4321]  INFO -- : Found 1 orphaned package directories, total size 12.5 MB
I, [2026-10-07T10:15:03.456789 #4321]  INFO -- : To clean up these files run this command with DRY_RUN=false

$ sudo gitlab-rake gitlab:cleanup:orphan_package_files DRY_RUN=false

I, [2026-10-07T10:16:10.123456 #4400]  INFO -- : Looking for orphaned package files to clean up...
I, [2026-10-07T10:16:11.234567 #4400]  INFO -- : Did move to lost and found /var/opt/gitlab/gitlab-rails/shared/packages/4f/c8/4fc82b26aecb47d2868c4efbe3581732a3e7cbcc6c2efb32062c08170a05eeb8 -> /var/opt/gitlab/gitlab-rails/shared/packages/-/package-lost-found/4f/c8/4fc82b26aecb47d2868c4efbe3581732a3e7cbcc6c2efb32062c08170a05eeb8 (12.5 MB)
I, [2026-10-07T10:16:11.345678 #4400]  INFO -- : Moved 1 orphaned package directories, total size 12.5 MB

Remove orphaned pool repository records for removed storages

If you permanently remove a Gitaly storage from your configuration, database records for object deduplication pools on that storage can be left behind. These records cannot be cleaned up by regular housekeeping because the storage they reference no longer exists.

This Rake task deletes pool_repositories database records that:

• Are on one of the storages you specify.
• Have no source project.
• Are not referenced by any project.

Records still referenced by a project are never deleted, and are reported in the task output instead.

By default, the task runs in dry-run mode: it writes the records it would delete to the CSV file given in OUTPUT_FILE without deleting anything. Review that file first:

sudo gitlab-rake gitlab:pool_repositories:cleanup_orphaned_on_missing_shards \
  SHARD_NAMES=old-storage-1,old-storage-2 OUTPUT_FILE=/tmp/orphaned_pools.csv

To delete the records, run the command again with DRY_RUN=false. Set the required DELETED_IDS_FILE to a path different from OUTPUT_FILE:

sudo gitlab-rake gitlab:pool_repositories:cleanup_orphaned_on_missing_shards \
  SHARD_NAMES=old-storage-1,old-storage-2 OUTPUT_FILE=/tmp/deleted_pools.csv \
  DELETED_IDS_FILE=/tmp/deleted_pool_ids.csv DRY_RUN=false

OUTPUT_FILE is a recovery CSV with all database columns and the shard name. Keep this file safe for recovery. The task writes and flushes each batch to the CSV before deletion, then deletes only backed-up IDs that still meet the original conditions. Backed-up records can survive if they gain a source or member project. Newly eligible records without a backup wait for the next run.

Real runs also create the file specified by DELETED_IDS_FILE, an audit CSV with a single Pool ID column from the actual DELETE RETURNING results. This file identifies deleted records, while the recovery CSV can include records that survived. A crash after deletion but before the audit write can leave the audit incomplete. The recovery data has already been written. Dry runs require only OUTPUT_FILE and create only the recovery CSV.

Completion logs report the total records in the recovery CSV and the total deleted IDs. The CSV files contain headers and data only, with no total row. The task refuses to run if any file it would write already exists. Use a new output path for each run.

Remove expired ActiveSession lookup keys

To remove expired ActiveSession lookup keys:

sudo gitlab-rake gitlab:cleanup:sessions:active_sessions_lookup_keys

bundle exec rake gitlab:cleanup:sessions:active_sessions_lookup_keys RAILS_ENV=production

Container registry garbage collection

Container Registry can use considerable amounts of disk space. To clear up unused layers, the registry includes a garbage collect command.