• Log Levels
  • Override default log level
• Log Rotation
• production_json.log
• production.log
• api_json.log
• application.log
• application_json.log
• integrations_json.log
• kubernetes.log
• git_json.log
• audit_json.log
• Sidekiq Logs
  • sidekiq.log
  • sidekiq_client.log
• gitlab-shell.log
  • For GitLab versions 12.10 and up
  • For GitLab versions 12.5 through 12.9
  • For GitLab 12.5 and earlier
• Gitaly Logs
  • grpc.log
  • gitaly_ruby_json.log
  • gitaly_hooks.log
• Puma Logs
  • puma_stdout.log
  • puma_stderr.log
• repocheck.log
• importer.log
• exporter.log
• features_json.log
• auth.log
• graphql_json.log
• migrations.log
• mail_room_json.log (default)
• Reconfigure logs
• sidekiq_exporter.log and web_exporter.log
• database_load_balancing.log
• elasticsearch.log
• exceptions_json.log
• service_measurement.log
• geo.log
• update_mirror_service_json.log
• Registry Logs
• NGINX Logs
• Pages Logs
• Mattermost Logs
• Workhorse Logs
• PostgreSQL Logs
• Prometheus Logs
• Redis Logs
• Alertmanager Logs
• Crond Logs
• Grafana Logs
• LogRotate Logs
• GitLab Monitor Logs
• GitLab Exporter
• GitLab agent server
• Praefect Logs
• Backup log
• Performance bar stats
• Gathering logs
  • Briefly tail the main logs
  • GitLabSOS
  • Fast-stats

Log system

GitLab has an advanced log system where everything is logged, so you can analyze your instance using various system log files. In addition to system log files, GitLab Enterprise Edition provides Audit Events.

System log files are typically plain text in a standard log file format. This guide talks about how to read and use these system log files.

Read more about the log system and using the logs:

• Customize logging on Omnibus GitLab installations including adjusting log retention, log forwarding, switching logs from JSON to plain text logging, and more.
• How to parse and analyze JSON logs.

Log Levels

Each log message has an assigned log level that indicates its importance and verbosity. Each logger has an assigned minimum log level. A logger emits a log message only if its log level is equal to or above the minimum log level.

The following log levels are supported:

GitLab loggers emit all log messages because they are set to DEBUG by default.

Override default log level

You can override the minimum log level for GitLab loggers using the GITLAB_LOG_LEVEL environment variable. Valid values are either a value of 0 to 5, or the name of the log level.

Example:

GITLAB_LOG_LEVEL=info

For some services, other log levels are in place that are not affected by this setting. Some of these services have their own environment variables to override the log level. For example:

Log Rotation

The logs for a given service may be managed and rotated by:

• logrotate
• svlogd (runit’s service logging daemon)
• logrotate and svlogd
• Or not at all

The following table includes information about what’s responsible for managing and rotating logs for the included services. Logs managed by svlogd are written to a file called current. The logrotate service built into GitLab manages all logs except those captured by runit.

production_json.log

Depending on your installation method, this file is located at:

• Omnibus GitLab: /var/log/gitlab/gitlab-rails/production_json.log
• Installations from source: /home/git/gitlab/log/production_json.log

It contains a structured log for Rails controller requests received from GitLab, thanks to Lograge. Requests from the API are logged to a separate file in api_json.log.

Each line contains JSON that can be ingested by services like Elasticsearch and Splunk. Line breaks were added to examples for legibility:

{
  "method":"GET",
  "path":"/gitlab/gitlab-foss/issues/1234",
  "format":"html",
  "controller":"Projects::IssuesController",
  "action":"show",
  "status":200,
  "time":"2017-08-08T20:15:54.821Z",
  "params":[{"key":"param_key","value":"param_value"}],
  "remote_ip":"18.245.0.1",
  "user_id":1,
  "username":"admin",
  "queue_duration_s":0.0,
  "gitaly_calls":16,
  "gitaly_duration_s":<