Working with Prometheus Metrics
Adding to the library
We strive to support the 2-4 most important metrics for each common system service that supports Prometheus. If you are looking for support for a particular exporter which has not yet been added to the library, additions can be made to the common_metrics.yml
file.
Query identifier
The requirement for adding a new metric is to make each query to have an unique identifier which is used to update the metric later when changed:
- group: Response metrics (NGINX Ingress)
metrics:
- title: "Throughput"
y_axis:
name: "Requests / Sec"
format: "number"
precision: 2
queries:
- id: response_metrics_nginx_ingress_throughput_status_code
query_range: 'sum(rate(nginx_upstream_responses_total{upstream=~"%{kube_namespace}-%{ci_environment_slug}-.*"}[2m])) by (status_code)'
unit: req / sec
label: Status Code
Update existing metrics
After you add or change an existing common metric, you must re-run the import script that will query and update all existing metrics.
Or, you can create a database migration:
id:
) is removed it will not be removed from database by default.
You might want to add additional database migration that makes a decision what to do with removed one.
For example: you might be interested in migrating all dependent data to a different metric.class ImportCommonMetrics < ActiveRecord::Migration[4.2]
include Gitlab::Database::MigrationHelpers
DOWNTIME = false
def up
::Gitlab::DatabaseImporters::CommonMetrics::Importer.new.execute
end
def down
# no-op
end
end
GitLab Prometheus metrics
GitLab provides Prometheus metrics to monitor itself.
Adding a new metric
This section describes how to add new metrics for self-monitoring (example).
-
Select the type of metric:
Gitlab::Metrics.counter
Gitlab::Metrics.gauge
Gitlab::Metrics.histogram
Gitlab::Metrics.summary
- Select the appropriate name for your metric. Refer to the guidelines for Prometheus metric names.
- Update the list of GitLab Prometheus metrics.
- Trigger the relevant page/code that will record the new metric.
- Check that the new metric appears at
/-/metrics
.
Help and feedback
If there's something you don't like about this feature
To propose functionality that GitLab does not yet offer
To further help GitLab in shaping new features
If you didn't find what you were looking for
If you want help with something very specific to your use case, and can use some community support
POST ON GITLAB FORUM
If you have problems setting up or using this feature (depending on your GitLab subscription)
REQUEST SUPPORT
To view all GitLab tiers and features or to upgrade
If you want to try all features available in GitLab.com
If you want to try all features available in GitLab self-managed
If you spot an error or a need for improvement and would like to fix it yourself in a merge request
EDIT THIS PAGE
If you would like to suggest an improvement to this doc