GitLab Prometheus metrics development guidelines
GitLab provides Prometheus metrics to monitor itself.
Adding a new metric
This section describes how to add new metrics for self-monitoring (example).
Select the type of metric:
Gitlab::Metrics.counter
Gitlab::Metrics.gauge
Gitlab::Metrics.histogram
Gitlab::Metrics.summary
Select the appropriate name for your metric. Refer to the guidelines for Prometheus metric names.
Update the list of GitLab Prometheus metrics.
Carefully choose what labels you want to add to your metric. Values with high cardinality, like
project_path
, orproject_id
are strongly discouraged because they can affect our services availability due to the fact that each set of labels is exposed as a new entry in the/metrics
endpoint. For example, a histogram with 10 buckets and a label with 100 values would generate 1000 entries in the export endpoint.Trigger the relevant page or code that records the new metric.
Check that the new metric appears at
/-/metrics
.
For metrics that are not bounded to a specific context (request
, process
, machine
, namespace
, etc),
generate them from a cron-based Sidekiq job:
- For Geo related metrics, check
Geo::MetricsUpdateService
. - For other “global” / instance-wide metrics, check:
Metrics::GlobalMetricsUpdateService
.
When exporting data from Sidekiq in an installation with more than one Sidekiq instance, you are not guaranteed that the same exporter will always be queried.
You can read more and understand the caveats in issue 406583, where we also discuss a possible solution using a push-gateway.
Docs
Edit this page to fix an error or add an improvement in a merge request.
Create an issue to suggest an improvement to this page.
Product
Create an issue if there's something you don't like about this feature.
Propose functionality by submitting a feature request.
Feature availability and product trials
View pricing to see all GitLab tiers and features, or to upgrade.
Try GitLab for free with access to all features for 30 days.
Get help
If you didn't find what you were looking for, search the docs.
If you want help with something specific and could use community support, post on the GitLab forum.
For problems setting up or using this feature (depending on your GitLab subscription).
Request support