- Requirements
- Metrics supported
- Configuring NGINX Ingress monitoring
- Specifying the Environment label
Monitoring NGINX Ingress Controller with VTS metrics
Introduced in GitLab 9.5.
GitLab has support for automatically detecting and monitoring the Kubernetes NGINX Ingress controller. This is provided by leveraging the included VTS Prometheus metrics exporter in version 0.9.0 through 0.15.x.
Requirements
Prometheus integration must be active.
Metrics supported
Name | Query |
---|---|
Throughput (req/sec) | sum(rate(nginx_upstream_responses_total{upstream=~"%{kube_namespace}-%{ci_environment_slug}-.*"}[2m])) by (status_code)
|
Latency (ms) | avg(nginx_upstream_response_msecs_avg{upstream=~"%{kube_namespace}-%{ci_environment_slug}-.*"})
|
HTTP Error Rate (%) | sum(rate(nginx_upstream_responses_total{status_code="5xx", upstream=~"%{kube_namespace}-%{ci_environment_slug}-.*"}[2m])) / sum(rate(nginx_upstream_responses_total{upstream=~"%{kube_namespace}-%{ci_environment_slug}-.*"}[2m])) * 100
|
Configuring NGINX Ingress monitoring
If you have deployed NGINX Ingress using GitLab’s Kubernetes cluster integration, it will automatically be monitored by Prometheus.
For other deployments, there is some configuration required depending on your installation:
- NGINX Ingress should be version 0.9.0 or above, with metrics enabled.
- NGINX Ingress should be annotated for Prometheus monitoring.
- Prometheus should be configured to monitor annotated pods.
About managed NGINX Ingress deployments
NGINX Ingress is deployed into the gitlab-managed-apps
namespace, using the official Helm chart. NGINX Ingress will be externally reachable via the Load Balancer’s Endpoint.
NGINX is configured for Prometheus monitoring, by setting:
-
enable-vts-status: "true"
, to export Prometheus metrics. -
prometheus.io/scrape: "true"
, to enable automatic discovery. -
prometheus.io/port: "10254"
, to specify the metrics port.
When used in conjunction with the GitLab deployed Prometheus service, response metrics will be automatically collected.
Manually setting up NGINX Ingress for Prometheus monitoring
Version 0.9.0 and above of NGINX Ingress has built-in support for exporting Prometheus metrics. To enable, a ConfigMap setting must be passed: enable-vts-status: "true"
. Once enabled, a Prometheus metrics endpoint will start running on port 10254.
Next, the Ingress needs to be annotated for Prometheus monitoring. Two new annotations need to be added:
prometheus.io/scrape: "true"
prometheus.io/port: "10254"
Managing these settings depends on how NGINX Ingress has been deployed. If you have deployed via the official Helm chart, metrics can be enabled with controller.stats.enabled
along with the required annotations. Alternatively it is possible edit the NGINX Ingress YAML directly in the Kubernetes dashboard.
Specifying the Environment label
In order to isolate and only display relevant metrics for a given environment, GitLab needs a method to detect which labels are associated. To do this, GitLab will search for metrics with appropriate labels. In this case, the upstream
label must be of the form <KUBE_NAMESPACE>-<CI_ENVIRONMENT_SLUG>-*
.
If you have used Auto Deploy to deploy your app, this format will be used automatically and metrics will be detected with no action on your part.
Help and feedback
If there's something you don't like about this feature
To propose functionality that GitLab does not yet offer
To further help GitLab in shaping new features
If you didn't find what you were looking for
If you want help with something very specific to your use case, and can use some community support
POST ON GITLAB FORUM
If you have problems setting up or using this feature (depending on your GitLab subscription)
REQUEST SUPPORT
To view all GitLab tiers and features or to upgrade
If you want to try all features available in GitLab.com
If you want to try all features available in GitLab self-managed
If you spot an error or a need for improvement and would like to fix it yourself in a merge request
EDIT THIS PAGE
If you would like to suggest an improvement to this doc