Local setup and debugging

note
To track user interactions in the browser, Do-Not-Track (“DNT”) needs to be disabled. DNT is disabled by default for most browsers.

Internal events are using a tool called Snowplow under the hood. To develop and test internal events, there are several tools to test frontend and backend events:

Testing Tool Frontend Tracking Backend Tracking Local Development Environment Production Environment Shows individual events
Internal Events Monitor Yes Yes Yes Yes Yes, if running Snowplow Micro
Snowplow Micro Yes Yes Yes No Yes
Manual check in GDK Yes Yes Yes Yes No
Snowplow Analytics Debugger Chrome Extension Yes No Yes Yes Yes
Remote event collector Yes No Yes No Yes

For local development we recommend using the internal events monitor when actively developing new events.

Internal Events Monitor

Watch the demo video about the Internal Events Tracking Monitor

To understand how events are triggered and metrics are updated while you use the GitLab application locally or rails console, you can use the monitor.

Start the monitor and list one or more events that you would like to monitor. In this example we would like to monitor i_code_review_user_create_mr. 

rails runner scripts/internal_events/monitor.rb i_code_review_user_create_mr

The monitor can show two tables:

  • The RELEVANT METRICS table lists all the metrics that are defined on the i_code_review_user_create_mr event. The second right-most column shows the value of each metric when the monitor was started and the right most column shows the current value of each metric.

  • The SNOWPLOW EVENTS table lists a selection of properties from all Snowplow events that match the event name. This table is only visible if you also set up Snowplow Micro.

If a new i_code_review_user_create_mr event is fired, the metrics values get updated and a new event appears in the SNOWPLOW EVENTS table.

The monitor looks like below. 

Updated at 2023-10-11 10:17:59 UTC
Monitored events: i_code_review_user_create_mr

+--------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|                                                                          RELEVANT METRICS                                                                          |
+-----------------------------------------------------------------------------+------------------------------+-----------------------+---------------+---------------+
| Key Path                                                                    | Monitored Events             | Instrumentation Class | Initial Value | Current Value |
+-----------------------------------------------------------------------------+------------------------------+-----------------------+---------------+---------------+
| counts_monthly.aggregated_metrics.code_review_category_monthly_active_users | i_code_review_user_create_mr | RedisHLLMetric        | 13            | 14            |
| counts_monthly.aggregated_metrics.code_review_group_monthly_active_users    | i_code_review_user_create_mr | RedisHLLMetric        | 13            | 14            |
| counts_weekly.aggregated_metrics.code_review_category_monthly_active_users  | i_code_review_user_create_mr | RedisHLLMetric        | 0             | 1             |
| counts_weekly.aggregated_metrics.code_review_group_monthly_active_users     | i_code_review_user_create_mr | RedisHLLMetric        | 0             | 1             |
| redis_hll_counters.code_review.i_code_review_user_create_mr_monthly         | i_code_review_user_create_mr | RedisHLLMetric        | 8             | 9             |
| redis_hll_counters.code_review.i_code_review_user_create_mr_weekly          | i_code_review_user_create_mr | RedisHLLMetric        | 0             | 1             |
+-----------------------------------------------------------------------------+------------------------------+-----------------------+---------------+---------------+
+---------------------------------------------------------------------------------------------------------+
|                                             SNOWPLOW EVENTS                                             |
+------------------------------+--------------------------+---------+--------------+------------+---------+
| Event Name                   | Collector Timestamp      | user_id | namespace_id | project_id | plan    |
+------------------------------+--------------------------+---------+--------------+------------+---------+
| i_code_review_user_create_mr | 2023-10-11T10:17:15.504Z | 29      | 93           |            | default |
+------------------------------+--------------------------+---------+--------------+------------+---------+

Snowplow Micro

By default, self-managed instances do not collect event data through Snowplow. We can use Snowplow Micro, a Docker based Snowplow collector, to test events locally:

  1. Ensure Docker is installed and working.

  2. Enable Snowplow Micro: 

    gdk config set snowplow_micro.enabled true

  3. Optional. Snowplow Micro runs on port 9091 by default, you can change to 9092 by running: 

    gdk config set snowplow_micro.port 9092

  4. Regenerate your Procfile and YAML configuration by reconfiguring GDK: 

    gdk reconfigure

  5. Restart the GDK: 

    gdk restart

  6. You can now see all events being sent by your local instance in the Snowplow Micro UI and can filter for specific events.

Introduction to Snowplow Micro UI and API

Watch the video about Snowplow Micro

Manual check in GDK

As a quick test of whether an event is getting triggered & metric is updated, you can check the latest values in the rails console. Make sure to load the helpers below so that the most recent events & records are included in the output.

To view the entire service ping payload: 

require_relative 'spec/support/helpers/service_ping_helpers.rb'
ServicePingHelpers.get_current_service_ping_payload

To view the current value for a specific metric: 

require_relative 'spec/support/helpers/service_ping_helpers.rb'
ServicePingHelpers.get_current_usage_metric_value(key_path)

Snowplow Analytics Debugger Chrome Extension

Snowplow Analytics Debugger is a browser extension for testing frontend events. It works in production, staging, and local development environments. It is especially suited to verifying correct events are getting sent in a deployed environment.

  1. Install the Snowplow Analytics Debugger Chrome browser extension.
  2. Open Chrome DevTools to the Snowplow Debugger tab.
  3. Any event triggered on a GitLab page should appear in the Snowplow Debugger tab.

Remote event collector

On GitLab.com events are sent to a collector configured by GitLab. By default, self-managed instances do not have a collector configured and do not collect data with Snowplow.

You can configure your self-managed GitLab instance to use a custom Snowplow collector.

  1. On the left sidebar, at the bottom, select Admin.
  2. Select Settings > General.
  3. Expand Snowplow.

  4. Select Enable Snowplow tracking and enter your Snowplow configuration information. For example if your custom snowplow collector is available at your-snowplow-collector.net:

    Name Value
    Collector hostname your-snowplow-collector.net
    App ID gitlab
    Cookie domain .your-gitlab-instance.com
  5. Select Save changes.