AI Event Instrumentation Guide

This guide describes all fields required for instrumenting AI and GitLab Duo Agentic Platform (DAP) events. AI events use two context schemas:

• Standard Context (iglu:com.gitlab/gitlab_standard/jsonschema/*) - Contains general fields used across all GitLab events, including AI-specific fields like model information and token tracking
• AI Context (iglu:com.gitlab/ai_context/jsonschema/*) - Contains DAP-specific fields for workflow and session management

Overview

When instrumenting AI events, you need to include fields from both contexts to provide complete tracking information. This page consolidates all required and optional fields in one place, clearly indicating which context each field belongs to.

When to Use This Guide

Use this guide when:

• Instrumenting GitLab Duo features (Duo Chat, Duo Workflow, AI-powered suggestions)
• Tracking DAP (Duo Agentic Platform) events
• Recording AI model interactions and token usage
• Monitoring AI session and workflow execution
• Events have classification: duo in their event definition

Field Reference by Context

The tables below show all available fields for AI events, organized by which context they belong to.

Standard Context Fields (Required)

These fields are part of the Standard Context and are required for all events.

Standard Context Fields (Optional)

These fields are part of the Standard Context and should be included when available. For a complete reference of all Standard Context fields including descriptions, types, and examples, see the Standard Context Fields documentation.

Key Standard Context fields for AI events include:

• User and instance information - user_id, instance_id, host_name, realm
• Project and namespace context - project_id, namespace_id, ultimate_parent_namespace_id
• Client information - client_name, client_version, client_type, interface
• Billing and correlation - correlation_id, billing_event_id, feature_enablement_type
• Model information - model_provider, model_engine, model_name

AI Context Fields

These fields are part of the AI Context schema and are specific to DAP (Duo Agentic Platform) and AI Gateway events.

Session and Workflow Identifiers

Workflow and Agent Information

Token Tracking

Token tracking fields capture the usage of AI model tokens for cost and performance monitoring.

Complete Instrumentation Example

Here’s a complete example showing how to instrument a DAP event with both Standard Context and AI Context fields:

track_internal_event(
  "request_duo_workflow_success",
  # Standard Context fields (automatically included)
  user: user,
  project: project,
  namespace: namespace,
  
  # Additional Standard Context fields for AI events
  additional_properties: {
    # Model information (Standard Context)
    model_provider: "anthropic",
    model_engine: "claude-3-5",
    model_name: "claude-3-5-sonnet-20241022",
    
    # Feature and billing (Standard Context)
    feature_enablement_type: "duo_pro",
    app_id: "gitlab_duo_workflow",
    correlation_id: SecureRandom.uuid,
    billing_event_id: SecureRandom.uuid
  },
  
  # AI Context fields
  ai_context: {
    # Session and workflow identifiers
    session_id: session.id,
    
    # Flow and agent information
    flow_type: "software_development",
    agent_name: "code_generator",
    agent_type: "code_agent",
    flow_version: "2.1.0",
    
    # Token tracking (AI Context)
    input_tokens: 1500,
    output_tokens: 800,
    total_tokens: 2300,
    ephemeral_5m_input_tokens: 500,
    ephemeral_1h_input_tokens: 1000,
    cache_read: 200
  }
)

For additional information about AI Gateway trigger events and instrumentation patterns, see the AI Gateway instrumentation documentation.

Session-Level Events

The following events illustrate the lifecycle of a DAP session and should be tracked for most flows:

Field Details and Best Practices

Session and Workflow Identifiers

session_id (AI Context)

• Local identifier for a user’s session within a specific GitLab instance
• Generated by the instance
• Not globally unique across different instances
• Used for local session tracking and correlation

Token Tracking Best Practices

When tracking token usage in AI Context:

1. Always include total_tokens when tracking AI model interactions
2. Track both input and output tokens separately for accurate billing
3. Record cache usage (cache_read, ephemeral_5m_input_tokens, ephemeral_1h_input_tokens) to monitor cache effectiveness
4. Include model information (model_provider, model_engine, model_name) in Standard Context to enable model-specific analysis

Billing and Attribution

For proper billing and customer attribution:

1. Include correlation_id (Standard Context) - Critical for joining DAP events to billable events
2. Include billing_event_id (Standard Context) - Links to billable usage events
3. Include ultimate_parent_namespace_id (Standard Context) - Ensures customer attribution aligns with usage billing
4. Include feature_enabled_by_namespace_ids (Standard Context) - Current method for customer attribution in AI data models

Adding New Fields

Adding a Field to Standard Context

To add a new field to the Standard Context:

1. Create a merge request in the iglu repository to update the schema.

2. If the new field should be pseudonymized, add it to the ATTRIBUTE_TO_PSEUDONYMISE constant in the snowplow-pseudonymization project.

3. Update the GITLAB_STANDARD_SCHEMA_URL in tracking/standard_context.rb to match the new version from gitlab-org/iglu.

4. Start sending events that include the new field in Standard Context.

5. Update the Standard Context Fields documentation and this page.

Adding a Field to AI Context

You can add new fields to the AI Context if you want to track new properties that most AI events have in common.

To add a new field to the AI Context:

1. Create a merge request in the iglu repository to update the schema.

2. If the new field should be pseudonymized, add it to the appropriate pseudonymization configuration in the snowplow-pseudonymization project.

3. Update the AI Context implementation in the GitLab codebase to support the new field.

4. Start sending events that include the new field in AI Context.

5. Update this documentation to describe the new field.

Related Documentation

• Standard Context Fields - Complete reference for all Standard Context fields
• Internal Event Instrumentation - General guide for instrumenting internal events
• Cloud Connector Unit Primitives - Descriptions of Unit Primitives