- What is an experiment?
- Model candidate
- Disabling or enabling the Feature
- Feedback, roadmap and reports
When creating machine learning models, data scientists often experiment with different parameters, configurations, feature engineering, and so on, to improve the performance of the model. Keeping track of all this metadata and the associated artifacts so that the data scientist can later replicate the experiment is not trivial. Machine learning experiment tracking enables them to log parameters, metrics, and artifacts directly into GitLab, giving easy access later on.
An experiment is a collection of comparable model candidates. Experiments can be long lived (for example, when they represent a use case), or short lived (results from hyperparameter tuning triggered by a merge request), but usually hold model candidates that have a similar set of parameters and metrics.
A model candidate is a variation of the training of a machine learning model, that can be eventually promoted to a version of the model. The goal of a data scientist is to find the model candidate whose parameter values lead to the best model performance, as indicated by the given metrics.
- Algorithm (linear regression, decision tree, and so on).
- Hyperparameters for the algorithm (learning rate, tree depth, number of epochs).
- Features included.
An experiment is always associated to a project. Only users with access to the project an experiment is associated with can view that experiment data.
To list the current active experiments, navigate to
https/-/ml/experiments. To display all trials
that have been logged, along with their metrics and parameters, select an experiment. To display details for a candidate,
Trial artifacts are saved as generic packages, and follow all their
conventions. After an artifact is logged for a candidate, all artifacts logged for the candidate are listed in the
package registry. The package name for a candidate is
ml_candidate_<candidate_id>, with version
-. The link to the
artifacts can also be accessed from the Experiment Candidates list or Candidate detail.
- Searching experiments, searching trials, visual comparison of trials, and creating, deleting and updating experiments and trials through GitLab UI is under development.
On self-managed GitLab, ML Experiment Tracking is disabled by default. To enable the feature, ask an administrator to disable the feature flag named
On GitLab.com, this feature is currently on private testing.
For updates on the development, refer to the development epic.
For feedback, bug reports and feature requests, refer to the feedback issue.