Annotate metrics and variables to track per run
Dotscience can track information about each run of a model. This information is made available, and explorable, in Dotscience Dashboard. In aggregate, this information provides insights into your model’s behaviour. For example, it can show the effect of hyperparameter values on performance scores, or the accuracy given different feature sets.
Dotscience can track the value of any variable in the model code: this includes, but is not limited to, the model parameters and hyperparameters. We recommend tracking the parts of a model that vary as you tune the model – whether you tune it by hand or automatically. In particular, you can track:
- Model inputs
- Model outputs, such as serialised models, and any intermediary files or plots written by the model
- Model parameters and hyperparameters
- Values of summary metrics, such as loss or accuracy scores
- Tags and comments about each run
To track information in your model, annotate the model source code using the Dotscience client library.