We demonstrate how you can use Dotscience to provision a dashboard to monitor your models in production.
When a model is deployed into production with Dotscience, a dashboard gets automatically created for that model. These dashboards are powered by a Grafana instance that Dotscience provisions for you. We look into more detail around the monitoring dashboard in this tutorial.
For an end-to-end tutorial on deploying and monitoring models in production see, Notebook based development with Dotscience and Using Dotscience in remote mode with Python scripts
Monitor the behaviour of the model in production
Here, we assume a model has already been built and deployed to production with Dotscience.
Navigate to the deployments tab, find the model you just deployed and open the monitoring dashboard for it by clicking ‘Monitor’. The monitoring dashboard for your model will track requests to your model and its behaviour based on real world data.
This is a prototype to demonstrate monitoring. For enterprise and other use cases, please contact us so we can enable monitoring at a user/project level. The credentials for hosted Grafana dashboard are:
Username: playground Password: password
Initially the monitoring dashboard will be empty, as there are no requests being sent to the model.
For the convenience of this tutorial, we have a demo app to send requests to the model at https://deploy-demo.dotscience.com/. You will need the model deployment URL, found under heading ‘Host’ for your model at https://cloud.dotscience.com/models/deployments. Copy the URL by clicking ‘Copy to clipboard’ (note: that the entire URL is not displayed).
Navigate to https://deploy-demo.dotscience.com/ and select the demo app (in this case, it’s Road Sign Predictor) and paste the model deployment URL into the app and send requests to it by clicking the roadsigns. Observe requests being sent to the model and its behaviour on the monitoring dashboard above.