Machine Learning Model Testing

When:
2018-11-19 @ 17:30 – 18:30
2018-11-19T17:30:00-05:00
2018-11-19T18:30:00-05:00
Where:
Royal Bank Waterpark Place - Auditorium
88 Queens Quay W
Toronto
ON M5J 0B8
Cost:
Free
Contact:
Alison Burnham
818-284-7856
Machine learning
Model testing and validation for production-level systems
Leo Guelman
Chief Statistician and Director, Data Science
Royal Bank of Canada

Building reliable machine learning (ML) models for use in production-level systems present specific risk factors not commonly addressed in the practitioners’ literature. As ML continues to play a central role in decision making processes, it is critical to evaluate models under several conditions to identify potential defects. In this session, we introduce model testing and validation methods to ensure the production-readiness of a model. In particular, we discuss concepts such as statistical model criticism, performance uncertainty, model staleness, calibration, algorithmic bias, and interpretability. The underlying methods to tackle these issues are essential for the long-term health of ML production systems, and should be seamlessly integrated in data science pipelines.