Building reliable machine learning (ML) models for use in production-level systems present specific risk factors not commonly addressed in the practitioners’ literature. As ML continues to play a central role in decision making processes, it is critical to evaluate models under several conditions to identify potential defects. In this session, we introduce model testing and validation methods to ensure the production-readiness of a model. In particular, we discuss concepts such as statistical model criticism, performance uncertainty, model staleness, calibration, algorithmic bias, and interpretability. The underlying methods to tackle these issues are essential for the long-term health of ML production systems, and should be seamlessly integrated in data science pipelines.