Lecture 10: ML Testing & Explainability (Full Stack Deep Learning - Spring 2021)

In this lecture, you will expose to concepts and methods to help you, your teams, and your users: (1) understand at a deeper level how well your model is performing, (2) become more confident in your model’s ability to perform well in production, (3) understand the model’s performance envelope. 00:00 - What’s Wrong With Black-Box Predictions 06:47 - Types of Software Tests 08:05 - Software Testing Best Practices 21:02 - Sofware Testing In Production 26:42 - Continuous Integration and Continuous Delivery 29:25 - Testing Machine Learning Systems 36:39 - Infrastructure Tests 38:13 - Training Tests 41:24 - Functionality Tests 42:51 - Evaluation Tests 01:01:27 - Shadow Tests 01:03:58 - A/B Tests 01:05:40 - Labeling Tests 01:07:36 - Expectation Tests 01:11:43 - Challenges and Solutions Operationalizing ML Tests 01:17:29 - Overview of Explainable and Interpretable AI 01:20:00 - Use An Interpretable Family of Models 01:23:49 - Distill A Complex To An Interpretable One 01:27:52 - Understand The Contribution of Featu