DSC291a: Interpretable & Explainable ML

Instructor: Prof. Berk Ustun

This course introduced me to a new way of learning, emphasizing thorough reading of research papers and rigorous brainstorming and critique during class discussions. It was both intellectually stimulating and challenging.

We explored 17 papers spanning the domains of interpretable machine learning (IML) and explainable machine learning (XML), including case studies and user studies on the practical implementation of IML in organizations. The course began with foundational IML literature, covering key methods such as decision trees, decision sets, and Concept Bottleneck Models (CBMs). It then transitioned to XML techniques like LIME, SHAP, and saliency maps, examining their advantages, limitations, and applicability.

Key Takeaways:

• Model size as a proxy for interpretability can be misleading: For example, larger decision trees may sometimes be more comprehensible due to the inclusion of more informative attributes that make them intuitive for specific users and contexts.
• The Rashomon Effect and predictive multiplicity: These concepts highlight how multiple models can achieve similar performance while differing significantly in structure and feature sets, impacting model selection, validation, and post-hoc explanations.
• Purpose-driven interpretability: Interpretability in machine learning should align with specific use cases, such as debugging, actionable recourse, personalization, or feature selection.
• Contextual evaluation of explanation methods: The success of an explanation method depends on the use case and the chosen success metrics. Explanation methods should balance sensitivity to both model parameters and data while maintaining robustness and fidelity.
• The myth of the interpretability-performance tradeoff: Contrary to common belief, the tradeoff between interpretability and performance is often overstated.
• Post-hoc explanation limitations: Post-hoc methods rely heavily on approximations, making it crucial to evaluate their reliability critically.

Project Work: For the project, I presented new developments in inherently interpretable models, focusing on Concept Bottleneck Models (CBMs) and writing a critical peer review for the paper -- Critical Review.

Although this course was time-intensive, it was incredibly rewarding. I enjoyed exploring the breadth of IML and XML, and I’m confident the ideas and concepts I learned will prove invaluable when applying machine learning in practice.

Fall 2024