Never Let the Truth Get in the Way of a Good Story: The Importance of Multilevel Human Understanding in Explainable Artificial Intelligence
| An interactive presentation given at the University of New South Wales (UNSW), discussing different levels at which explainability techniques and their insights need to be understood (using the example of surrogate explainers). | |
|---|---|
| Where: | School of Computer Science and Engineering, University of New South Wales, Australia. |
| When: | 4.00–5.00pm on Tuesday, November 15th, 2022. |
Table of Contents
About the Presentation
Myriad approaches exist to help humans peer inside automated decision-making systems based on artificial intelligence and machine learning algorithms. These tools and the insights they produce, however, tend to be complex socio-technological constructs themselves, hence subject to technical limitations as well as human biases and (possibly ill-defined) preferences. Under these conditions, how can we ensure that explanations are scientifically sound, technically correct and socially meaningful, therefore fulfil their role of leading to understanding?
In this talk I will provide a high-level introduction to and overview of (key concepts in) explainable AI and interpretable ML, followed by a deep dive into practical aspects of a popular transparency algorithm. Specifically, I will discuss the XAI and IML taxonomy captured by Explainability Fact Sheets; my attempt to define explainability in artificial intelligence and machine learning; and a preliminary framework intended to unify evaluation of explainability approaches on a conceptual level. Additionally, I will demonstrate how different configurations of an explainer – that is often presented as a monolithic, end-to-end tool – may adversely impact the resulting insights, using the example of a surrogate explainer. I will then show the importance of the strategy employed to present these pieces of information to a user, arguing in favour of a clear separation between the technical and social aspects of explainability techniques. Importantly, understanding these dependencies can help us to build bespoke explainers that are robust, reliable, trustworthy and suitable for the unique challenge at hand.
FAT Forensics (Software)
To support the goals of this talk, I employ
FAT Forensics – an open source Python package
that can inspect selected fairness, accountability and transparency aspects
of data (and their features), models and predictions.
The toolbox spans all of the FAT domains because many of them share underlying
algorithmic components that can be reused in multiple different
implementations, often across the FAT borders.
This interoperability allows, for example, a counterfactual data point
generator to be used as a post-hoc explainer of black-box predictions on
one hand, and as an individual fairness (disparate treatment) inspection tool
on the other.
The modular architecture12 enables
FAT Forensics to deliver robust and tested
low-level FAT building blocks as well as a collection of FAT tools built on top
of them.
Users can choose from these ready-made tools or, alternatively, combine the
available building blocks to create their own bespoke algorithms without the
need of modifying the codebase.
Resources
The presentation is provided as an interactive set of slides utilising iPyWidgets and built with a Jupyter Notebook based on RISE and reveal.js. This notebook (hence the presentation) can be executed locally on one’s own machine or launched directly in the web browser through Google Colab or MyBidner.
| Resources |
|---|
| Slides |
| Recording |
Instructors
Kacper Sokol
Kacper is a Research Fellow at the ARC Centre of Excellence for Automated Decision-Making and Society (ADM+S), affiliated with the School of Computing Technologies at RMIT University, Australia; and an Honorary Research Fellow at the Intelligent Systems Laboratory, University of Bristol, United Kingdom.
His main research focus is transparency – interpretability and explainability – of data-driven predictive systems based on artificial intelligence and machine learning algorithms. In particular, he has done work on enhancing transparency of predictive models with feasible and actionable counterfactual explanations and robust modular surrogate explainers. He has also introduced Explainability Fact Sheets – a comprehensive taxonomy of AI and ML explainers – and prototyped Glass-Box – a dialogue-driven interactive explainability system.
Kacper is the designer and lead developer of FAT Forensics – an open source fairness, accountability and transparency Python toolkit. Additionally, he is the main author of a collection of online interactive training materials about machine learning explainability, created in collaboration with the Alan Turing Institute – the UK’s national institute for data science and artificial intelligence.
Kacper holds a Master’s degree in Mathematics and Computer Science, and a doctorate in Computer Science from the University of Bristol, United Kingdom. Prior to joining ADM+S he has held numerous research positions at the University of Bristol, working with projects such as REFrAMe, SPHERE and TAILOR – European Union’s AI Research Excellence Centre. Additionally, he was a visiting researcher at the University of Tartu (Estonia); Simons Institute for the Theory of Computing, UC Berkeley (California, USA); and USI – Università della Svizzera italiana (Lugano, Switzerland). In his research, Kacper collaborated with numerous industry partners, such as THALES, and provided consulting on explainable artificial intelligence and transparent machine learning.
- Contact
- Kacper.Sokol@rmit.edu.au
References
-
Kacper Sokol, Alexander Hepburn, Rafael Poyiadzi, Matthew Clifford, Raul Santos-Rodriguez, and Peter Flach. 2020. FAT Forensics: A Python Toolbox for Implementing and Deploying Fairness, Accountability and Transparency Algorithms in Predictive Systems. Journal of Open Source Software, 5(49), p.1904. https://joss.theoj.org/papers/10.21105/joss.01904 ↩
-
Kacper Sokol, Raul Santos-Rodriguez, and Peter Flach. 2022. FAT Forensics: A Python Toolbox for Algorithmic Fairness, Accountability and Transparency. Software Impacts, 14, p.100406. https://doi.org/10.1016/j.simpa.2022.100406 ↩