Responsible AI

Interpretability, Explainability, Robustness and Fairness in AI

My research in responsible AI focuses on developing systems that prioritize fairness (Ahmad et al., 2021), explainability (Ahmad et al., 2018), and trustworthiness (Ahmad et al., 2023), particularly in healthcare applications. This involves also ensuring compliance with regulatory standards (Ahmad et al., 2021) and ethical guidelines. In a domain where decisions can significantly impact patient lives, ensuring that AI systems operate without bias and deliver equitable outcomes across dispirate populations is critical. I investigate techniques that mitigate algorithmic bias (Ahmad et al., 2020), foster fairness (Ahmad et al., 2020), ensuring that healthcare AI tools do not disproportionately harm or disadvantage certain groups (Yuan et al., 2021). Alongside fairness, I work on explainability of AI models (Kovalerchuk et al., 2021) (Ahmad et al., 2019), this is to ensures that machine learning models are transparent and understandable for clinicians and other stakeholders. Recent work has focused on creating trustworthy LLMs by reducing Hallucinations (Ahmad et al., 2023) and avoiding catastrophic forgetting. To summarize, by embedding ethical principles throughout the AI lifecycle, my work seeks to foster trust and ensure that healthcare AI systems serve the best interests of all patients.

References

2023

arXiV
Creating trustworthy llms: Dealing with hallucinations in healthcare ai

Muhammad Aurangzeb Ahmad, Ilker Yaramis, and Taposh Dutta Roy

arXiv preprint arXiv:2311.01463, 2023

Abs Bib PDF

Large language models have proliferated across multiple domains in as short period of time. There is however hesitation in the medical and healthcare domain towards their adoption because of issues like factuality, coherence, and hallucinations. Give the high stakes nature of healthcare, many researchers have even cautioned against its usage until these issues are resolved. The key to the implementation and deployment of LLMs in healthcare is to make these models trustworthy, transparent (as much possible) and explainable. In this paper we describe the key elements in creating reliable, trustworthy, and unbiased models as a necessary condition for their adoption in healthcare. Specifically we focus on the quantification, validation, and mitigation of hallucinations in the context in healthcare. Lastly, we discuss how the future of LLMs in healthcare may look like.
@article{ahmad2023creating, title = {Creating trustworthy llms: Dealing with hallucinations in healthcare ai}, author = {Ahmad, Muhammad Aurangzeb and Yaramis, Ilker and Roy, Taposh Dutta}, journal = {arXiv preprint arXiv:2311.01463}, year = {2023}, url = {https://arxiv.org/abs/2311.01463}, }

2021

ICHI 2021
Fairness in healthcare AI

Muhammad Aurangzeb Ahmad, Carly Eckert, Christine Allen, and 3 more authors

In 2021 IEEE 9th International Conference on Healthcare Informatics (ICHI), 2021

Abs Bib

The issue of bias and fairness in healthcare has been around for centuries. With the integration of AI in healthcare the potential to discriminate and perpetuate unfair and biased practices in healthcare increases many folds. The tutorial focuses on the challenges, requirements and opportunities in the area of fairness in healthcare AI and the various nuances associated with it. The problem healthcare as a multi-faceted systems level problem that necessitates careful consideration of different notions of fairness in healthcare to corresponding concepts in machine learning is elucidated via different real world examples.
@inproceedings{ahmad2021fairness, title = {Fairness in healthcare AI}, author = {Ahmad, Muhammad Aurangzeb and Eckert, Carly and Allen, Christine and Kumar, Vikas and Hu, Juhua and Teredesai, Ankur}, booktitle = {2021 IEEE 9th International Conference on Healthcare Informatics (ICHI)}, pages = {554--555}, year = {2021}, organization = {IEEE}, url = {https://ieeexplore.ieee.org/abstract/document/9565788}, }
KDD
Software as a medical device: regulating AI in healthcare via responsible AI

Muhammad Aurangzeb Ahmad, Steve Overman, Christine Allen, and 3 more authors

In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, 2021

Abs Bib

With the increased adoption of AI in healthcare, there is a growing recognition and demand to regulate AI in healthcare to avoid potential harm and unfair bias against vulnerable populations. Around a hundred governmental bodies and commissions as well as leaders in the tech sector have proposed principles to create responsible AI systems. However, most of these proposals are short on specifics which has led to charges of ethics washing. In this tutorial we offer a guide to help navigate through complex governmental regulations and explain the various constituent practical elements of a responsible AI system in healthcare in the light of proposed regulations. Additionally, we breakdown and emphasize that the recommendations from regulatory bodies like FDA or the EU are necessary but not sufficient elements of creating a responsible AI system. We elucidate how regulations and guidelines often focus on epistemic concerns to the detriment of practical concerns e.g., requirement for fairness without explicating what fairness constitutes for a use case. FDA’s Software as a medical device document and EU’s GDPR among other AI governance documents talk about the need for implementing sufficiently good machine learning practices. In this tutorial we elucidate what that would mean from a practical perspective for real world use cases in healthcare throughout the machine learning cycle i.e., Data Management, Data Specification, Feature Engineering, Model Evaluation, Model Specification, Model Explainability, Model Fairness, Reproducibility, checks for data leakage and model leakage. We note that conceptualizing responsible AI as a process rather than an end goal accords well with how AI systems are used in practice. We also discuss how a domain centric stakeholder perspective translates into balancing requirements for multiple competing optimization criteria.
@inproceedings{ahmad2021software, title = {Software as a medical device: regulating AI in healthcare via responsible AI}, author = {Ahmad, Muhammad Aurangzeb and Overman, Steve and Allen, Christine and Kumar, Vikas and Teredesai, Ankur and Eckert, Carly}, booktitle = {Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery \& Data Mining}, pages = {4023--4024}, year = {2021}, url = {https://dl.acm.org/doi/abs/10.1145/3447548.3470823}, }
ArXiV
Assessing fairness in classification parity of machine learning models in healthcare

Ming Yuan, Vikas Kumar, Muhammad Aurangzeb Ahmad, and 1 more author

arXiv preprint arXiv:2102.03717, 2021

Abs Bib PDF

Fairness in AI and machine learning systems has become a fundamental problem in the accountability of AI systems. While the need for accountability of AI models is near ubiquitous, healthcare in particular is a challenging field where accountability of such systems takes upon additional importance, as decisions in healthcare can have life altering consequences. In this paper we present preliminary results on fairness in the context of classification parity in healthcare. We also present some exploratory methods to improve fairness and choosing appropriate classification algorithms in the context of healthcare.
@article{yuan2021assessing, title = {Assessing fairness in classification parity of machine learning models in healthcare}, author = {Yuan, Ming and Kumar, Vikas and Ahmad, Muhammad Aurangzeb and Teredesai, Ankur}, journal = {arXiv preprint arXiv:2102.03717}, year = {2021}, url = {https://arxiv.org/pdf/2102.03717}, }
Book Chapter
Survey of explainable machine learning with visual and granular methods beyond quasi-explanations

Boris Kovalerchuk, Muhammad Aurangzeb Ahmad, and Ankur Teredesai

Interpretable artificial intelligence: A perspective of granular computing, 2021

Abs Bib PDF

This chapter surveys and analyses visual methods of explainability of Machine Learning (ML) approaches with focus on moving from quasi-explanations that dominate in ML to actual domain-specific explanation supported by granular visuals. The importance of visual and granular methods to increase the interpretability and validity of the ML model has grown in recent years. Visuals have an appeal to human perception, which other methods do not. ML interpretation is fundamentally a human activity, not a machine activity. Thus, visual methods are more readily interpretable. Visual granularity is a natural way for efficient ML explanation. Understanding complex causal reasoning can be beyond human abilities without “downgrading” it to human perceptual and cognitive limits. The visual exploration of multidimensional data at different levels of granularity for knowledge discovery is a long-standing research focus. While multiple efficient methods for visual representation of high-dimensional data exist, the loss of interpretable information, occlusion, and clutter continue to be a challenge, which lead to quasi-explanations. This chapter starts with the motivation and the definitions of different forms of explainability and how these concepts and information granularity can integrate in ML. The chapter focuses on a clear distinction between quasi-explanations and actual domain specific explanations, as well as between potentially explainable and an actually explained ML model that are critically important for the further progress of the ML explainability domain. We discuss foundations of interpretability, overview visual interpretability and present several types of methods to visualize the ML models. Next, we present methods of visual discovery of ML models, with the focus on interpretable models, based on the recently introduced concept of General Line Coordinates (GLC). This family of methods take the critical step of creating visual explanations that are not merely quasi-explanations but are also domain specific visual explanations while these methods themselves are domain-agnostic. The chapter includes results on theoretical limits to preserve n-D distances in lower dimensions, based on the Johnson-Lindenstrauss lemma, point-to-point and point-to-graph GLC approaches, and real-world case studies. The chapter also covers traditional visual methods for understanding multiple ML models, which include deep learning and time series models. We illustrate that many of these methods are quasi-explanations and need further enhancement to become actual domain specific explanations. The chapter concludes with outlining open problems and current research frontiers.
@article{kovalerchuk2021survey, title = {Survey of explainable machine learning with visual and granular methods beyond quasi-explanations}, author = {Kovalerchuk, Boris and Ahmad, Muhammad Aurangzeb and Teredesai, Ankur}, journal = {Interpretable artificial intelligence: A perspective of granular computing}, pages = {217--267}, year = {2021}, publisher = {Springer}, url = {https://dl.acm.org/doi/abs/10.1145/3447548.3470823}, }

2020

FAccT
Fairness, accountability, transparency in AI at scale: Lessons from national programs

Muhammad Aurangzeb Ahmad, Ankur Teredesai, and Carly Eckert

In Proceedings of the 2020 conference on fairness, accountability, and transparency, 2020

Abs Bib

The panel aims to elucidate how different national govenmental programs are implementing accountability of machine learning systems in healthcare and how accountability is operationlized in different cultural settings in legislation, policy and deployment. We have representatives from three different govenments, UAE, Singapore and Maldives who will discuss what accountability of AI and machine learning means in their contexts and use cases. We hope to have a fruitful conversation around FAT ML as it is operationalized ccross cultures, national boundries and legislative constraints.
@inproceedings{ahmad2020fairness, title = {Fairness, accountability, transparency in AI at scale: Lessons from national programs}, author = {Ahmad, Muhammad Aurangzeb and Teredesai, Ankur and Eckert, Carly}, booktitle = {Proceedings of the 2020 conference on fairness, accountability, and transparency}, pages = {690--690}, year = {2020}, url = {https://dl.acm.org/doi/10.1145/3351095.3375690}, }
KDD
Fairness in machine learning for healthcare

Muhammad Aurangzeb Ahmad, Arpit Patel, Carly Eckert, and 2 more authors

In Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining, 2020

Abs Bib

The issue of bias and fairness in healthcare has been around for centuries. With the integration of AI in healthcare the potential to discriminate and perpetuate unfair and biased practices in healthcare increases many folds The tutorial focuses on the challenges, requirements and opportunities in the area of fairness in healthcare AI and the various nuances associated with it. The problem healthcare as a multi-faceted systems level problem that necessitates careful of different notions of fairness in healthcare to corresponding concepts in machine learning is elucidated via different real world examples.
@inproceedings{ahmad2020fairnessKDD, title = {Fairness in machine learning for healthcare}, author = {Ahmad, Muhammad Aurangzeb and Patel, Arpit and Eckert, Carly and Kumar, Vikas and Teredesai, Ankur}, booktitle = {Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery \& data mining}, pages = {3529--3530}, year = {2020}, url = {http://apsychoserver.psych.arizona.edu/JJBAReprints/PSYC621/Ahmad%20et%20al%202020.pdf}, }

2019

IJCAI
The challenge of imputation in explainable artificial intelligence models

Muhammad Aurangzeb Ahmad, Carly Eckert, and Ankur Teredesai

In AISafety Workshop at IJCAI, 2019

Abs Bib PDF

Explainable models in Artificial Intelligence are often employed to ensure transparency and accountability of AI systems. The fidelity of the explanations are dependent upon the algorithms used as well as on the fidelity of the data. Many real world datasets have missing values that can greatly influence explanation fidelity. The standard way to deal with such scenarios is imputation. This can, however, lead to situations where the imputed values may correspond to a setting which refer to counterfactuals. Acting on explanations from AI models with imputed values may lead to unsafe outcomes. In this paper, we explore different settings where AI models with imputation can be problematic and describe ways to address such scenarios.
@inproceedings{ahmad2019challenge, title = {The challenge of imputation in explainable artificial intelligence models}, author = {Ahmad, Muhammad Aurangzeb and Eckert, Carly and Teredesai, Ankur}, booktitle = {AISafety Workshop at IJCAI}, year = {2019}, url = {https://arxiv.org/pdf/1907.12669}, }

2018

ACM-BCB
Interpretable machine learning in healthcare

Muhammad Aurangzeb Ahmad, Carly Eckert, and Ankur Teredesai

Proceedings of the 2018 ACM international conference on bioinformatics, computational biology, and health informatics, 2018

Abs Bib PDF

This tutorial extensively covers the definitions, nuances, challenges, and requirements for the design of interpretable and explainable machine learning models and systems in healthcare. We discuss many uses in which interpretable machine learning models are needed in healthcare and how they should be deployed. Additionally, we explore the landscape of recent advances to address the challenges model interpretability in healthcare and also describe how one would go about choosing the right interpretable machine learnig algorithm for a given problem in healthcare.
@article{ahmad2018interpretable, title = {Interpretable machine learning in healthcare}, author = {Ahmad, Muhammad Aurangzeb and Eckert, Carly and Teredesai, Ankur}, year = {2018}, journal = {Proceedings of the 2018 ACM international conference on bioinformatics, computational biology, and health informatics}, pages = {559--560}, url = {https://dl.acm.org/doi/abs/10.1145/3233547.3233667}, }