Addressing Bias in AI - Toolkit for Fairness, Explainability and Privacy
by Ashok Balasubramanian Updated: Mar 13, 2023
Dilbert was dropped from hundreds of newspapers over Scott Adams’ racist comments. Multiple researchers have documented over the past few months how ChatGPT can be prompted to provide racist responses.
A three-decade globally famous comic strip has been canceled because of the creator’s racist comments in his YouTube show. ChatGPT, Bing Bot, and many such AI Bots are conversing with millions of users daily and have been documented to provide misleading, inaccurate, and biased responses. How can we hold AI to the same high standards we expect from society, especially when AI is now generative and scaled for global consumer use?
While no silver bullet exists, multiple aspects can make AI more responsible. Having open AI models is a great start. Hugging Face, EleutherAI, and many others are championing an open approach to AI. Openness and collaboration can bring in diverse contributions, reviews, and rigorous testing of AI models and help reduce bias.
NIST’s AI risk management guidelines released recently provide a comprehensive view across the AI lifecycle consisting of collecting and processing Data & Input, the build, and validation of the AI model, its deployment, and monitoring in the context of usage. Acknowledging the possibility of bias, eliminating data capture biases, or unconscious biases when generating synthetic data, designing for counterfactual fairness, and human-in-loop designs can reduce the risk of bias.
Use the below tools for assessment and to improve the fairness and robustness of your models.
A Python package to assess and improve fairness of machine learning models.
Python 1560 Version:v0.8.0 License: Permissive (MIT)
A comprehensive set of fairness metrics for datasets and machine learning models, explanations for these metrics, and algorithms to mitigate bias in datasets and models.
Python 2023 Version:v0.5.0 License: Permissive (Apache-2.0)
TextAttack 🐙 is a Python framework for adversarial attacks, data augmentation, and model training in NLP https://textattack.readthedocs.io/en/master/
Python 2366 Version:v0.3.8 License: Permissive (MIT)
Identify bias and measure fairness of your data
Python 72 Version:v0.1.0 License: Permissive (BSD-3-Clause)
The LinkedIn Fairness Toolkit (LiFT) is a Scala/Spark library that enables the measurement of fairness in large scale machine learning workflows.
Scala 139 Version:v0.3.0 License: Permissive (BSD-2-Clause)
Use the below tools for Explainability, Interpretability, and Monitoring.
Responsible AI Toolbox is a suite of tools providing model and data exploration and assessment user interfaces and libraries that enable a better understanding of AI systems. These interfaces and libraries empower developers and stakeholders of AI systems to develop and monitor AI more responsibly, and take better data-driven actions.
TypeScript 847 Version:v0.27.0 License: Permissive (MIT)
XAI - An eXplainability toolbox for machine learning
Python 720 Version:v0.1.0 License: Permissive (MIT)
Python Library for Model Interpretation/Explanations
Python 977 Version:v1.1.2 License: Permissive (UPL-1.0)
Google toolkit on Tensorflow for Privacy, Federated Learning, and Explainability.
A framework for implementing federated learning
Python 2119 Version:v0.59.0 License: Permissive (Apache-2.0)
Library for training machine learning models with privacy for training data
Python 1774 Version:v0.8.8 License: Permissive (Apache-2.0)
Source code/webpage/demos for the What-If Tool
HTML 806 Version:v1.8.1 License: Permissive (Apache-2.0)
Code for the TCAV ML interpretability project
Jupyter Notebook 570 Version:0.2 License: Permissive (Apache-2.0)