Addressing Bias in AI - Toolkit for Fairness, Explainability and Privacy
by Ashok Balasubramanian Updated: Mar 13, 2023
Guide Kit
Dilbert was dropped from hundreds of newspapers over Scott Adams’ racist comments. Multiple researchers have documented over the past few months how ChatGPT can be prompted to provide racist responses.
A three-decade globally famous comic strip has been canceled because of the creator’s racist comments in his YouTube show. ChatGPT, Bing Bot, and many such AI Bots are conversing with millions of users daily and have been documented to provide misleading, inaccurate, and biased responses. How can we hold AI to the same high standards we expect from society, especially when AI is now generative and scaled for global consumer use?
While no silver bullet exists, multiple aspects can make AI more responsible. Having open AI models is a great start. Hugging Face, EleutherAI, and many others are championing an open approach to AI. Openness and collaboration can bring in diverse contributions, reviews, and rigorous testing of AI models and help reduce bias.
NIST’s AI risk management guidelines released recently provide a comprehensive view across the AI lifecycle consisting of collecting and processing Data & Input, the build, and validation of the AI model, its deployment, and monitoring in the context of usage. Acknowledging the possibility of bias, eliminating data capture biases, or unconscious biases when generating synthetic data, designing for counterfactual fairness, and human-in-loop designs can reduce the risk of bias.
Use the below tools for assessment and to improve the fairness and robustness of your models.
fairlearnby fairlearn
A Python package to assess and improve fairness of machine learning models.
fairlearnby fairlearn
Python 1591 Version:v0.8.0 License: Permissive (MIT)
AIF360by Trusted-AI
A comprehensive set of fairness metrics for datasets and machine learning models, explanations for these metrics, and algorithms to mitigate bias in datasets and models.
AIF360by Trusted-AI
Python 2048 Version:v0.5.0 License: Permissive (Apache-2.0)
TextAttackby QData
TextAttack 🐙 is a Python framework for adversarial attacks, data augmentation, and model training in NLP https://textattack.readthedocs.io/en/master/
TextAttackby QData
Python 2377 Version:v0.3.8 License: Permissive (MIT)
fairlensby synthesized-io
Identify bias and measure fairness of your data
fairlensby synthesized-io
Python 72 Version:v0.1.0 License: Permissive (BSD-3-Clause)
LiFTby linkedin
The LinkedIn Fairness Toolkit (LiFT) is a Scala/Spark library that enables the measurement of fairness in large scale machine learning workflows.
LiFTby linkedin
Scala 139 Version:v0.3.0 License: Permissive (BSD-2-Clause)
Use the below tools for Explainability, Interpretability, and Monitoring.
responsible-ai-toolboxby microsoft
Responsible AI Toolbox is a suite of tools providing model and data exploration and assessment user interfaces and libraries that enable a better understanding of AI systems. These interfaces and libraries empower developers and stakeholders of AI systems to develop and monitor AI more responsibly, and take better data-driven actions.
responsible-ai-toolboxby microsoft
TypeScript 862 Version:v0.28.0 License: Permissive (MIT)
xaiby EthicalML
XAI - An eXplainability toolbox for machine learning
xaiby EthicalML
Python 720 Version:v0.1.0 License: Permissive (MIT)
Skaterby oracle
Python Library for Model Interpretation/Explanations
Skaterby oracle
Python 977 Version:v1.1.2 License: Permissive (UPL-1.0)
Google toolkit on Tensorflow for Privacy, Federated Learning, and Explainability.
federatedby tensorflow
A framework for implementing federated learning
federatedby tensorflow
Python 2119 Version:v0.59.0 License: Permissive (Apache-2.0)
privacyby tensorflow
Library for training machine learning models with privacy for training data
privacyby tensorflow
Python 1779 Version:v0.8.8 License: Permissive (Apache-2.0)
what-if-toolby PAIR-code
Source code/webpage/demos for the What-If Tool
what-if-toolby PAIR-code
HTML 806 Version:v1.8.1 License: Permissive (Apache-2.0)
tcavby tensorflow
Code for the TCAV ML interpretability project
tcavby tensorflow
Jupyter Notebook 570 Version:0.2 License: Permissive (Apache-2.0)