Two of the trickiest qualities to balance in the world of machine learning are fairness and accuracy. Algorithms optimized for accuracy may unintentionally perpetuate bias against specific groups, while those prioritizing fairness may compromise accuracy by misclassifying some data points.
With this challenge in mind, a team from CSAIL has taken the lead in devising a framework that enables a more nuanced approach to balancing these qualities.
Instead of forcing a binary decision in labeling all data points as “good” or “bad,” their framework uses their Reject Option Classification (ROC) algorithm which assigns a third category of “rejected samples,” allowing it to identify instances where the model might be less certain or where predictions could potentially lead to unfairness.
By rejecting these instances, the framework can reduce the likelihood of unfair outcomes for certain groups (i.e., by age or gender) without drastically sacrificing overall accuracy.
The “rejected” instances can also be further analyzed to understand potential sources of bias in the data or the model. This information, in turn, can be used to improve the model and the data collection process to mitigate future bias.
The development of the framework was led by MIT researcher and former professor Amar Gupta alongside research affiliate Rashmi Nagpal and MEng student Ariba Khan. Gupta says that existing systems often focus on optimizing only for “group fairness,” ensuring nondiscrimination across protected groups. In contrast, their framework integrates both group fairness and individual fairness, which involves treating comparable individual users similarly.
For example, say a machine learning model is being used to predict the likelihood of loan approval for individuals applying for a mortgage. “Group fairness” requires that the model predict loan approval at similar rates for both males and females, ensuring equitable treatment across gender. In contrast, “individual fairness” means the model would provide similar predictions for individuals with comparable qualifications, regardless of their age, gender, or ethnicity.
The team’s experimental analysis comparing their ROC-based framework to similar systems demonstrated its ability to achieve both high accuracy and fairness. On a dataset of German credit scores, it achieved an accuracy of more than 94%, meaning that the model should provide similar predictions for individuals with comparable qualifications and circumstances, regardless of sensitive features like age, gender, or ethnicity.”
Gupta says that most existing studies that have been done in this area have involved public datasets, but the team wanted to explore more private datasets to improve the applicability of de-biasing the algorithms that are used so pervasively across so many different industry sectors.
“These problems of equity and fairness are not limited to one organization or one industry, nor to just one isolated factor,” says Gupta. “A tool like ROC could really be used anywhere you need to make discerning judgments about data that you have, from finance to health care.”
The team presented their framework as part of a paper published in a special issue of Machine Learning and Knowledge Extraction. A second related paper (“Optimizing fairness and accuracy: a Pareto optimal approach for decisionmaking”) was also published earlier this year in the AI and Ethics journal.
In the second paper, the researchers worked closely with colleagues in Ernst and Young and other CSAIL affiliate companies to investigate a decision-making approach grounded in the economics concept of Pareto optimality. This approach seeks a state of resource allocation where improving one aspect of a set of solutions (i.e., accuracy) cannot occur without degrading another aspect (fairness).
The researchers specifically developed an extension of a framework called “Minimax Pareto Fairness” (MMPF), which uses a multi-objective loss function that, again, combines elements of both group and individual fairness to achieve Pareto optimality.
The team tested their framework on several open-source datasets, including the Adult Census Income, COMPAS and German Credit datasets, and showed a significant reduction in the accuracy-fairness trade-off across various sensitive features.
The new framework focuses on balancing fairness and performance using two fairness metrics, with plans to explore more in the future. The researchers aim to combine training methods with pre- or post-processing strategies to enhance outcomes. Nagpal says that next steps include fine-tuning fairness trade-offs using differential weights, and refining the Pareto-reweighing process to assign individual weights to data points for better optimization.
More information:
Rashmi Nagpal et al, A Multi-Objective Framework for Balancing Fairness and Accuracy in Debiasing Machine Learning Models, Machine Learning and Knowledge Extraction (2024). DOI: 10.3390/make6030105
Rashmi Nagpal et al, Optimizing fairness and accuracy: a Pareto optimal approach for decision-making, AI and Ethics (2024). DOI: 10.1007/s43681-024-00508-4
Provided by
Massachusetts Institute of Technology
Citation:
Creating AI that’s fair and accurate: Framework moves beyond binary decisions to offer a more nuanced approach (2024, November 8)
