Reconciling Statistical and Causal Metrics of Fairness in Machine Learning on Data-Driven Systems

In the digital age, machine learning (ML) algorithms are becoming increasingly important in decision-making processes across a wide range of domains, including criminal justice, healthcare, and finance. While these algorithms provide significant benefits, they also pose the risk of perpetuating and exacerbating societal biases, especially when fairness is not taken into account during their design and implementation. We address the critical issue of fairness in machine learning, with a focus on combining statistical and causal fairness metrics to provide a more comprehensive approach to evaluate and ensure fairness by selecting the most suitable metric. To tackle this problem, we developed a research methodology aimed at systematically reviewing the existing literature while focusing on four research questions targeting the relationship between statistical and causal fairness metrics, which drove our analysis and categorization of papers. Based on the results of this review, we built a new fairness decision tree that integrates both types of metrics, which can guide users to choose the most suitable metric.