Bias in an AI system is not a single problem with a single fix. It enters at different points — in the data, in the model's design, and in how outputs are used — and each entry point calls for a different category of mitigation. The practical answer for any team is to first diagnose where the bias originates, then match the correction to the stage of the AI lifecycle you can actually control: pre-processing the data, in-processing during model training, or post-processing the outputs. This article walks through that diagnosis and delivers a selection framework you can apply to your own project.
Understanding AI Bias: Origins and Impact
What is algorithmic bias? IBM's Alexandra Jonker and Julie Rogers define it as systematic errors in machine learning algorithms that produce unfair or discriminatory outcomes, often reflecting or reinforcing existing socioeconomic, racial, and gender biases. A California state technology guidance document, drawing on Info-Tech Research Group, describes it similarly as "systematic and repeatable errors in a computer system that create unfair outcomes, such as privileging one arbitrary group of users over others."
Where does it come from? The first source is the data itself. Historical societal biases baked into records, unrepresentative training data, and measurement errors all shape what a model learns before training even begins. These biases are present before model training and directly impact algorithmic decisions.
The second source is the model and the people building it. As SAP's guidance notes, even if the data is unbiased, the way algorithms process and prioritize certain features over others can result in discriminatory outcomes. Human cognitive bias can also seep in through subjective decisions in data labeling and model development. Algorithmic and human biases can lead to systematically prejudiced results even with seemingly unbiased data.
The consequences are concrete. The California guidance catalogues real cases: a gender-discrimination investigation into the Apple Card, racial bias found in a major health care risk algorithm, and a UK government abandoning algorithmically generated exam results after student protests. A research article on discriminatory algorithmic decision-making in hiring describes the harm plainly: a candidate matching all requisites not selected for an interview "because of their being the wrong gender," with no explanation available.
Identifying and Measuring Bias: A Practical Checklist
How do you find bias before your users do? Testing guidance from testRigor, authored by Anushree Chatterjee, recommends a structured checklist that surfaces hidden issues early and reduces the risk of unfair outcomes in production. The core checks are:
- Verify that the training dataset includes diverse and balanced user groups, and check whether any demographic group is underrepresented.
- Test the AI system using different real-world user scenarios and perform edge-case testing to uncover hidden bias patterns.
- Compare model accuracy and fairness across multiple subgroups.
- Identify proxy variables that may indirectly represent sensitive attributes.
- Use explainability tools to understand how the AI makes decisions.
- Include human review to validate fairness and ethical behavior.
Two of these deserve emphasis. Proxy variables — features that stand in for protected attributes like race or gender — can quietly reintroduce discrimination even when sensitive fields are removed. And human review matters because, as Chatterjee notes, regular human review helps detect hidden issues that automated fairness checks or algorithms may fail to recognize.
Before any of this, define what fairness means in your specific context and use appropriate metrics to assess it. That definition, as the testing guidance stresses, spans all stages and is what lets you evaluate whether mitigation actually worked.
AI Bias Mitigation Decision Framework
Once you know bias exists, the question becomes which category of technique fits your situation. The framework below classifies mitigation techniques by their point of application in the AI lifecycle, creating a selection matrix that maps your project stage to a solution category. Use it by locating the stage you are in — or the stage you can influence — and reading across to the appropriate approach.
| AI Lifecycle Stage | Source of Bias Addressed | Mitigation Approach | Key Actions and Techniques |
|---|---|---|---|
| Data collection and preparation (pre-processing) | Historical societal bias in data; unrepresentative data; measurement errors | Fix the data before training | Re-sampling, re-weighting, and data augmentation; audit datasets for over- and under-representation; check labeling practices; document dataset provenance and intended use |
| Model design and training (in-processing) | Algorithm design, feature selection, and human cognitive bias in development | Embed fairness into the learning algorithm | Fairness constraints and regularizers; adversarial debiasing; fairness-by-design with impact assessments during the design phase |
| Model output (post-processing) | Unfair predictions from an already-trained model | Adjust outputs without altering the model | Re-calibration and threshold adjustment to achieve fairness after training |
| Testing and auditing (all stages) | Bias that is hidden, emergent, or proxy-driven | Detect and quantify | Fairness metrics, subgroup analysis, explainability tools, human-in-the-loop review |
| Post-deployment | Bias that drifts or evolves over time | Monitor and retrain | Continuous monitoring of performance and fairness; periodic reviews and retraining |
The reasoning behind this matrix is straightforward: the stage where bias enters, or where you have leverage, determines the category of fix. If you control the data, pre-processing is available. If you are still training, in-processing can guide the model itself. If the model is already deployed and cannot be altered, post-processing is your only lever — the testing guidance notes this category exists precisely for "achieving fairness after the model has been trained, without altering the model itself."
Pre-processing Techniques: Addressing Data Bias
Pre-processing techniques are applied before model training and address biases in the training data through methods like re-sampling, re-weighting, and data augmentation. These methods aim to create more balanced and representative datasets for model training.
The Berkeley Haas responsible-AI playbook offers the most detailed playbook for this stage. It recommends adopting a "data with depth" approach that integrates social science methods into technical data science methods for generating and collecting data, placing checks on labeling practices, and documenting the provenance, creation, and use of machine learning datasets so that a dataset's purpose aligns with its intended use.
It also advises assessing existing datasets to check for over- and under-representation of certain identities, and for underlying inequities that reflect reality but are ultimately problematic — a crucial distinction, because some data accurately records an unfair world. Datasets should be maintained as living resources rather than frozen artifacts.
In-processing Techniques: Fairer Model Training
In-processing techniques are applied during model training and incorporate fairness constraints or regularizers into the learning algorithm, with adversarial debiasing as a named example. The goal is to guide the model to learn fair representations and decision boundaries during its development.
This category also includes what SAP calls fairness-by-design: a proactive approach that integrates fairness considerations into the AI development lifecycle from the beginning rather than as an afterthought, including developing fair algorithms and conducting impact assessments during the design phase.
The trade-off is real. In-processing requires access to the training pipeline and the expertise to modify it, which is why it suits teams still building their systems rather than those consuming or deploying finished models.
Post-processing Techniques: Adjusting Model Outputs
Post-processing techniques are applied after model training to adjust model predictions or outputs to achieve fairness, including re-calibration and threshold adjustment. These methods modify the model's final decisions to ensure equitable outcomes across groups.
Their advantage is accessibility: they work when you cannot retrain or alter the model, such as when using a third-party or already-deployed system. Their limitation follows from the same fact — they correct outputs without addressing the underlying data or learning process, so the root cause remains.
Continuous Monitoring and Governance
Fixing bias once is not fixing it. A review of AI pitfalls in health care, published in PMC, concludes that "bias occurs across the AI life cycle, and hence must be tackled at each step from problem definition all the way through post-deployment monitoring." Bias can drift over time, requiring ongoing vigilance and adaptation of mitigation strategies.
The Berkeley Haas playbook adds a sober note about scope: most efforts focus on addressing "technical" biases through mathematical fixes and fail to address systematic patterns of exclusion and power inequities in the field of AI. Its survey data shows what organizations actually do — 93% of respondents flag when data and outcomes differ from training data, 60% work on measuring AI decision-making factors to catch disadvantaging proxies, and 59% use algorithms to mitigate hidden biases in training data.
Human oversight remains essential throughout. Models may still produce biased, incorrect, or misleading results, especially with complex real-world scenarios or incomplete data, and human reviewers catch what automated checks miss. The health care review also stresses the need for diverse teams working on the problem, noting early findings of worsening performance for experts presented with biased model outputs.
Sustaining Fair AI: A Commitment to Continuous Oversight
The decision this evidence supports is to implement a multi-stage bias mitigation strategy, integrating detection and correction throughout the AI lifecycle, and to commit to continuous monitoring post-deployment. Concretely: audit and balance your data before training, embed fairness constraints if you control the pipeline, adjust outputs if you do not, and keep humans in the loop at every stage.
The measurable indicator of success is regular audit reports showing consistent performance across demographic subgroups and a stable or decreasing bias metric over time. Track model performance and fairness in deployment, conduct periodic reviews, and retrain when subgroup results diverge. If your bias metric rises or subgroup performance gaps widen between audits, that is your signal to return to the framework and re-diagnose.
