Discussions about bias often stall on a false choice: either the model is \u201cobjective because it is mathematics\u201d, or it is \u201cdiscriminatory because it was built wrong\u201d. Neither helps.
What it means concretely
A machine learning model extracts regularities from its training data. If a correlation exists in that data — even one reflecting a historical inequality rather than a causal relationship — the model will use it, because it improves prediction.
The classic example: a system trained on past hiring decisions will learn the patterns in those decisions, including the ones the organisation now wants to correct. The system does not \u201cdiscriminate\u201d in the sense of intent. It reproduces.
Three distinct sources
Training data reflects an unequal reality. If a group has been historically disadvantaged, the data shows it, and the model learns it.
Data collection is uneven. A group under-represented in the data gets weaker predictions — not because it is \u201charder\u201d, but because the model saw fewer examples.
The label itself is problematic. Many systems do not predict what we care about but a proxy for it. If the proxy is itself affected by an inequality, the model learns the inequality, not the phenomenon.
What can be done
Measure first. Performance across relevant subgroups is step one. You cannot correct what you have not measured, and \u201cwe do not have the data to measure it\u201d is itself a finding worth reporting.
Decide what \u201cfair\u201d means for your case. There are several mathematical definitions of fairness, and some are mutually incompatible: you cannot satisfy them all at once. The choice is a policy decision, not a technical one, and must be made explicitly and documented.
Check the proxies. Postcode, type of school, employment history can all act as stand-ins for protected characteristics. Removing the direct variable does not remove the effect.
Test with constructed cases. Pairs of inputs identical except for one characteristic quickly show whether the system treats equivalent situations differently.
What does not solve it
Deleting protected characteristics from the data does not solve it — the model reconstructs them from correlations.
Nor does a correction applied at the end, if the problem lies in the labels: you adjust the symptom, not the cause.
And it is certainly not solved by more data, if the additional data comes from the same process that produced the original inequality.
The practical conclusion
Bias is not a defect you remove once. It is a property you measure, document and monitor, with explicit decisions about which trade-off is acceptable and who takes responsibility for it. An organisation that can show that documentation is in a completely different position from one that can show only an accuracy figure.