Researchers have introduced Fairness Pruning, a novel structural intervention designed to locate and manage demographic bias in large language models (LLMs). The method, detailed in a recent paper, focuses on causal bias localization using minimally contrastive prompt pairs and inference-time activation capture, identifying neurons that react differentially to demographic attributes within GLU architectures.
The approach evaluates signals at the down_proj input and has been tested on models up to 3 billion parameters, including the Llama-3.2 family and Salamandra-2B. By zeroing out identified neurons, the researchers observed that the model's responses to demographic variables change, but the effect is not a simple flat mitigation. Instead, the intervention causes bidirectional bias destabilization, as the unsigned BiasScore mixes neurons that push toward and against stereotypes, with the net effect depending on which sign dominates.
Remarkably, the intervention is extremely surgical. In Llama-3.2-1B, zeroing at most 40 neurons—less than 0.031% of the total MLP width—achieves a mean retention of 99.49% in reasoning and general knowledge capabilities. Even changing just 5 neurons is enough to significantly alter biased responses in models ranging from 1B to 3B parameters.
This work is the first in a broader research line. The next step is moving from blind zeroing to directional steering, incorporating a signed BiasScore that separates amplifier neurons from regulator neurons. All code, datasets, and results are openly available on GitHub and Hugging Face.