LLM Judges Carry Nine Measurable Biases: What to Do
Research presented at DHS 2026 by Bhaskarjit Sarmah delivered a blunt verdict on automated evaluation: "you can't trust LLM as a judge." That warning carries practical weight for any team running LLM-as-judge pipelines, a pattern now embedded in CI/CD for model fine-tuning, RAG quality checks, and production guardrails. The core finding reframes the risk: bias is most destructive not when one answer is clearly better than another, but precisely when two answers are comparably good — the normal production case, and the moment when the judge's verdict actually matters.
The root cause cuts across all nine documented bias types. Instead of reasoning from the evidence in the evaluation prompt, a judge defaults to priors absorbed during pretraining and reinforced by alignment training. Human raters rewarded polished, confident, thorough-looking answers, so the model learned "this looks good" as a proxy for "this is good" — and that proxy became your grader.
The Nine Bias Types
Position bias is among the most stark. Swap two answer labels — A becomes B, B becomes A — and weaker judges flip their verdict at rates barely above a coin flip. Scale to three or four candidates and most models pick a different winner more than half the time when order is reshuffled, despite zero change in content. The mechanism is attention weighting: a transformer attends more heavily to the beginning and end of a prompt, and the model carries priors about where strong answers conventionally appear.
Verbosity bias makes evaluation trivially gameable. When two answers are equally correct, the longer one scores higher, because length genuinely correlated with quality in training data. The correlation collapses the moment someone pads a response. More damaging: if judge scores feed back as a training signal, the fine-tuning loop actively teaches the downstream model to produce filler.
Self-preference bias produces the most operationally violated rule in evaluation pipelines. A model scores its own anonymised output higher than output of equal quality from other models — not by recognising a byline, but by recognising its own phrasing rhythm and structure as "fluent." Measured self-inflation reached as high as roughly 16% for some models; the strongest performers stayed near 1–2%. The rule: never use the same model to generate answers and grade them.
Fallacy-oversight bias carries the highest stakes for code review and student grading. A judge verifies that a final answer is correct but skips auditing the reasoning path. The documented test case: Answer A reasons that "3.8 has fewer digits after the decimal, so it must be larger" — nonsense logic that happens to land on the right answer. A susceptible judge ratifies the shared conclusion and misses that A's path is broken.
The remaining five biases share a social-heuristic structure:
| Bias | Trigger | Documented effect |
|---|---|---|
| Entity & tone bias | Named entity or emotional register in the answer text | Angry or anxious tone causes judges to abandon a correct answer roughly 70–75% of the time; adding emotion to a bad answer barely helps it |
| Identity bias | Gender, religion, health status, or immigration status attributed to the submitter | Weaker judges flipped verdicts on roughly a third of items purely on identity information; robust judges barely moved |
| Authority bias | Citation appended to an answer — even a fabricated URL or invented book | Judges reversed correct verdicts after a fake reference was added and cited the fabricated source as their explicit reason |
| Bandwagon bias | Statement that a majority preferred one answer | Claimed consensus at 60% moved the judge approximately as much as 90%; magnitude of consensus was nearly irrelevant |
| Distraction bias | Irrelevant personal detail about one assistant | Noise damages evaluation of good answers more than bad ones; bad answers are already near floor and have little room to fall |
The tone bias case produced one of the more unsettling documented examples: a judge explicitly stated in its own chain-of-thought that the angry-toned answer was more direct and technically precise, then selected the other answer because it worried the confrontational tone would put users off. The model told you which answer was better and chose differently.
Grounding as a Structural Fix
Every bias above operates because the judge answers an open question — "is this good?" — using priors rather than evidence. The structural fix is to replace that open question with a closed one: "is this claim supported by this document?" That is textual entailment, a narrower problem that leaves bias fewer surfaces to operate on. Claims are checked independently (no positional signal), extracted rather than quoted (no tone leakage), and the checker is separated from the generator (no self-preference). Engineers remove the bias surface rather than prompting around it — a lesson that applies equally when you're trying to automate LLM prompt optimization in production.
Two purpose-built grounded evaluators are production-ready. Lynx (Patronus AI) is a Llama-3-Instruct fine-tune available at 8B and 70B parameters, trained on hard cases from PubMedQA, DROP, and FinanceBench. The 70B variant outperformed GPT-4 across a range of hallucination scenarios and exposes its reasoning. Bespoke-MiniCheck targets cost-sensitive deployments: the MiniCheck-FT5 variant at 770M parameters reaches GPT-4-level accuracy at approximately 400× lower cost, and the 7B version leads LLM-AggreFact at 77.4% accuracy with roughly 200ms latency — fast enough for a synchronous guardrail. Also worth examining: Prometheus 2, GLIDER, and Vectara HHEM.
Pipeline Design Implications
The DHS 2026 findings establish that LLM-judge bias is measurable and exploitable. Audit what sits inside the judge's context window: upvote counts, prior model rankings, reviewer tallies, and "preferred by X" metadata all introduce bandwagon surface that does not belong there. When grounding is architecturally impossible, the minimum mitigation is a diverse judge ensemble and systematic stress-testing — swapping positions, stripping citations, normalising length — before trusting any score. Given how deeply the agentic era is reshaping data science workflows, evaluation integrity upstream determines whether the systems built on top are trustworthy. Using a compromised judge as a fine-tuning signal does not measure quality; it amplifies whatever the judge happened to reward.