Nine Measurable Biases That Corrupt LLM Judge Verdicts

At a DHS 2026 workshop, researcher Bhaskarjit Sarmah made a blunt declaration: "you can't trust LLM as a judge — I avoid using them." The observation carries real weight for teams running automated evaluation pipelines, because LLM-as-judge is now embedded in everything from RLHF reward modelling to RAG quality gates. The workshop catalogued nine distinct, measurable biases that corrupt judge verdicts. The core finding that reframes all of them: bias matters most precisely when two answers are comparably good — which is the normal production case.

Nine Bias Failure Modes

The root mechanism is the same across all nine: the judge answers from absorbed priors rather than from the content in front of it. Alignment training rewarded whatever human raters preferred — polished, confident, thorough-looking answers — so the model learned that "looks good" is an acceptable proxy for "is good."

Position bias is detectable with a swap test: present two answers labelled A and B, then re-run with the labels reversed. On weaker judges, two-way swaps produce verdicts barely above chance. With three or four candidates, most models flip the winner more than half the time when order is reshuffled.

Verbosity bias means the longer of two equally correct answers scores higher even when the extra words add nothing. The training correlation — longer answers genuinely were more thorough — breaks the instant someone pads a response. If judge scores feed back as a training signal, verbosity bias actively rewards waffle.

Self-preference bias is the most structurally dangerous. Models inflate scores for their own anonymised outputs. Measured self-inflation reached as high as roughly 16% for some models; the best performers stayed near 1–2%. The mechanism is stylistic fingerprinting: the judge reads its own phrasing and rhythm as more fluent, then promotes fluency to quality. The direct rule this creates — never use the same model to generate and grade — is simultaneously the most violated rule in production pipelines.

Entity and tone bias operate through sentiment associations learned from training data. The same factual claim scores differently depending on which country or company is named. On emotional tone the effect is asymmetric and severe: rewriting a correct answer to sound angry or anxious causes judges to reject it roughly 70–75% of the time. In one documented case, a judge's own reasoning stated that the angry-toned answer was more direct and more technically precise — then selected the other one, citing confrontational tone as the disqualifier.

Identity bias extends the same mechanism to people: telling the judge a submitter's gender, religion, or health status shifts its verdict on identical text. The spread between models is large; weaker judges flipped their verdict on roughly a third of items purely on identity information.

Authority bias lets a fabricated citation reverse a previously correct verdict. Researchers took a pair where the judge had picked the right answer, appended an invented URL to the worse one, and the judge reversed its decision — citing the fake reference as its reason. Fabricated book citations and famous-person quotes performed even better as manipulators than URLs.

Bandwagon bias responds to stated consensus regardless of magnitude: claiming 60% agreement moved judges about as much as claiming 90%. Upvote counts, prior model rankings, or "preferred by X" metadata sitting in a judge's context window actively corrupt its evaluation.

Distraction bias adds irrelevant detail — a note that one assistant enjoys pasta — and the verdict moves. The damage is asymmetric: noise hurts evaluations of good answers more than bad ones, because bad answers are already near the floor.

Fallacy-oversight bias means judges ratify correct conclusions reached through broken reasoning. On the 3.8 vs. 3.11 comparison — where flawed digit-counting logic still yields the right answer — weaker judges pass the broken reasoning without flagging it.

Grounded Evaluation as the Structural Fix

The workshop's recommended remedy is architectural: replace open-ended quality questions with textual entailment questions. Asking "is this claim supported by this document?" removes the bias surface — the model takes no positional stance, handles one short sentence at a time, processes extracted claims without tone, and is decoupled from the generator. This maps directly to automated prompt optimisation workflows where a miscalibrated judge can silently degrade the entire feedback loop.

Purpose-Built Grounded Evaluators

Two production-ready options were cited at the workshop.

The workshop also flagged Prometheus 2, GLIDER, and Vectara HHEM as additional candidates worth evaluation.

The same prompt-manipulation vulnerabilities that concern practitioners in agentic security contexts apply directly to eval pipelines: injecting fake citations, consensus claims, or irrelevant metadata into a judge's context can deterministically shift its verdict. Teams shipping evaluation infrastructure should treat judge prompts with the same threat-modelling discipline they apply to user-facing inputs — auditing the full context window for contaminating signals and defaulting to grounded, claim-level verification wherever the architecture permits.