hallucination on S Anand

Can AI Replace Human Paper Reviewers?

Tue, 13 Jan 2026 09:55:45 +0530

Stanford ran a conference called Agents for Science. It’s a conference for AI-authored papers, peer reviewed by AI.

They ran three different AI systems on every paper submitted, alongside some human reviewers. The details of each of the 315 papers and review are available on OpenReview.

I asked Codex to scrape the data, ChatGPT to analyze it, and Claude to render it as slides.

The results are interesting! I think they’re also a reasonably good summary of the current state of using AI for peer review.

The three AI reviewers wildly disagree with each other.

Imagine hiring three movie critics to rate the same film. One gives it 2 stars, another gives it 6 stars, and the third gives it 4 stars. Same movie, completely different conclusions. That’s what’s happening with these AI reviewers-on almost half of all papers.
“Averaging” the three AIs doesn’t actually help

You might think: “Just average the three scores! That’ll balance out their biases.” But here’s the problem: the generous AI (AIRev2) uses much bigger numbers. When you average, its voice drowns out the others. It’s like having three judges, but one shouts and two whisper.
Every AI claims to be 100% confident - even when they’re wrong

Reviewers are asked “How confident are you in your assessment?” on a 1-5 scale. Every single AI review said “5 out of 5-totally confident.” All 751 of them. Even when two AIs looked at the same paper and reached opposite conclusions, both claimed maximum confidence.
AI and human reviewers see different things

On papers that got both AI and human reviews, we compared their scores. The AIs were almost always more generous than humans-by about 1 full point on average. And in some cases, AI said “excellent!” while the human said “this is broken.”
AI reviewers can catch obvious problems

AI reviewers successfully flagged papers with impossible claims-like citing AI models that don’t exist yet, or referencing datasets from the future. These are “fact check” problems that don’t require deep expertise, just attention to detail.
Use AI disagreement as a signal, not noise

When the “generous AI” loves a paper but the “skeptical AI” hates it, that’s not random noise-it’s useful information. It means the paper’s fate depends on standards (rigor vs. novelty), not just quality. These are exactly the papers humans should look at.

Humans have taught LLMs well

Thu, 08 Jan 2026 14:29:56 +0800

Human	LLM
Bullshitting: Humans confidently assert wrong information, from flat-earth beliefs to misremembered historical “facts” and fake news that spread through sheer conviction	Hallucination: LLMs generate plausible but factually incorrect content, stating falsehoods with the same fluency as facts
People-Pleasing: Humans optimize for social harmony at the expense of honesty, nodding along with the boss’s bad idea or validating a friend’s flawed logic to avoid conflict	Sycophancy: LLMs trained with human feedback tell users what they want to hear, even confirming obviously wrong statements to avoid disagreement
Zoning Out: Humans lose focus during the middle of meetings, remembering the opening and closing but losing the substance sandwiched between	Lost in the Middle: LLMs perform well when key information appears at the start or end of input but miss crucial details positioned in the middle
Overconfidence: Humans often feel most certain precisely when they’re least informed—a pattern psychologists have documented extensively in studies of overconfidence	Poor Calibration: LLMs express high confidence even when wrong, with stated certainty poorly correlated with actual accuracy
Trees for the Forest: Humans can understand each step of a tax form yet still get the final number catastrophically wrong, failing to chain simple steps into complex inference	Compositional Reasoning Failure: LLMs fail multi-hop reasoning tasks even when they can answer each component question individually
First Impressions: Humans remember the first and last candidates interviewed while the middle blurs together, judging by position rather than merit	Position Bias: LLMs systematically favor content based on position—preferring first or last items in lists regardless of quality
Tip-of-the-Tongue: Humans can recite the alphabet forward but stumble backward, or remember the route to a destination but get lost returning	Reversal Curse: LLMs trained on “A is B” cannot infer “B is A”—knowing Tom Cruise’s mother is Mary Lee Pfeiffer but failing to answer who her son is
Framing Effects: Humans give different answers depending on whether a procedure is framed as “90% survival rate” versus “10% mortality rate,” despite identical meaning	Prompt Sensitivity: LLMs produce dramatically different outputs from minor, semantically irrelevant changes to prompt wording
Rambling: Humans conflate length with thoroughness, trusting the thicker report and the longer meeting over concise alternatives	Verbosity Bias: LLMs produce unnecessarily verbose responses and, when evaluating text, systematically prefer longer outputs regardless of quality
Armchair Expertise: Humans hold forth on subjects they barely understand at dinner parties rather than simply saying “I don’t know”	Knowledge Boundary Blindness: LLMs lack reliable awareness of what they know, generating confident fabrications rather than admitting ignorance
Groupthink: Humans pass down cognitive biases through culture and education, with students absorbing their teachers’ bad habits	Bias Amplification: LLMs exhibit amplified human cognitive biases including omission bias and framing effects, concentrating systematic errors from their training data
Self-Serving Bias: Humans rate their own work more generously than external judges would, finding their own prose clearer and arguments more compelling	Self-Enhancement Bias: LLMs favor outputs from themselves or similar models when evaluating responses

Via Claude

Inspired by LLM problems observed in humans.

Scrabble image generation

Tue, 06 Jan 2026 22:12:34 +0800

AI image generation still has a long way to go. Here are two images generated by Gemini and ChatGPT from the same prompt: “Create a funny scrabble board of dysfunctional family relationships!”

Gemini

It’s probably showing off, with coffee stains, and spelling “DYSFUNCTIONAL” right. But “ABLOMY”? “PASSIAVE”? “RGUCT_SVA”? “SORDSP”? Most of the vertical letters are wrong. Some horizontals (“DTENSION”?) are off, too.

Also: “Z” has 2 points? “C” has “C” points? “DOUBLE STTER SCORE”? “UUT SCORE SCORE” instead of “TRIPLE WORD SCORE”?

But one thing is clear, the number of times “DOUBLE STTER SCORE” appears indicates that it does some mental copy-pasting!

ChatGPT

This is almost as bad. “FAKEHUY”? “MREGSUUEAHL”? The verticals are worse than the horizontals, but some horizontals are off, too. “DISAPPOINTI”? “THERPY”?

“INLAWS” are hanging outside the board. The “TRIPLE WORD” at the bottom right and bottom left are not at the corners, and are missing a “SCORE”. More importantly, some of the letters aren’t printed right.

Overall, slightly fewer errors, but slightly poorer style, too.

Sat, 10 May 2025 09:35:14 +0000

How can we rely on unreliable LLMs?" people ask me.

Double-checking with another LLM," is my top response. That’s what we do with unreliable humans, anyway.

LLMs feel magical until they start confidently hallucinating. When I asked 11 cheap LLMs to classify customer service messages into billing, refunds, order changes, etc. they got it wrong ~14%. Not worse than a human, but in scale-sensitive settings, that’s not good enough.

But different LLMs make DIFFERENT mistakes. When double-checking with two LLMs, they were both wrong only 4% of the time. With 4 LLMs, it was only 1%.

Double-checking costs almost nothing. When LLMs disagree, a human can check it. Also, multiple LLMs rarely agree on the same wrong answer.

So, instead of 100% automation at 85% quality, double-check with multiple LLMs. You can get 80% automation with 99% quality.

Full analysis: https://sanand0.github.io/llmevals/double-checking/
Code and data: https://github.com/sanand0/llmevals/tree/main/double-checking

Thu, 19 Sep 2024 11:00:49 +0000

Today, I learned that I began my career at TCS not IBM, and I never worked at the Boston Consulting Group (BCG)

I am very curious (but a bit scared) to ask an #LLM whom I’m married to.

Tue, 06 Feb 2024 03:21:07 +0000

For those in #Singapore and interested in #datavisualization & #llms, I’m talking about Visualizing LLM Hallucinations at SUTD on Thu 8 Feb at 7 pm SGT.

This is for a non-technical audience. We’ll visualize the basics of how LLMs work, how they make mistakes, and at least one technique on how to spot these.

https://www.meetup.com/data-vis-singapore/events/298902921/