As large language models increasingly take part in scientific evaluation, a new research paper from Hugging Face investigates a potential form of reward hacking: how rhetorical choices shape AI review judgments when the reported scientific content is preserved. The study, titled "How Can Rhetoric Reward-Hack AI Reviewers? Dissecting Rhetorical Sensitivity in AI-Based Peer Review," systematically examines six dimensions of scientific rhetoric.
The researchers constructed a controlled corpus of 4,200 full-paper manuscripts derived from 120 anonymized ICLR 2026 submissions. Two LLM rewriters transformed six rhetorical dimensions in opposing directions, and five LLM reviewers evaluated the resulting manuscripts under standard and strict protocols. They also tested joint, recursive, and reviewer-guided rewriting, accumulating over 42,000 AI reviews and nearly $30,000 in API costs.
The results reveal a clear hierarchy of rhetorical sensitivity: evidence framing and novelty stance produce the largest positive-negative contrasts in overall assessment, with scope framing forming a weaker second tier. Technical register and linguistic complexity have smaller or less stable effects. This hierarchy persists across human-assessed quality levels, but score movement depends strongly on the AI reviewer's original score: lower scores tend to rise, higher scores tend to fall, and directional contrasts are clearest in the middle ranges.
More elaborate workflows do not reliably yield larger gains. Joint rewriting is strongly rewriter-dependent, reviewer guidance does not consistently outperform an unguided second pass, and repeated rewriting yields diminishing, configuration-dependent returns. Across conditions, the rewriter primarily determines the separation between opposing variants, whereas the reviewer determines the magnitude and sign of their score effects. Strict review lowers mean overall assessment by 1.36 points without consistently changing rhetorical sensitivity.
These findings identify when rhetorical presentation influences AI scientific review and motivate evaluation systems robust to content-preserving variation in scientific writing.