AI Wrote a Lot of Peer Reviews at a Major AI Conference

The newest wrinkle in artificial intelligence is an old one, reinvigorated by so-called deep learning: AI models known as neural networks can be trained to learn certain tasks and to seek conclusions that are drawn well — “generalize” beyond the specifics of the training data. A report in Nature finds that a large percentage of peer reviews submitted to a prestigious machine-learning conference were authored entirely or partly by large language models, forcing the field to confront questions about authenticity, oversight, and the future of scientific gatekeeping.

The results amplify a pressure cooker on the field. Peer review was already overwhelmed by a torrent of submissions and impossible deadlines; generative AI has provided an attractive backdoor that some reviewers — and even authors — seem to be using with real consequences for decisions and reputations.

The numbers behind the claims of AI-written peer review

US start-up Panagram, which creates AI-writing detection tools, analysed about 19,490 submissions and 75,800 reviews for a leading AI conference. The company’s analysis, which it has shared with organizers and was reported by Nature, found about 21% of the reviews were completely AI-written, while more than half indicated some degree of AI help. From the authors, 1% of submissions were categorized as being fully AI-generated and 9% had more than half their text written by an AI.

These numbers haven’t been independently vetted, and they carry some important caveats: Detection is probabilistic; thresholds make a difference; false positives are possible. Still, the magnitude of the signal is hard to ignore, and it squares with a growing volume of anecdotal evidence from researchers who say they are encountering shallow, generic, or factually incorrect critiques that seem like machine-produced writing.

A peer review system creaking under the scale of submissions

Peer review in A.I. is mostly volunteer work, and the field has grown dramatically. The premier conferences in OpenReview draw thousands of papers and tens of thousands of reviews. That workload falls on a finite number of scholars, who have to balance their own teaching, grants, and research. It’s hardly surprising, then, that some are looking to LLMs for drafting assistance, even if the line between aid and outsourcing is starting to blur.

Researchers have voiced alarm. Graham Neubig of Carnegie Mellon University publicly challenged questionable reviews, and in turn encouraged more scrutiny. Desmond Elliott at the University of Copenhagen described a review of a student’s paper as “missed the point,” later identified as fully AI-generated by Panagram. According to Nature, some authors retracted submissions after receiving reviews that included demonstrably incorrect statements.

Why AI-generated reviews can go awry in scientific peer review

Peer review is about more than just prose. It includes wrestling with experimental design, probing failure modes, verifying theoretical claims, triaging novelty against prior art. LLMs are good at generating plausible text, not independently verifying math, stress-testing statistical assumptions, or reproducing results. They are also susceptible to hallucinations that can become confident but inaccurate complaints or manufactured citations.

Confidentiality is another fault line. Reviews often contain nonpublic information; running them through online tools could result in exposure on vendor logs or be used as part of model training for future models. Many journals and conferences prohibit submission of manuscripts to third-party services, including AI ones that cannot ensure confidentiality.

Detection has boundaries, but governance can help

AI-written text detectors are imperfect. Stylometric signals can be faked, and reviewers vary in native fluency, complicating matters of judgment. That’s why there are also calls from experts that detection shouldn’t be a singular signal, but should be coupled with audits by area chairs and demands for reviewer accountability.

There are guardrails to borrow from. The Committee on Publication Ethics calls for transparency when the content is driven by AI, and etches accountability on actual humans. Top journals call for AI involvement to be disclosed and dismiss AI as a credited author. Conferences have the ability to graft them into their review process: make disclosure mandatory, require attestation, and give chairs the power to pull keywords or paper titles for spot checks of suspicious language or citations.

What conferences can do now to improve AI-era peer review

1. First, raise the bar for reviewer choice and training, do short modules on responsible AI assistance and examples of unacceptable use.
2. Second, make it compulsory for any reviewer to submit a signed statement detailing the role of AI and/or prompts tested, as well as efforts to check if claims are valid — kept secret but auditable.
3. Third, apply AI judiciously on the organizer side: not to write reviews but rather to bring anomalies — sudden swings in tone, repetitive verbiage between disparate reviews, references that don’t exist — to the attention of humans for follow-up.
4. Fourth, establish structured rubrics that force reviewers to interact with methods, ablations, and limitations. Shallow summaries are easier for LLMs; deep methodological probes are not.
5. Fifth, design in punitive measures: a pattern of AI-outsourced reviewing should result in exclusion from the reviewer pools and warnings to institutions.

The stakes for A.I. research and the credibility of the field

When AI contributes to authoring the reviews that sift through AI research, there is a potential for a feedback loop that magnifies hype, obscures shortcomings, and undermines trust. The costs are felt now and eventually, as reproducibility and public confidence in science wane.

And this isn’t the first profession shaken up by generative models; courts have already received pleadings replete with made-up citations, according to reporting in The New York Times. But the irony is more acute in AI, where an entire field’s credibility depends on demonstrating that its own governance can withstand the allure of the tools it creates. The solution is not to ban support, but instead regain human control, disclose use, and keep critical thinkers at the center of human peer review.