# Stress-test AI-written research before you trust it

Sakana says a paper written by its AI Scientist-v2 scored above the acceptance threshold (6, 7 and 6) at an ICLR 2025 workshop, then withdrew it before meta-review, as agreed.

Organisation: Sakana AI, with researchers at the University of British Columbia and the University of Oxford
Status: Research
Place: Japan
Categories: Science & discovery
Try it with: Claude, ChatGPT / OpenAI models, Gemini, Grok
Time to try: 30 minutes

## Why it matters

It is an honest test of where AI-generated science stands: good enough, Sakana says, to score above one workshop's acceptance threshold, with clear limits. Useful for anyone deciding how much to trust AI research.

## How it works

With the agreement of ICLR leadership and the workshop organisers, Sakana submitted three AI-generated papers for double-blind review. The one that passed averaged 6.33 and was withdrawn before publication, as planned (Sakana). TechCrunch notes that workshop acceptance rates tend to be higher than for the main conference track.

## Who it's for

Researchers, editors and research-integrity teams.

## The prompt

```
You are a skeptical reviewer for [venue]. Review the paper I paste below. Only paste a paper you wrote or have permission to share: many venues forbid putting confidential submissions into AI tools.

1. Summarise its claim in 3 sentences.
2. Check every number in the abstract against the results section and list mismatches.
3. List missing baselines, ablations or statistics, and any claims the experiments do not support.
4. Check that each cited paper exists and says what the text claims. If you cannot look a paper up, mark it 'not verified' and do not guess.
5. Score it from 1 to 10 using the venue's criteria, with your confidence.

Be specific and quote the passages you criticise.

[Paste the paper]
```

## Caveats

One of three AI papers passed a workshop review; it did not go through a main-conference review.

## Sources

1. Sakana AI: Scores 6/7/6; withdrawn by protocol; no meta-review. The AI Scientist generates its first peer-reviewed scientific publication (https://sakana.ai/ai-scientist-first-publication/)
2. TechCrunch: Three papers submitted; workshop acceptance rates higher; no meta-review. Sakana claims its AI paper passed peer review, but it's a bit more nuanced (https://techcrunch.com/2025/03/12/sakana-claims-its-ai-paper-passed-peer-review-but-its-a-bit-more-nuanced-than-that/)

Fact-checked: 16 Oct 2026

Page: https://escapevelocitysociety.com/picks/sakana-ai-scientist-peer-review

See Selected Projects: https://escapevelocitysociety.com/picks.md
