The development

What happened

GitHub introduced ReviewBench on 5 October, with 219 public pull requests across 19 languages. Its reference findings combine human review, frontier models and static analysis, while evaluation distinguishes known findings from newly discovered valid issues. GitHub reports alignment with its production experiments. These results describe the benchmark and the publisher's validation, rather than proving general effectiveness across every codebase.

Key facts

  • GitHub announced ReviewBench on 5 October 2026 as an open benchmark for AI code review agents.

  • The corpus contains 219 public pull requests across 19 languages. Sampling was informed by distributions from 103.9 million GitHub pull requests, rather than a corpus of that size.

  • Reference findings combine human reviewers, frontier models and static analysis.

  • Evaluation distinguishes precision and recall for known findings and newly discovered valid issues, with findings labeled by severity and category.

  • GitHub reports validation by senior engineers and alignment with its own production experiments; these are publisher-reported results.

The context

Why it matters

ReviewBench offers a common evaluation method for comparing what AI reviewers catch and how much noise they produce. Its corpus and precision/recall measures can inform tool selection, but performance on this benchmark does not by itself establish effectiveness on a team's own repositories.