Summary
An analysis comparing ICLR 2025 and 2026 paper review scores revealed a significant drop in inter-human reviewer correlation for 2026, indicating increased inconsistency. While ICLR 2025 showed a correlation of 0.41, the current project found a much lower figure for 2026. This suggests a worsening "lottery" effect in top conference reviews, highlighting a growing problem with the reliability and fairness of peer review in major AI conferences.
Editorial note
AI Dose summarizes public reporting and links to original sources when they are available. Review the Editorial Policy, Disclaimer, or Contact page if you need to flag a correction or understand how this site handles sources.
Continue Reading
Explore related coverage about community news and adjacent AI developments: [r/ML] [D] MYTHOS-INVERSION STRUCTURAL AUDIT, [r/LocalLLaMA] karpathy / autoresearch, [r/ML] [D] Will Google’s TurboQuant algorithm hurt AI demand for memory chips? [D], [HN] Show HN: Ship of Theseus License.
Related Articles
- [r/ML] [D] MYTHOS-INVERSION STRUCTURAL AUDIT
March 29, 2026
- [r/LocalLLaMA] karpathy / autoresearch
March 10, 2026
- [r/ML] [D] Will Google’s TurboQuant algorithm hurt AI demand for memory chips? [D]
April 12, 2026
- [HN] Show HN: Ship of Theseus License
April 10, 2026
Next read
[r/ML] [D] MYTHOS-INVERSION STRUCTURAL AUDIT
Stay with the thread by reading one adjacent story before leaving this update.
Comments
Sign in to leave a comment.