RealFeedApp Self-verification cannot reach pass@N
Increasing inference compute reduces the variance of model judgments but does not eliminate bias, causing accuracy to plateau below pass@N. The authors analyze recent test-time scaling methods through the lens of these two factors.
- Field
- training language models
- What they did
- Researchers showed that increasing compute for self-verification makes model evaluations more consistent, but not necessarily correct, as systematic errors persist.
- Why it matters
- This helps understand fundamental limits of methods where models check their own answers and simplifies design choice selection for such systems.
#self-evaluation#bias#variance#test-time compute#llm
