Skip to content

Paper: Jev matches GPT-6 Astra on RAG claim checks within 0.6 pts at ~187× lower cost, but lets more unsupported claims through.

Benchmarks

Rohan Paul @rohanpaul_ai

EN

  • 4.7k views
  • 59 likes
  • 25 bookmarks
  • 7 reposts
Poster from Rohan Paul (@rohanpaul_ai)

About this case

Benchmark/paper thread with media. rohanpaul_ai summarizes a study comparing Jev (label+probability) to GPT-6 Astra on 495 human-labelled RAG claims, plus a cascade. Jev was within 0.6 accuracy points at 187× lower cost but approved 23.2% unsupported claims vs Astra 14.1%. Advice: measure false accepts, not only headline accuracy.

Original posthttps://x.com/rohanpaul_ai/status/2115026823377748246

Related cases