Skip to content

@erik_kokalj tested the new Gemini Decisions API on a recycling line and says it is more accurate, and more expensive, than the other multimodal System 1 alternatives he names, Jev and OAI. Accuracy on this clip: Gemini Decision 1 76%, OAI Decisions API (GPT-6 Luna) 53%, Jev 1.14 72%. Cost per 1,000 images / median speed: Gemini $0.13 / ~340 ms, OAI $0.08 / ~270 ms, Jev $0.08 / ~130 ms. He says @typesafeai’s Jev still has the best performance per price.

Benchmarks

Erik Kokalj @erik_kokalj

EN

  • 6.2k views
  • 125 likes
  • 106 bookmarks
  • 5 reposts
Poster from Erik Kokalj (@erik_kokalj)

About this case

Benchmark. Erik Kokalj (@erik_kokalj) says the rankings on this clip mostly line up with the models’ own benchmarks. Video attached.

Original posthttps://x.com/erik_kokalj/status/2121254759453671918

Related cases