@erik_kokalj tested the new Gemini Decisions API on a recycling line and says it is more accurate, and more expensive, than the other multimodal System 1 alternatives he names, Jev and OAI. Accuracy on this clip: Gemini Decision 1 76%, OAI Decisions API (GPT-6 Luna) 53%, Jev 1.14 72%. Cost per 1,000 images / median speed: Gemini $0.13 / ~340 ms, OAI $0.08 / ~270 ms, Jev $0.08 / ~130 ms. He says @typesafeai’s Jev still has the best performance per price.
- 6.2k views
- 125 likes
- 106 bookmarks
- 5 reposts
About this case
Benchmark. Erik Kokalj (@erik_kokalj) says the rankings on this clip mostly line up with the models’ own benchmarks. Video attached.
Original posthttps://x.com/erik_kokalj/status/2121254759453671918