@0xironyAditya says @liquidai released another model, so he tested d1.2-ultra against the other decision models on two real jobs: grading storm and fire damage on before/after satellite shots, and sorting Serengeti camera-trap bursts into empty or which animal. The setup is 200 picture decisions, the same pictures and the same questions. d1.2-ultra versus Jev 1.14 versus gpt-6 luna versus clef, all through @OpenRouter: exact damage grade 47% / 46% / 42% / 41%; send a team first 70% / 64% / 68% / 69%; animal or empty 99% / 96% / 96% / 98%; species 62% / 63% / 62% / 65%; cost for all 200, $0.018 / $0.034 / $0.034 / $0.083. He writes that accuracy is a near tie at the top, and that d1.2-ultra got there at half the price of Jev and luna and almost 4.7× cheaper than clef. The only catch is that it was the slowest of the four. These cannot replace the person doing the job yet, but they can help sort the pile, double-check calls, and clear out the empty frames.
- 94 views
- 5 likes
- 0 bookmarks
- 1 reposts
About this case
Benchmark. irony.aditya (@0xironyAditya) attaches a video. The post text has no URL.
Original posthttps://x.com/0xironyAditya/status/2123153498875547855