@Mav1972 checked 54 storefront screenshots with short yes/no questions. Jev answered in 0.4 s a call, caught 19 of 22 planted defects, and raised 0 of 24 false alarms on harmless changes.
- 12 views
- 0 likes
- 0 bookmarks
- 0 reposts
I checked 54 storefront screenshots with short yes/no questions about what each screen must show. Jev answered them in 0.4 s a call, caught 19 of 22 planted def…
About this case
Benchmark note. Mav1972 says they checked 54 storefront screenshots with short yes/no questions about what each screen must show. Jev answered them in 0.4 s a call, caught 19 of 22 planted defects, and raised 0 of 24 false alarms on harmless changes. One photo is attached. Its alt text calls it a bar chart of defects caught out of 22 and false alarms out of 24 for nine screenshot checkers, and names Jev with yes/no assertions at 19 caught and 0 false alarms, Gemini at 22 and 0, and pixel diff at 13 and 16.
Original posthttps://x.com/Mav1972/status/2117756513171485150