@ikkistan: Jev, Haiku, and clef-flash reach AUROC 0.86 to 0.87 on unsafe images, Rekognition 0.70, at 4 to 18 cents per 1,000 images versus $1.
ikki 🇵🇸🇸🇩 @ikkistan
EN
- 71 views
- 4 likes
- 0 bookmarks
- 0 reposts
Jev, Haiku and clef-flash rank unsafe images about equally well (AUROC 0.86 to 0.87, Rekognition 0.70) at 4 to 18 cents per 1,000 images vs $1. Code, per-image…
About this case
Benchmark post, text only. Jev, Haiku, and clef-flash rank unsafe images about equally (AUROC 0.86 to 0.87; Rekognition 0.70) at 4 to 18 cents per 1,000 images versus $1. Code, per-image outputs, and a script to recheck every number: https://github.com/Luminary-AI-Solutions/image-moderation-bench. For people comparing moderation cost and AUROC.
Original posthttps://x.com/ikkistan/status/2112993870317572195