@honnibal ran Jev with Ellf on data from Opus few-shot classifiers; Jev agreed with Opus on 95% of the inputs.

Benchmark with a photo. A user project was built with many few-shot classifiers powered by Opus. Ellf ran Jev over the same data, and Jev agreed with the Opus decision on 95% of the inputs. Samples where the classifiers disagreed were queued for annotation, and those annotations identified a few necessary refinements to the prompt for Jev. For people comparing Jev with an existing Opus classifier.