@kevintheo says he uses a closed choice model, Jev by @typesafeai, as a prompt-injection guard for his email agent. On one test of 1,102 texts it found 82.6% of attacks and 40 of 40 indirect ones, with 0.7% false alarms (old keyword filter 27.3%).
- 63 views
- 2 likes
- 0 bookmarks
- 0 reposts
About this case
Agent. Kevin Theo (@kevintheo) says he is curious how Gemini Decision 1 does on that job. Text only. He quotes Sundar Pichai (@sundarpichai), who says they are launching the Gemini Decisions API, powered by the new Gemini Decision 1 model in @GoogleAIStudio. Instead of generating text token by token, it is built for closed-set choices, scoring up to 1,000 questions across text, images, audio, video and PDFs in a single pass. The three decision types listed are yes/no, multiple choice up to 255 options, and rubric scoring. He states $0.10 per 1M input tokens, with $0 billed for output tokens, about 5x cheaper than Flash-Lite, and that it leads on several benchmark evals with the lowest calibration error. Google teams are already building with it, and he points to Stitch Rapid designing a full product flow in seconds, shown at 1x speed.
Original posthttps://x.com/kevintheo/status/2120586339523694876