@milindlabs does computer use without screenshots or an LLM leaving the Mac: a local CoreML model segments buttons, on-device OCR reads labels, and Jev returns a probability over those elements and clicks in a loop until the goal is done. They report about 90ms per decision.

Milind S (@milindlabs) published this on 2026-09-17. @milindlabs does computer use without screenshots or an LLM leaving the Mac: a local CoreML model segments buttons, on-device OCR reads labels, and Jev returns a probability over those elements and clicks in a loop until the goal is done. They report about 90ms per decision. Okay so Jev can actually do computer use really well Without any screenshots, or LLMs and no Pixels leave my mac I dont even read the Dom elements A local CoreML model segments every button and UI element on screen. How Jev or TypeSafe is used, in the post's own words: “Okay so Jev can actually do computer use really well Without any screenshots, or LLMs and no Pixels leave my mac I dont even read the Dom elements A local CoreML model segments every button and UI element on screen.” The post does not name a separate audience. It is for people reading what @milindlabs shows here (it quotes @CompleteSkeptic; it includes media).