@skeptrune wouldnt this be a lossy approximaition though?
since jev outputs logprobs and your method outputs a tokenised prediction, it would weight towards 0.9, 0.65, ect. "human" numbers
Still looks useful for 90% of jev's use cases
A fun use case of jev is adding it as a input parameter to your local models. allows for a sort of tailored intelligence, while deferring the annoying stuff. Its a nice upgrade from what i've seen so far!
I recently wrote a post about a version of reward models becoming a powerful part of the future, with intelligent and cheap routing deployed at scale.
Jev is this! (though not a reward model)
The future is here, and faster than i thought (as always)
After co-inventing ChatGPT, I kept asking myself: why have superhuman chat models not led to AGI?
I’ve spent the last 2 years in stealth building a new way to train models (RLCD), and a new type of frontier AI model that we are releasing today: Jev
• 20-200x faster
• 40-400x cheaper (w/ output tokens free)
• Frontier composable intelligence optimized for decisions
AFAICT the shortest path to AI-based economic revolution
I trained a small model to do syntax highlighting in the browser with GPU.
Meet gpu-lexer from Vercel Labs: Small (27.5KB), fast (runs on WebGPU), and language-agnostic (model guesses the syntax).
https://t.co/adphvt2Lv4
It is experimental and built for learning!
yep, in my startup data labelling is fully autonomous.
I have a gold calibration corpus, i run an autoresearch loop to optimise the prompt (usually hits 1.0 F1 if i label consistently) and then apply the model at scale to label a ridiculous amout of data.
Allows for some cool and capable internal models!