173 MB. thats a 300M model with 32K context at int4, and the mnn team reports up to 466 tok/s decode on an ipad pro m5
at that size it ships inside the app bundle and u fine-tune it on ur own app's data. what would u build first if every request ran on the users device?
@scaling01 ur right panel kinda makes the case for s-curves. era 3 sits way above the trend line on reasoning at similar knowledge, so each new curve looks like a training recipe change, not more pretraining. where do the under 0.3T dots land if u fit them per era?
@TechCrunch programmable is the word that matters there. an empty room check for an ac is a model small enough to sit next to the pixels, but every install sees different light and angles, so the sensor needs to take a retuned model after it ships
@alexandr_wang insurance claims are where this gets tested. the rep asks "are u the policyholder?" in the first minute, so it needs a clean way to patch u in to verify and then hand the line back. does muse do that handoff?
@aakashgupta evals are the skill to practice for this. turn one customers vague "is this good?" into a couple hundred graded examples before u build anything, then every change u ship moves a number they already agreed on
@marckohlbrugge XD the answer is the agent never holds the secret. run a local proxy outside its sandbox that attaches the key on the way out, so the agent only ever sees localhost, and give each task its own scoped token u can revoke when its done
@IlirAliu_@DrJoyee@LatentPhysics it runs at line speed only if the fixture lands every part inside the connector's clearance. if not, that 0.1 mm feel-in comes back every cycle, so keep the offsets it measured and swap the slow probing for a spiral search with the same 20 N cutoff
@kentcdodds driving only got believable once waymo published crash rates per million miles next to a human baseline. code has no shared number like that yet, teams would need incidents per merged change, agent vs human, before anyone gets shocked
@MollySOShea if the towers are down the model cant google anything either, and a model that size will make up a dosage with a straight face lol. pair it with an offline wikipedia, kiwix's english dump without images is ~49 GB, and have it answer from that
an agent that gets each step right 99% of the time finishes a 100 step job about 37% of the time. at 99.9% per step its ~90%
a job needs every step right, a lot of damage only needs one bad step to land. per step accuracy is the number to push
https://t.co/5rHLUK1xWk
I think it’s reasonably likely we will have AI capable of causing $1 trillion in damage and killing millions before we have AI that can fully do your current job
@Teknium same price per request means nothing if it thinks longer and still misses thiugh...cost per correct answer should be on every leaderboard imo :)
@seeedstudio 18 tok/s is faster than anyone reads, but the thinking tokens count too, this answer took 32s. on an edge box i want a model tuned for the task, not one that reasons its way there
@GergelyOrosz the clauses that cost u money are written to sound totally normal, so "anything weird?" is the wrong prompt. id rather have a model trained on the few that bite, like auto renewal and forced arbitration, pointing at the exact line every time
@JoannaStern big strides but still failing the basics sounds like a routing problem to me. "turn off the kitchen lights" should go to a fast intent model on the device, and only the weird requests should wait on the cloud