@omooretweets only thing that actually matters is whether the model makes the right decision, pulls the right info, uses the right tool, at the right moment (issues we have at current LLM dependency on voice)
@omooretweets Pricing’s too steep to actually run this at production scale and know if it works , So nobody’s really testing it, they’re demoing it tbh
The real shift: stop trying to guarantee outputs. Start building systems that survive being wrong : fallbacks, human handoff, tight scopes, constant production monitoring instead of pre-launch theater.
Trying to fully test an LLM is like trying to catch smoke with your bare hands. You can see it, you can wave your arms through it, you can even predict roughly where it’ll drift, but you’re never actually holding it.
These platforms take the upside and none of the responsibility. You built the business. You eat the risk. That imbalance doesn’t go away as you scale , it gets worse.
Getting LLMs to return the correct time, understand intent, fetch the right knowledge, and keep STT/TTS quality stable at scale is harder than most people realize. Voice AI is not for the faint-hearted.
5 minutes ago, @karpathy just dropped karpathy/jobs!
he scraped every job in the US economy (342 occupations from BLS), scored each one's AI exposure 0-10 using an LLM, and visualized it as a treemap.
if your whole job happens on a screen you're cooked.
average score across all jobs is 5.3/10.
software devs: 8-9.
roofers: 0-1.
medical transcriptionists: 10/10 💀
https://t.co/7MWRgdtLDI
Tonight, we reached an agreement with the Department of War to deploy our models in their classified network.
In all of our interactions, the DoW displayed a deep respect for safety and a desire to partner to achieve the best possible outcome.
AI safety and wide distribution of benefits are the core of our mission. Two of our most important safety principles are prohibitions on domestic mass surveillance and human responsibility for the use of force, including for autonomous weapon systems. The DoW agrees with these principles, reflects them in law and policy, and we put them into our agreement.
We also will build technical safeguards to ensure our models behave as they should, which the DoW also wanted. We will deploy FDEs to help with our models and to ensure their safety, we will deploy on cloud networks only.
We are asking the DoW to offer these same terms to all AI companies, which in our opinion we think everyone should be willing to accept. We have expressed our strong desire to see things de-escalate away from legal and governmental actions and towards reasonable agreements.
We remain committed to serve all of humanity as best we can. The world is a complicated, messy, and sometimes dangerous place.