humans have had continual learning for 200k+ years and most of us are still stuck in local maxima doing npc jobs. it sounds great but when you try to implement it, you realize the gradients just violently fight each other in the latent space. neuroplasticity just isn't a native property of these architectures. backprop fundamentally wants to find a local minimum and stay there. this forces it into continuous real-time updates, and it fights with the math.
you probably need to solve meta/emprical learning first to figure out what's actually pareto optimal to keep.
yeah, this is the baseline scenario. this is why open source has to win, badly. superintelligence in the hands of an oligopoly is so much more dangerous than it just being commoditized for everyone.
very very good read. some thoughts around headlessness in bits and pixels world -
> inference costs will soon hit almost zero. there will be infinite tokens and infinite intelligence for everyone. everyone will spin up disposable and non-disposable software. headless or with beautiful ui, is one of the categorizations.
> but the most probable and final one is this category - there will be either crystalline software or autopoietic software.
> crystalline software facilitate processing and storage of records across different use cases. the data profile of the user may evolve but the software does not change. in an infinite tokens world, users will spin up crystal custom CRUD apps for a transient need and will discard it after use. the economic value of this class will disappear. regardless of it being headless or with beautiful UI.
> the other class is autopoietic software that will learn from its environmental entropy and will evolve it's own topology. this class of software will consume continuous feedback and recursively improve. the quality and number of users will drive divergence and moat. if a high value consumer is feeding your software with their working principles, will make it lucrative for other consumers to reap software evolution benefits.
> defensibility will move from code to system's velocity to learn and self mutate. there will be more forms of beautiful UIs in the future and I believe interfaces are here to stay for a very very long period of time.
there is also a massive data opportunity in deliberately discovering and engineering entirely new spectrums of observability, new verticals that have never been measured before.
let alone the progress in frontier intelligence, this will birth civilizational economic biomes.
the more general purpose a robot's form factor, the lower its efficiency and reliability at any specific task compared to specialized automation
buyers will learn this the hard way that they paid a premium for versatility they didn't actually need, sacrificing the throughput they needed. and so will the labs.
switching to another model and harness is probably the right move here. the false-positive rate alone should alarm anyone who works in ML, RL, or training pipelines.
for what it’s worth, I am having a weird experience trying to use fable through claude code to set up Pi harness for some mujoco automations. it felt oddly resistant, almost like it didn’t want to help with that path. i don’t have proof of anything, so I’m not making a strong claim there.
this will backfire hard. there are now hundreds of thousands of ML engineers and researchers with a fresh, highly personal reason to reduce their dependence on closed models.
for all the criticism openai gets, they’ve been genuinely open in this specific way. their models don’t have this kind of baked-in, anti-competitor bias. I hope they keep releasing models that make that contrast impossible to ignore.
We're releasing a whole new category of voice models.
Introducing DramaBox — our state-of-the-art, open source voice model built for cinematic use cases.
Traditional TTS gives you a voice. DramaBox by @resembleai gives you a performance.
For too long, Voice AI has been stuck in "robotic assistant" mode. If you wanted dramatic emotion, sighs, or a voice cracking with grief, you had to hire an actor or spend hours editing. We fixed that.
every country that isn't the US or China is about to have an AGI sovereignty crisis. most of them just don't know it yet.
right now, governments in india, the Gulf states, Southeast Asia are in the "AI is exciting, let's promote it" phase. they're building AI ministries, hosting summits, announcing national strategies. vibes are good.
but the phase shift will come. the moment foundation models start touching critical infra like defense, healthcare, finances, these governments will wake up and realize they have zero sovereign capability. no domestic foundation models. no compute. no talent pipeline to build either. complete dependency on two countries, one of which they can't trust and one of which might restrict access at ny time.
that's when "let's promote AI" turns into "we need AI partnerships yesterday". that is when whoever has deep institutional relationships in these countries captures a decade of compounding leverage.
OpenAI and Anthropic are leaving an absurd amount on the table here. the GTM in these regions isn't PLG. it's government down. a minister talks to a sovereign wealth fund talks to the top 20 enterprises and suddenly you're the national AI partner. one relationship will unlock an entire economy.
labs will need to build squads who understand the institutional grammar as much as they understand the bay area tech scenes. someone who grew up in these systems AND understands what the lab actually offers at a technical level.
if you're early, you're a partner. late, you're a vendor competing on price against local alternatives that governments will fund out of national security panic whether they're good or not.
and your prize is that you will become the infrastructure layer for countries whose entire AI strategy will depend on you.
what used to be a complete company is now a skill in hermes. @Teknium really cooked with this one.
hermes is a learning harness. it will watch itself solve something a couple of times and improve its skills with each subsequent iteration. if you have an engineering or scientific problem that you repeatedly solve that follows a pattern, I highly recommend using hermes and liberate yourself for the next big quest.
gpus appreciating while everything they produce gets cheaper is a genuinely weird economic dynamic. no one has priced this in yet. but the price is tied to the access to abundant intelligence. compute scarcity gating who gets to use intelligence must lure you way more than unemployment.
ironic to me when they say ai just matrix multiplications. because the brain is just electrochemistry, music is just air vibrations, love is just oxytocin, and the universe is just a simulation.
do not fear artificial superintelligence, fear artificial mediocrity deployed for billions. the mid facebook model making mid-decisions for billions of people will make it more dangerous than a genius model making genius decisions for just millions.
i'm living on the agentphasic sleep. only claude code and codex hitting limits can put me in REM sleep. i only rest during the compute starvation periods.
might as well call agent-in-the-loop circadian rhythm
> still no unlimited tokens
> still have to tell Opus what to do
> codex isn’t running at 50k tokens/sec
> still gotta pay the rent
> ai still can’t win arguments with my wife for me
when it will be singularity, you will know
deploy frontier models on subclass tasks, and it will put the value ceiling below the cost floor. model intelligence will become a commodity, and commodities are won on price. ccp models wholesaling tokens at 1/10th the cost will compete with american labs at pricing on tasks where capability is irrelevant. frontier labs either reprice for the commodity tier or cede the volume market entirely.