Grateful for every memory, and a thank you to everyone who backed me along the way.
A special journey, but time to hang up the boots. On to the next chapter ❤️🏉
Recursive Language Models (RLMs) let agents manage 10M+ tokens by delegating tasks recursively.
This Google Cloud Community Article explains why ADK was the perfect choice for re-implementing the original RLM codebase in a more enterprise-ready format →https://t.co/p3MsNtLVJL
To perfectly understand a phenomenon is to perfectly compress it, to have a model of it that cannot be made any simpler.
If a DL model requires millions parameters to model something that can be described by a differential equation of three terms, it has not really understood it, it has merely cached the data.
"A turkey is fed for a thousand days by a butcher; every day confirms to its staff of analysts that butchers love turkeys with increased statistical confidence." - Nassim Nicholas Taleb in Antifragile
When you store your knowledge and skills as parametric curves (as all deep learning models do), the only way you can generalize is via interpolation on the curve. The problem is that interpolated points *correlate* with the truth but have no *causal* link to the truth. Hence hallucinations.
The fix is to start leveraging causal symbolic graphs as your representation substrate (e.g. computer programs of the kind we write as software engineers). The human-written software stack, with its extremely high degree of reliability despite its massive complexity, is proof of existence of exact truthiness propagation.
Our thesis at Ndea: simple theories should be discoverable using little data and little compute.
And everything humans have ever invented is a fairly simple composition of fairly simple theories.
Engineering is rarely the application of a well-understood theory. Most of the time it's a two-way dialogue, forcing theory to become more robust, more nuanced, or even to be discarded and rebuilt. But sometimes there's no theory at all, just a bag of poorly understood tricks guessed from past experiments.
Obviously there were always types of writing that were like code: recipes, checklists, procedures that had to be followed. What LLMs prove is that the codeness extended well beyond these.