Etienne

[Cerebras @cerebras]

https://t.co/FvKy1StDpQ

[Sriram Krishnan @sriramk]

It is clear open source models and harnesses are having a moment. There's a few factors at work

1/ It is now obvious that you can catch up to near-SOTA performance and do so with a clear training lineage. See:@thinkymachines Inkling launch today.

2/ There are several well-funded, talented teams building open weight models now in the US and abroad. Along with the explosing of other near SOTA models (Grok/Cursor, Muse Spark), it is clear we are going to have a diverse ecosystem of models atleast on coding and agentic use.

3/ Organizations are increasingly looking for control over how their data is used and are willing to trade off some access to frontier level tokens for this control. Organizations and countries are increasingly nervous about the frontier labs potentially competing with them down the road and don't want their data to enable a future competitor.

4/ Open source is a slider: you could bring your own open harness, your evals, your business context and are free to pick and choose your model of choice.

5/ Companies have now actively shifted from "how do we get our people to use tokens" to being uncomfortable with their token cost ballooning without a clear line to revenue.

6/ Geo-politically, countries will be weighing open weight models as a way to get frontier-level tokens inside controlled environments that may not be otherwise possible.

All of this leads to more choice for all of us !

[Hamel Husain @HamelHusain]

LLM bullshit knife, to cut through bs

RAG -> Provide relevant context
Agentic -> Function calls that work
CoT -> Prompt model to think/plan
FewShot -> Add examples
PromptEng -> Someone w/good written comm skills.
Prompt Optimizer -> For loop to find best examples.

[François Chollet @fchollet]

A mental model for working with coding agents is that they're blind squirrels running into a maze and bumping into walls. You must place the walls (verifiable constraints) strategically so that they end up in the general region you want them in.

[clem 🤗 @ClementDelangue]

Narrative violation: according to @Stanford research, local models can answer 71.3% of real-world chat and reasoning queries accurately, up from 23.2% in 2023. Obviously at a fraction of the cost and energy consumption of frontier APIs.

The obvious conclusion: you don't need a frontier model for most tasks. The future is multi-model: local, open-source, smaller and cheaper for the majority of workloads, frontier APIs when no other choices!

[nxthompson @nxthompson]

.@erikbryn is one of the smartest minds on economics and AI. There are so many open questions about how this technology is going to change the workforce, and it was a pleasure discussing them with him on the recent episode of my podcast.

You can watch our full chat below; here are a few takeaways. 1/ https://t.co/myxTAzyuLk

[Poe Zhao @poezhao0605]

China just released its first dedicated policy framework for AI agents.

Three agencies (CAC, NDRC, MIIT) jointly issued "Implementation Opinions on Standardized Application and Innovative Development of Intelligent Agents."
Key points:

- Defines AI agents as autonomous systems with perception, memory, decision-making, interaction, and execution capabilities.
- Lays out 19 specific application scenarios across scientific research, industrial development, consumer spending, public welfare, and governance.
- Establishes a "safety first, innovation second" principle. Development must be controllable and orderly.

This is a signal that Beijing sees AI agents as the next major frontier after foundation models.

[Jeremy Howard @jeremyphoward]

The only correct answer when a VC asks: "What's your moat?"

[Luis Garicano 🇪🇺🇺🇦 @lugaricano]

A brilliant, must read, post by @sebkrier extracting the consequences of two observations that appear correct to me:
(1) Diminishing returns to raw scaling: each increment in capability requires exponentially more compute, e.g. we already see it in OpenAI's projections of larger increases in inference costs than revenue;
(2) The key lesson of agentic tools like Claude code is the crucial role of scaffolding. The model is not necessarily smarter, but the harness around the agents doing separate tasks with different contexts is.
For Europe, this speaks (to me!) of the feasibility of the "smart second mover" strategy I have been arguing for in Silicon Continent. If the value is increasingly in how you orchestrate and deploy models, not in who trains the biggest one, you don't need to win the compute race to capture most of the gains.

[Etienne @vtecftwy]

Very insightful like always.
Practical way to make AI work for you on your own agenda and your own goals. And this build your own knowledge base, with time.

[Andrej Karpathy @karpathy]

Farzapedia, personal wikipedia of Farza, good example following my Wiki LLM tweet.

I really like this approach to personalization in a number of ways, compared to "status quo" of an AI that allegedly gets better the more you use it or something:

1. Explicit. The memory artifact is explicit and navigable (the wiki), you can see exactly what the AI does and does not know and you can inspect and manage this artifact, even if you don't do the direct text writing (the LLM does). The knowledge of you is not implicit and unknown, it's explicit and viewable.
2. Yours. Your data is yours, on your local computer, it's not in some particular AI provider's system without the ability to extract it. You're in control of your information.
3. File over app. The memory here is a simple collection of files in universal formats (images, markdown). This means the data is interoperable: you can use a very large collection of tools/CLIs or whatever you want over this information because it's just files. The agents can apply the entire Unix toolkit over them. They can natively read and understand them. Any kind of data can be imported into files as input, and any kind of interface can be used to view them as the output. E.g. you can use Obsidian to view them or vibe code something of your own. Search "File over app" for an article on this philosophy.
4. BYOAI. You can use whatever AI you want to "plug into" this information - Claude, Codex, OpenCode, whatever. You can even think about taking an open source AI and finetuning it on your wiki - in principle, this AI could "know" you in its weights, not just attend over your data.

So this approach to personalization puts *you* in full control. The data is yours. In Universal formats. Explicit and inspectable. Use whatever AI you want over it, keep the AI companies on their toes! :)

Certainly this is not the simplest way to get an AI to know you - it does require you to manage file directories and so on, but agents also make it quite simple and they can help you a lot. I imagine a number of products might come out to make this all easier, but imo "agent proficiency" is a CORE SKILL of the 21st century. These are extremely powerful tools - they speak English and they do all the computer stuff for you. Try this opportunity to play with one.

[Andrej Karpathy @karpathy]

LLM Knowledge Bases

Something I'm finding very useful recently: using LLMs to build personal knowledge bases for various topics of research interest. In this way, a large fraction of my recent token throughput is going less into manipulating code, and more into manipulating knowledge (stored as markdown and images). The latest LLMs are quite good at it. So:

Data ingest:
I index source documents (articles, papers, repos, datasets, images, etc.) into a raw/ directory, then I use an LLM to incrementally "compile" a wiki, which is just a collection of .md files in a directory structure. The wiki includes summaries of all the data in raw/, backlinks, and then it categorizes data into concepts, writes articles for them, and links them all. To convert web articles into .md files I like to use the Obsidian Web Clipper extension, and then I also use a hotkey to download all the related images to local so that my LLM can easily reference them.

IDE:
I use Obsidian as the IDE "frontend" where I can view the raw data, the the compiled wiki, and the derived visualizations. Important to note that the LLM writes and maintains all of the data of the wiki, I rarely touch it directly. I've played with a few Obsidian plugins to render and view data in other ways (e.g. Marp for slides).

Q&A:
Where things get interesting is that once your wiki is big enough (e.g. mine on some recent research is ~100 articles and ~400K words), you can ask your LLM agent all kinds of complex questions against the wiki, and it will go off, research the answers, etc. I thought I had to reach for fancy RAG, but the LLM has been pretty good about auto-maintaining index files and brief summaries of all the documents and it reads all the important related data fairly easily at this ~small scale.

Output:
Instead of getting answers in text/terminal, I like to have it render markdown files for me, or slide shows (Marp format), or matplotlib images, all of which I then view again in Obsidian. You can imagine many other visual output formats depending on the query. Often, I end up "filing" the outputs back into the wiki to enhance it for further queries. So my own explorations and queries always "add up" in the knowledge base.

Linting:
I've run some LLM "health checks" over the wiki to e.g. find inconsistent data, impute missing data (with web searchers), find interesting connections for new article candidates, etc., to incrementally clean up the wiki and enhance its overall data integrity. The LLMs are quite good at suggesting further questions to ask and look into.

Extra tools:
I find myself developing additional tools to process the data, e.g. I vibe coded a small and naive search engine over the wiki, which I both use directly (in a web ui), but more often I want to hand it off to an LLM via CLI as a tool for larger queries.

Further explorations:
As the repo grows, the natural desire is to also think about synthetic data generation + finetuning to have your LLM "know" the data in its weights instead of just context windows.

TLDR: raw data from a given number of sources is collected, then compiled by an LLM into a .md wiki, then operated on by various CLIs by the LLM to do Q&A and to incrementally enhance the wiki, and all of it viewable in Obsidian. You rarely ever write or edit the wiki manually, it's the domain of the LLM. I think there is room here for an incredible new product instead of a hacky collection of scripts.

[Andrej Karpathy @karpathy]

- Drafted a blog post
- Used an LLM to meticulously improve the argument over 4 hours.
- Wow, feeling great, it’s so convincing!
- Fun idea let’s ask it to argue the opposite.
- LLM demolishes the entire argument and convinces me that the opposite is in fact true.
- lol

The LLMs may elicit an opinion when asked but are extremely competent in arguing almost any direction. This is actually super useful as a tool for forming your own opinions, just make sure to ask different directions and be careful with the sycophancy.

[Sebastian Raschka @rasbt]

I think the same thing happened with the ctx package a few years back (also through PyPI).

Not perfect, but I think the best way to avoid is to:

1. Download a source code snapshot of the package (e.g., from github)

2. Audit it (traditionally manually, but now also LLM agent-aided)

3. Include the source code in your own library (since most open source licenses are compatible with each other anyways; just need to include the license and acknowledge that; it's a pattern I learned from scikit-learn maybe a decade ago)

[Etienne @vtecftwy]

Interesting thread for those who are interested by both AI and improv 😄

[Alexis Gallagher @alexisgallagher]

Using specifics is a trick I picked up over the years I was teaching improv. It turns out LLMs are natural improvisers. In improv you "yes and"—listen, then add. LLMs do this one token at a time. Autoregression IS improvisation. So the engine for conversation is already there.

[François Chollet @fchollet]

The best way to use AI is an interface to information that lets you deepen and improve your own knowledge and mental models. The worst way to use AI is as a crutch to outsource and forsake your own cognition

[MIT Technology Review @techreview]

Exclusive eBook: The great Al hype correction of 2025 https://t.co/rbjrytrOVW

[Michael Levin @drmichaellevin]

New preprint with @GiovanniPezzulo :
https://t.co/migcMiTGA2
Bootstrapping Life-Inspired Machine Intelligence: The Biological Route from Chemistry to Cognition and Creativity

"Achieving advanced machine intelligence remains a central challenge in AI research, often approached through scaling neural architectures and generative models. However, biological systems offer a broader repertoire of strategies for adaptive, goal-directed behavior - strategies that emerged long before nervous systems evolved. This paper advocates a genuinely life-inspired approach to machine intelligence, drawing on principles from biology that enable robustness, autonomy, and open-ended problem-solving across scales. We frame intelligence as flexible problem-solving, following William James, and develop the concept of cognitive light cones to characterize the continuum of intelligence in living systems and machines. We argue that biological evolution has discovered a scalable recipe for intelligence - and the progressive expansion of organisms' "cognitive light cone", predictive and control capacities. To explain how this is possible, we distill five design principles - multiscale autonomy, growth through self-assemblage of active components, continuous reconstruction of capabilities, exploitation of physical and embodied constraints, and pervasive signaling enabling self-organization and top-down control from goals - that underpin life's ability to navigate creatively diverse problem spaces. We discuss how these principles contrast with current AI paradigms and outline pathways for integrating them into future autonomous, embodied, and resilient artificial systems."

[Erik Brynjolfsson @erikbryn]

Last August, @BharatKChandar, @RuyuChen and I posted our "Canaries in the Coal Mine?" paper.

Since there has been an explosion of research seeking to assess the relationship between AI and employment, especially for early career workers in highly exposed occupations, where we found some of the most striking effects.

While several studies have confirmed our initial findings, others have raised questions in two areas in particular:

1. Can the sharp increase in interest rates in 2022 explain the employment effects we observed better than AI-exposure?
2. Is the overall timing of the employment effects consistent with AI-exposure?

We've written a note to address these questions.

In brief, we find that:
1. While interest rates affect overall employment, existing evidence does NOT suggest they are a good explanation for the disproportionate decline in entry-level hiring in AI-exposed occupations.

2. We do find suggestive evidence that when you include the broadest set of controls (firm-time fixed-effects), the timing of the employment decline in AI-exposed occupations becomes significant only in 2024; some of the earlier declines are likely due to a combination of factors, not just AI.

A link to our note, "Canaries, Interest Rates and Timing", with more details, is in the next post.

[elvis @omarsar0]

New research from Meta and collaborators.

This is a good paper showing what's possible with proper world models.

World models need actions to predict consequences. The default approach today requires labeled action data, which is expensive to obtain and limited to narrow domains like video games or robotic manipulation.

But the vast majority of video data online has no action labels at all.

This new research tackles learning latent action world models directly from in-the-wild videos, expanding beyond the controlled settings of previous work to capture the full diversity of real-world actions.

The challenge is significant. In-the-wild videos contain actions far beyond simple navigation or manipulation: people entering frames, objects appearing and disappearing, dancers moving, fingers forming guitar chords. There's also no consistent embodiment across videos, unlike robotics datasets, where the same arm appears throughout.

So how do the authors address this?

Continuous but constrained latent actions, using sparse or noisy regularization, effectively capture this action complexity. Discrete quantization, the common approach in prior work, struggles to adapt. Without a shared embodiment, the model learns spatially-localized, camera-relative transformations.

The results demonstrate genuine action transfer.

Motion from a walking person can be applied to a flying ball. Actions like "someone entering the frame" transfer across completely different videos.

By training a small controller to map known actions to latent ones, the world model trained purely on natural videos can solve robotic manipulation and navigation tasks with performance close to models trained on domain-specific, action-labeled data.

Latent action spaces learned from unlabeled internet videos can serve as a universal interface for planning, removing the bottleneck of action annotation.

Paper: https://t.co/BL6mpuLZGD

Learn to build effective AI agents in our academy: https://t.co/JBU5beHQNs

[Yann LeCun @ylecun]

I think you missed the main ideas.
- The basic premise of JEPA is that training by reconstructio/prediction in input space is evil (or counterproductive). The details are almost always unpredictable. Hence prediction must take place in representation space, where unpredictable details are eliminated.
- The main issue with JEPA is how to prevent collapse (in the absence of reconstruction loss). There are two classes of methods:
(1) EMA: Using weights in target encoder that are an exponential moving average (EMA) of the weights in other encoder (I-JEPA, V-JEPA, DINO, BYOL).
(2) Infomax: Using a regularizer that attempts to maximize the information content of the representation (e.g. over a batch). There are two sets of methods for that:
(2a) sample-contrastive methods: that want to make each representation vector different from the others (Siamese nets, DrLIM, SimCLR, etc). They tend to not work well in high dimension, to require large batches, and hard negative mining
(2b) dimension-contrastive methods: that want to make each variable independent from the others (Barlow Twins, VICReg, SIGReg/ LeJEPA, MMCR, MCR2....)
Bottom line:
A. SSL by reconstruction/prediction doesn't work for high-dim, continuous, noisy data
B. EMA sucks: no loss function being minimized, requirement for weightmsharing....
C. Sample-contrastive informax doesn't scale to high dimension
D. My money is on dimension-contrastive methods like SIGReg/LeJEPA