Jawher Jabri 🍉

AI Research Engineer

[Andrej Karpathy @karpathy]

It is hard to communicate how much programming has changed due to AI in the last 2 months: not gradually and over time in the "progress as usual" way, but specifically this last December. There are a number of asterisks but imo coding agents basically didn’t work before December and basically work since - the models have significantly higher quality, long-term coherence and tenacity and they can power through large and long tasks, well past enough that it is extremely disruptive to the default programming workflow.

Just to give an example, over the weekend I was building a local video analysis dashboard for the cameras of my home so I wrote: “Here is the local IP and username/password of my DGX Spark. Log in, set up ssh keys, set up vLLM, download and bench Qwen3-VL, set up a server endpoint to inference videos, a basic web ui dashboard, test everything, set it up with systemd, record memory notes for yourself and write up a markdown report for me”. The agent went off for ~30 minutes, ran into multiple issues, researched solutions online, resolved them one by one, wrote the code, tested it, debugged it, set up the services, and came back with the report and it was just done. I didn’t touch anything. All of this could easily have been a weekend project just 3 months ago but today it’s something you kick off and forget about for 30 minutes.

As a result, programming is becoming unrecognizable. You’re not typing computer code into an editor like the way things were since computers were invented, that era is over. You're spinning up AI agents, giving them tasks *in English* and managing and reviewing their work in parallel. The biggest prize is in figuring out how you can keep ascending the layers of abstraction to set up long-running orchestrator Claws with all of the right tools, memory and instructions that productively manage multiple parallel Code instances for you. The leverage achievable via top tier "agentic engineering" feels very high right now.

It’s not perfect, it needs high-level direction, judgement, taste, oversight, iteration and hints and ideas. It works a lot better in some scenarios than others (e.g. especially for tasks that are well-specified and where you can verify/test functionality). The key is to build intuition to decompose the task just right to hand off the parts that work and help out around the edges. But imo, this is nowhere near "business as usual" time in software.

[bycloud @bycloudai]

OpenAI just dropped a paper on why LM hallucinates

basically:
- statistical limits of pretraining on long-tailed data
- representation/model limits
- data noise and shifts
- misaligned evals that reward guessing over uncertainty

[Jawher Jabri 🍉 @geawher]

@HoopMixOnly .

[DeepPCB @DeepPCB]

👋 Say hello to the DeepPCB API
🤩 You can now integrate PCB Place & Route into your preferred tool–no platform switching, full control!

Read the blog 👉 https://t.co/mfTvbZyyP4

[DeepPCB @DeepPCB]

We’ve talked routing—now let’s dive into placement! Discover how DeepPCB Placement elevates your PCB design process 🚀

Real-world use-cases showing why placement + routing = winning combo🏅

👉 https://t.co/zcsuPqmzqJ

[DeepSeek @deepseek_ai]

🚀 DeepSeek-R1 is here!

⚡ Performance on par with OpenAI-o1
📖 Fully open-source model & technical report
🏆 MIT licensed: Distill & commercialize freely!

🌐 Website & API are live now! Try DeepThink at https://t.co/v1TFy7LHNy today!

🐋 1/n

[The Tennis Letter @TheTennisLetter]

Federer, Djokovic, & Murray to Rafa Nadal:

Andy: “Your passion & intensity was something all tennis players aspire to”

Novak: “I’m honored & thrilled to be called your rival”

Roger: “It was such a privilege to play with you, & also against you” 🥹

[The Tennis Letter @TheTennisLetter]

Rafa Nadal during his retirement speech:

“The titles, numbers, they’re there. People probably know that. The way I’d like to be remembered more is like a good person from a small village in Mallorca.” 🥹

[Rohan Paul @rohanpaul_ai]

"Attention Is All You Need" paper was truly a landmark paper.

However, the original "vanilla" transformers are seldom used now.

The huge key upgrade is the use of RoPE, or Rotary Positional Embeddings.

**Vanilla Decoder**

- Input tokens -> Embeddings -> Embeddings + Positional Encoding -> Decoder Blocks

**RoPE Decoder**

- Input tokens -> Embeddings -> Decoder Blocks

**Rotary Positional Embeddings**

RoPE are used in attention blocks, which need to know token positions.

Attention blocks combine information from a lot of tokens and need to know their relative positions

For example, consider this sentence "It's a big thrill to climb a big mountain."

"mountain" should focus more on the nearby "big."

RoPE applies a rotational matrix to queries and keys, not values. If "mountain" is the 9th word, it rotates fully, while earlier words rotate less, aligning "mountain" more with the second "big."

This approach is efficient as it applies positional embeddings only where needed and keeps token magnitudes unchanged.

RoPE scales well to longer contexts, allowing models to be pre-trained on 4k contexts and fine-tuned for up to 4M by adjusting rotation speed.

[Andrej Karpathy @karpathy]

NotebookLM is quite powerful and worth playing with
https://t.co/EMHIjc15iU

It is a bit of a re-imagination of the UIUX of working with LLMs organized around a collection of sources you upload and then refer to with queries, seeing results alongside and with citations.

But the current most new/impressive feature (that is surprisingly hidden almost as an afterthought) is the ability to generate a 2-person podcast episode based on any content you upload. For example someone took my "bitcoin from scratch" post from a long time ago:
https://t.co/7ajZNZ0BGi
and converted it to podcast, quite impressive:
https://t.co/ZZn0LJgsnu

You can podcastify *anything*. I give it train_gpt2.c (C code that trains GPT-2):
https://t.co/gDrAqix4Iv
and made a podcast about that:
https://t.co/bgcwmQr5d7
I don't know if I'd exactly agree with the framing of the conversation and the emphasis or the descriptions of layernorm and matmul etc but there's hints of greatness here and in any case it's highly entertaining.

Imo LLM capability (IQ, but also memory (context length), multimodal, etc.) is getting way ahead of the UIUX of packaging it into products. Think Code Interpreter, Claude Artifacts, Cursor/Replit, NotebookLM, etc. I expect (and look forward to) a lot more and different paradigms of interaction than just chat.

That's what I think is ultimately so compelling about the 2-person podcast format as a UIUX exploration. It lifts two major "barriers to enjoyment" of LLMs. 1 Chat is hard. You don't know what to say or ask. In the 2-person podcast format, the question asking is also delegated to an AI so you get a lot more chill experience instead of being a synchronous constraint in the generating process. 2 Reading is hard and it's much easier to just lean back and listen.

[Philipp Schmid @_philschmid]

Llama can now see and run on your phone!👀🖼️ Llama 3.2 released with Multimodal support in Llama Vision and tiny llamas for on-device usage. 10 new llama released by @AIatMeta from 1B text only to 90B Multimodal (text+image) 🚀 But with EU restrictions 🇪🇺

TL;DR:
🖼️ Llama 3.2 Vision: multimodal (text+image to text) models in 11B and 90B sizes based on Llama 3.1 text models trained on 6 Billion image text pairs
📱 Llama 3.2 Edge: 1B and 3B multilingual text-only models for efficient local deployment.
⚡ All Llama 3.2 have 128k token context length
🤏🏻 Used Knowledge distillation and pruning from 8B/70B to train 1B/3B models
🔒 Llama Guard 3.2: 2 new enhanced safeguard model with vision support
📊 Llama 3.2 3B matches Llama 3.1 8B on IFEval, indicating strong use cases for on-device RAG or Agents
🇪🇺 Multimodal model is restricted to individuals and companies based in the European Union.
🤗 Available and integrated into @huggingface Ecosystem from Hub to Transformers and TGI.

Models: https://t.co/LKrKKzCHSf
Official Blog: https://t.co/DQTXrDgGIl

P.S. We are working on a comprehensive blog post with all you need to know and how to train and deploy it. Will come later today. 🚀

[AI at Meta @AIatMeta]

📣 Introducing Llama 3.2: Lightweight models for edge devices, vision models and more!

What’s new?
• Llama 3.2 1B & 3B models deliver state-of-the-art capabilities for their class for several on-device use cases — with support for @Arm, @MediaTek & @Qualcomm on day one.
• Llama 3.2 11B & 90B vision models deliver performance competitive with leading closed models — and can be used as drop-in replacements for Llama 3.1 8B & 70B.
• New Llama Guard models to support multimodal use cases and edge deployments.
• The first official distro of Llama Stack simplifies and supercharges the way developers & enterprises can build around Llama to support agentic applications and more.

Details in the full announcement ➡️ https://t.co/1bnEeLY9qf
Download Llama 3.2 models ➡️ https://t.co/DZoTQvESbG

These models are available to download now directly from Meta and @HuggingFace — and will be available across offerings from 25+ partners that are rolling out starting today, including @accenture, @awscloud, @AMD, @azure, @Databricks, @Dell, @Deloitte, @FireworksAI_HQ, @GoogleCloud, @GroqInc, @IBMwatsonx, @Infosys, @Intel, @kaggle, @NVIDIA, @OracleCloud, @PwC, @scale_AI, @snowflakeDB, @togethercompute and more.

With Llama 3.2 we’re making it possible to run Llama in even more places, with even more flexible capabilities. We’ve said it before and we’ll say it again: open source AI is how we ensure that these innovations reflect the global community they’re built for and benefit everyone. We’re continuing our drive to make open source the standard with Llama 3.2.

[Wei Ping @_weiping]

Introducing NVLM 1.0, a family of frontier-class multimodal LLMs that achieve state-of-the-art results on vision-language tasks, rivaling the leading proprietary models (e.g., GPT-4o) and open-access models (e.g., InternVL 2).
Remarkably, NVLM 1.0 shows improved text-only performance over its LLM backbone after multimodal training!
We are working towards releasing the model weights very soon 🤗 and will open-source the training code for the community.

For further details, please visit our project website:
https://t.co/1ZPGp3zFes

[Andrej Karpathy @karpathy]

It's a bit sad and confusing that LLMs ("Large Language Models") have little to do with language; It's just historical. They are highly general purpose technology for statistical modeling of token streams. A better name would be Autoregressive Transformers or something.

They don't care if the tokens happen to represent little text chunks. It could just as well be little image patches, audio chunks, action choices, molecules, or whatever. If you can reduce your problem to that of modeling token streams (for any arbitrary vocabulary of some set of discrete tokens), you can "throw an LLM at it".

Actually, as the LLM stack becomes more and more mature, we may see a convergence of a large number of problems into this modeling paradigm. That is, the problem is fixed at that of "next token prediction" with an LLM, it's just the usage/meaning of the tokens that changes per domain.

If that is the case, it's also possible that deep learning frameworks (e.g. PyTorch and friends) are way too general for what most problems want to look like over time. What's up with thousands of ops and layers that you can reconfigure arbitrarily if 80% of problems just want to use an LLM?

I don't think this is true but I think it's half true.

[Jawher Jabri 🍉 @geawher]

@MatthewBerman We are the product 🤷🏻

[Andrej Karpathy @karpathy]

Are we able to agree on what we mean by "AGI". I've been using this definition from OpenAI which I thought was relatively standard and ok:

https://t.co/lMHgmMaLgG

AGI: "a highly autonomous system that outperforms humans at most economically valuable work"
For "most economically valuable work" I like to reference the index of all occupations from U.S. Bureau of Labor Statistics:

https://t.co/tGVU57v7WG

Two common caveats:

1) In practice most people currently deviate from the above definition to only mean digital work (a relatively major concession looking at the list).

2) The definition above only considers the *existence* of such a system not its full deployment across all of the industry.

Some people say GPT-4 is already AGI, which per above definition would be clearly not true. LLMs are useful tools for most of these jobs but you clearly couldn't hire them to autonomously perform them in full and autonomously at human+ capability.

Last note some people say the goalposts keep moving, which I mostly disagree with. I think the definition above makes sense, it has been stable, and has clearly not been reached.

[Jawher Jabri 🍉 @geawher]

“If you're not paying for the product, you are the product” -- Andrew Lewis

[Matthew Berman @MatthewBerman]

"Meta has acknowledged that all text and photos that adult Facebook and Instagram users have publicly published since 2007 have been fed into its artificial intelligence models."

...wait...what? Didn't Mark say they would never use Facebook's data to train models?
https://t.co/XMOVisuG8T

[Jawher Jabri 🍉 @geawher]

Proud to have contributed to the development of DeepPCB! It's amazing to see how my work fits into this larger project, and I'm excited about the impact it will have. Looking forward to what comes next!

[DeepPCB @DeepPCB]

👋 Hello World! 🌎
Introducing DeepPCB Pro, our AI-powered design tool for industry-grade PCBs! Fast, precise, and compute-scalable with @googlecloud ☁️ Check it out at https://t.co/fa2Comabpf 👀

[Jim Fan @DrJimFan]

This may be the most important figure in LLM research since the OG Chinchilla scaling law in 2022. The key insight is 2 curves working in tandem. Not one.

People have been predicting a stagnation in LLM capability by extrapolating the training scaling law, yet they didn't foresee that inference scaling is what truly beats the diminishing return.

I posted in February that no self-improving LLM algorithm was able to gain much beyond 3 rounds. No one was able to reproduce AlphaGo's success in the realm of LLM, where more compute would carry the capability envelope beyond human level.

Well, we have turned the page.

[Noam Brown @polynoamial]

Today, I’m excited to share with you all the fruit of our effort at @OpenAI to create AI models capable of truly general reasoning: OpenAI's new o1 model series! (aka 🍓) Let me explain 🧵 1/