Sewanee CSci

The Computer Science program at Sewanee: The University of the South

[Sewanee CSci @SewaneeCS]

He'll get no argument from me!

[N. Bourbaki, Professor of Logic at Science U @d08890]

this is, however, all a distraction from what is objectively the most beautiful university campus in the united states, sewanee

[Sewanee CSci @SewaneeCS]

Enjoyed the good food and conversation with some of our soon-to-be CS graduates and friends at the SCS Senior Send-off last Thursday. At least 7 of our 14 seniors showed - see you all again on Graduation Sunday!

[PoIiMath @politicalmath]

In the Old Days (12 months ago), you would put together an interface for adding an item to your cart. And every you needed to add an "add item to cart" functionality, they would align with that

AI code doesn't do that. It doesn't. It writes the "add item" functionality fresh each time and slightly different.

But the reason we abstracted this functionality wasn't simply to save time. It was because we needed consistency across our code base. It was so that, when we made a mistake or introduced a vulnerability, we could fix it across the platform easily and quickly.

AI is going to break the patterns of good software. People who care about building good things will resist it. My optimism tells me that they will survive. My cynicism tells me that they will be destroyed by shitty organizations that move fast with shitty products that seem to work at first glance but are tremendously unstable.

[Lior Alexander @LiorOnAI]

Yann just bet a billion dollars that the entire industry is building on the wrong foundation.

Large language models predict the next word. They're trained on text, so they understand language. But the real world isn't made of words.

It's made of continuous sensor data: camera feeds, touch, sound. And most of that data is unpredictable.

You can't predict every pixel in a video the way you predict the next token in a sentence. Generative models fail here because they try to predict everything, including noise.

AMI Labs is building world models using JEPA (a method LeCun proposed in 2022 that learns abstract representations of reality and predicts in that compressed space, not in raw pixels).

Action-conditioned versions let AI simulate the consequences of actions before taking them. That's not generation. That's understanding.

This unlocks AI that can operate in the physical world without hallucinating:

1. Robotics that plans multi-step actions
2. Healthcare devices where errors kill patients
3. Industrial process control under safety constraints
4. Wearables that adapt to real-time sensor input

If JEPA works at scale, the next wave of AI companies won't fine-tune LLMs. They'll train world models on sensor data. LeCun's CEO already predicts every startup will rebrand as a "world model company" within six months.

The architecture war is starting.

[Jay Alammar @JayAlammar]

The Illustrated DeepSeek-R1

Spent the weekend reading the paper and sorting through the intuitions. Here's a visual guide and the main intuitions to understand the model and the process that created it.

Link in the first reply. All feedback welcome.

[Sewanee CSci @SewaneeCS]

Happy first week of classes to all Sewanee students, especially our CS majors and soon-to-be majors!

[Jonathan Roberts @JRobertsAI]

Is computer vision “solved”?

Not yet

Current models score 0% on ZeroBench

🧵1/6

[Jeff Blehar is *BOX OFFICE POISON* @EsotericCD]

A classic of the Venn Diagram genre:

[Andrej Karpathy @karpathy]

DeepSeek (Chinese AI co) making it look easy today with an open weights release of a frontier-grade LLM trained on a joke of a budget (2048 GPUs for 2 months, $6M).

For reference, this level of capability is supposed to require clusters of closer to 16K GPUs, the ones being brought up today are more around 100K GPUs. E.g. Llama 3 405B used 30.8M GPU-hours, while DeepSeek-V3 looks to be a stronger model at only 2.8M GPU-hours (~11X less compute). If the model also passes vibe checks (e.g. LLM arena rankings are ongoing, my few quick tests went well so far) it will be a highly impressive display of research and engineering under resource constraints.

Does this mean you don't need large GPU clusters for frontier LLMs? No but you have to ensure that you're not wasteful with what you have, and this looks like a nice demonstration that there's still a lot to get through with both data and algorithms.

Very nice & detailed tech report too, reading through.

[Andrej Karpathy @karpathy]

New 3h31m video on YouTube:
"Deep Dive into LLMs like ChatGPT"

This is a general audience deep dive into the Large Language Model (LLM) AI technology that powers ChatGPT and related products. It is covers the full training stack of how the models are developed, along with mental models of how to think about their "psychology", and how to get the best use them in practical applications.

We cover all the major stages:
1. pretraining: data, tokenization, Transformer neural network I/O and internals, inference, GPT-2 training example, Llama 3.1 base inference examples
2. supervised finetuning: conversations data, "LLM Psychology": hallucinations, tool use, knowledge/working memory, knowledge of self, models need tokens to think, spelling, jagged intelligence
3. reinforcement learning: practice makes perfect, DeepSeek-R1, AlphaGo, RLHF.

I designed this video for the "general audience" track of my videos, which I believe are accessible to most people, even without technical background. It should give you an intuitive understanding of the full training pipeline of LLMs like ChatGPT, with many examples along the way, and maybe some ways of thinking around current capabilities, where we are, and what's coming.

(Also, I have one "Intro to LLMs" video already from ~year ago, but that is just a re-recording of a random talk, so I wanted to loop around and do a lot more comprehensive version of this topic. They can still be combined, as the talk goes a lot deeper into other topics, e.g. LLM OS and LLM Security)

Hope it's fun & useful!
https://t.co/75mXcUBI8L

[Peyman Milanfar @docmilanfar]

Michael Jordan gave a short, excellent, and provocative talk recently in Paris - here's a few key ideas

- It's all just machine learning (ML) - the AI moniker is hype

- The late Dave Rumelhart should've received a Nobel prize for his early ideas on making backprop work

1/n

[Sewanee CSci @SewaneeCS]

Less than 12 hours to go until the first classes of the new term! Sounds like a good time to re-develop focused attention, something social media tends to degrade.

[MindBranches @MindBranches]

Book Summary: “Thinking in Systems”

[Retune the Sky @retunethesky]

Take note of the date 👇

[Andres Guadamuz @technollama]

The second draft of the General-Purpose AI Code of Practice has now been published. Not many surprises, but we're getting more detail of the information to be disclosed in transparency reports, as well as some more detail on copyright compliance.
https://t.co/8J825ZcWRK

[Dhruv Batra @DhruvBatra_]

Brilliant talk by @ilyasut, but he's wrong on one point.

We are NOT running out of data. We are running out human-written text.

We have more videos than we know what to do with. We just haven't solved pre-training in vision.

Just go out and sense the world. Data is easy.

[Tom Yeh @ProfTomYeh]

I just edited my lecture - Beginner's Guide to RAG - and posted to YouTube. I gave this lecture last May. Do you like it? If so, I will edit and post more lectures like this, whenever I have some free time. Link is in the comment below. 👇

[Shashank | 𝔽rAI @0xshai]

Reasoning isn't just pattern matching or step-by-step logic. Real intelligence merges both: continuous abstraction (vector spaces, deep learning) + discrete search (symbolic models). The key isn't choosing sides but integrating both into a unified system

[Sewanee CSci @SewaneeCS]

5 hours until go-time! Who's with me?

[Bertrand Meyer @Bertrand_Meyer]

On 9 Jan, right after I learned of Niklaus Wirth's passing away, I wrote an extensive article about him on the ACM blog. I have extended it a bit since. It is not a standard eulogy faking full agreement, but more like a candid posthumous discussion. See https://t.co/Oz4XYpjliH