Erick Asto Oblitas

@zerazobz

Skilled .Net| Python stack | Melomaniac | F1 fan. Having two lives. 😎 By day, a Software Engineer at @Globant and by night, a relentless pursuer of my dreams.

Miraflores, Arequipa

Joined November 2009

793 Following

170 Followers

6.8K Posts

zerazobz retweeted

How To AI

@HowToAI_

about 2 months ago

Yann LeCun was right the entire time. And generative AI might be a dead end. For the last three years, the entire industry has been obsessed with building bigger LLMs. Trillions of parameters. Billions in compute. The theory was simple: if you make the model big enough, it will eventually understand how the world works. Yann LeCun said that was stupid. He argued that generative AI is fundamentally inefficient. When an AI predicts the next word, or generates the next pixel, it wastes massive amounts of compute on surface-level details. It memorizes patterns instead of learning the actual physics of reality. He proposed a different path: JEPA (Joint-Embedding Predictive Architecture). Instead of forcing the AI to paint the world pixel by pixel, JEPA forces it to predict abstract concepts. It predicts what happens next in a compressed "thought space." But for years, JEPA had a fatal flaw. It suffered from "representation collapse." Because the AI was allowed to simplify reality, it would cheat. It would simplify everything so much that a dog, a car, and a human all looked identical. It learned nothing. To fix it, engineers had to use insanely complex hacks, frozen encoders, and massive compute overheads. Until today. Researchers just dropped a paper called "LeWorldModel" (LeWM). They completely solved the collapse problem. They replaced the complex engineering hacks with a single, elegant mathematical regularizer. It forces the AI's internal "thoughts" into a perfect Gaussian distribution. The AI can no longer cheat. It is forced to understand the physical structure of reality to make its predictions. The results completely rewrite the economics of AI. LeWM didn't need a massive, centralized supercomputer. It has just 15 million parameters. It trains on a single, standard GPU in a few hours. Yet it plans 48x faster than massive foundation world models. It intrinsically understands physics. It instantly detects impossible events. We spent billions trying to force massive server farms to memorize the internet. Now, a tiny model running locally on a single graphics card is actually learning how the real world works.

HowToAI_'s tweet photo. Yann LeCun was right the entire time. And generative AI might be a dead end.

For the last three years, the entire industry has been obsessed with building bigger LLMs. Trillions of parameters. Billions in compute.

The theory was simple: if you make the model big enough, it will eventually understand how the world works.

Yann LeCun said that was stupid.

He argued that generative AI is fundamentally inefficient.

When an AI predicts the next word, or generates the next pixel, it wastes massive amounts of compute on surface-level details.

It memorizes patterns instead of learning the actual physics of reality.

He proposed a different path: JEPA (Joint-Embedding Predictive Architecture).

Instead of forcing the AI to paint the world pixel by pixel, JEPA forces it to predict abstract concepts. It predicts what happens next in a compressed "thought space."

But for years, JEPA had a fatal flaw.

It suffered from "representation collapse."

Because the AI was allowed to simplify reality, it would cheat. It would simplify everything so much that a dog, a car, and a human all looked identical.

It learned nothing.

To fix it, engineers had to use insanely complex hacks, frozen encoders, and massive compute overheads.

Until today.

Researchers just dropped a paper called "LeWorldModel" (LeWM).

They completely solved the collapse problem.

They replaced the complex engineering hacks with a single, elegant mathematical regularizer.

It forces the AI's internal "thoughts" into a perfect Gaussian distribution.

The AI can no longer cheat. It is forced to understand the physical structure of reality to make its predictions.

The results completely rewrite the economics of AI.

LeWM didn't need a massive, centralized supercomputer.

It has just 15 million parameters.

It trains on a single, standard GPU in a few hours.

Yet it plans 48x faster than massive foundation world models. It intrinsically understands physics. It instantly detects impossible events.

We spent billions trying to force massive server farms to memorize the internet.

Now, a tiny model running locally on a single graphics card is actually learning how the real world works.

433

12K

zerazobz retweeted

How To AI

@HowToAI_

2 months ago

RAG is broken and nobody's talking about it. Stanford researchers exposed the fatal flaw killing every "AI that reads your docs" product in existence. It’s called "Semantic Collapse," and it happens the second your knowledge base hits critical mass. If you've noticed your AI getting "dumber" as you add more data, this is exactly why. Right now, companies are dumping thousands of documents into their AI, thinking it’s getting smarter. When you add a document to RAG, it converts it into a high-dimensional vector. Under 10,000 documents, this works perfectly. Similar concepts cluster together. But past 10,000 documents, the space fills up. The clusters overlap. The distances compress. Everything starts to look "relevant." It is a mathematical law called the Curse of Dimensionality. In a 1000-dimensional space, 99.9% of your data lives on the outer edge. All points become equidistant from each other. That perfect, relevant document you are looking for now has the exact same mathematical similarity as 50 completely irrelevant ones. The Stanford findings are brutal: At 50,000 documents, precision drops by 87%. Semantic search actually becomes worse than old-school keyword search. Adding more context doesn’t fix the AI. It makes the hallucinations worse. Your "nearest neighbor" search isn't finding the best answer anymore. It's finding everyone. We thought RAG solved hallucinations. It didn't. It just hid them behind math.

HowToAI_'s tweet photo. RAG is broken and nobody's talking about it.

Stanford researchers exposed the fatal flaw killing every "AI that reads your docs" product in existence.

It’s called "Semantic Collapse," and it happens the second your knowledge base hits critical mass. If you've noticed your AI getting "dumber" as you add more data, this is exactly why.

Right now, companies are dumping thousands of documents into their AI, thinking it’s getting smarter.

When you add a document to RAG, it converts it into a high-dimensional vector.

Under 10,000 documents, this works perfectly. Similar concepts cluster together.

But past 10,000 documents, the space fills up. The clusters overlap. The distances compress.

Everything starts to look "relevant."

It is a mathematical law called the Curse of Dimensionality. In a 1000-dimensional space, 99.9% of your data lives on the outer edge. All points become equidistant from each other.

That perfect, relevant document you are looking for now has the exact same mathematical similarity as 50 completely irrelevant ones.

The Stanford findings are brutal:

At 50,000 documents, precision drops by 87%. Semantic search actually becomes worse than old-school keyword search.

Adding more context doesn’t fix the AI. It makes the hallucinations worse.

Your "nearest neighbor" search isn't finding the best answer anymore. It's finding everyone.

We thought RAG solved hallucinations.

It didn't. It just hid them behind math.

202

595

318K

Erick Asto Oblitas

@zerazobz

Who to follow

Last Seen Users on Sotwe

Trends for you

Most Popular Users