ttrdrd

[ttrdrd @traffic_arg]

@PFAOficial @MinSeguridad_Ar @PDI_CHILE @LANACION @clarincom @INTERPOL_HQ

[ttrdrd @traffic_arg]

Human trafficking in Argentina
@PFAOficial @MinSeguridad_Ar @PDI_CHILE
su urgente atención sobre esto. Se exponen presuntos delitos de trata de menores y fraude documentario (ocultamiento de identidad de un menor argentino para presunta venta de pasaportes).

[alphaXiv @askalphaxiv]

“There Will Be a Scientific Theory of Deep Learning”

This paper argues that a real scientific theory of deep learning is beginning to emerge.

Not a theory that tracks every neuron individually, but a physics-like theory of learning itself. One that aims to characterize how training dynamics, representations, weights, and performance evolve.

It reframes deep learning theory from abstract guarantees and trial-and-error practice toward falsifiable, quantitative predictions about real systems.

[Andrej Karpathy @karpathy]

https://t.co/Lb6T42n5jl

[BURKOV @burkov]

This paper really is groundbreaking. It solves a long-standing embarrassment in machine learning: despite all the hype around deep learning, traditional tree-based methods (XGBoost, CatBoost, random forests, etc) have dominated tabular data—the most common data format in real-world applications—for two decades. Deep learning conquered images, text, and games, but spreadsheets remained stubbornly resistant.

This paper's (published in Nature by the way) main contribution is a foundation model that finally beats tree-based methods convincingly on small-to-medium datasets, and does so very fast. TabPFN in 2.8 seconds outperforms CatBoost tuned for 4 hours—a 5,000× speedup. That's not incremental; it's a different regime entirely.

The training approach is also fundamentally different. GPT trains on internet text; CLIP trains on image-caption pairs. TabPFN trains on entirely synthetic data—over 100 million artificial datasets generated from causal graphs.

TabPFN generates training data by randomly constructing directed acyclic graphs where each edge applies a random transformation (using neural networks, decision trees, discretization, or noise), then pushes random noise through the root nodes and lets it propagate through the graph—the intermediate values at various nodes become features, one becomes the target, and post-processing adds realistic messiness like missing values and outliers. By training on millions of these synthetic datasets with very different structures, the model learns general prediction strategies without ever seeing real data.

The inference mechanism is also unusual. Rather than finetuning or prompting, TabPFN performs both "training" and prediction in a single forward pass. You feed it your labeled training data and unlabeled test points together, and it outputs predictions immediately. There's no gradient descent at inference time—the model has learned how to learn from examples during pretraining.

The architecture respects tabular structure with two-way attention (across features within a row, then across samples within a column), unlike standard transformers that treat everything as a flat sequence.

So, the transformer has basically learned to do supervised learning.

Talk to the paper on ChapterPal: https://t.co/hmWIA1dYji

Download the PDF: https://t.co/uxElyS85ge

[Andrej Karpathy @karpathy]

In era of pretraining, what mattered was internet text. You'd primarily want a large, diverse, high quality collection of internet documents to learn from.

In era of supervised finetuning, it was conversations. Contract workers are hired to create answers for questions, a bit like what you'd see on Stack Overflow / Quora, or etc., but geared towards LLM use cases.

Neither of the two above are going away (imo), but in this era of reinforcement learning, it is now environments. Unlike the above, they give the LLM an opportunity to actually interact - take actions, see outcomes, etc. This means you can hope to do a lot better than statistical expert imitation. And they can be used both for model training and evaluation. But just like before, the core problem now is needing a large, diverse, high quality set of environments, as exercises for the LLM to practice against.

In some ways, I'm reminded of OpenAI's very first project (gym), which was exactly a framework hoping to build a large collection of environments in the same schema, but this was way before LLMs. So the environments were simple academic control tasks of the time, like cartpole, ATARI, etc. The @PrimeIntellect environments hub (and the `verifiers` repo on GitHub) builds the modernized version specifically targeting LLMs, and it's a great effort/idea. I pitched that someone build something like it earlier this year:
https://t.co/ANHhasxzD8
Environments have the property that once the skeleton of the framework is in place, in principle the community / industry can parallelize across many different domains, which is exciting.

Final thought - personally and long-term, I am bullish on environments and agentic interactions but I am bearish on reinforcement learning specifically. I think that reward functions are super sus, and I think humans don't use RL to learn (maybe they do for some motor tasks etc, but not intellectual problem solving tasks). Humans use different learning paradigms that are significantly more powerful and sample efficient and that haven't been properly invented and scaled yet, though early sketches and ideas exist (as just one example, the idea of "system prompt learning", moving the update to tokens/contexts not weights and optionally distilling to weights as a separate process a bit like sleep does).

[signüll @signulll]

google’s core problem is that it was built to organize a web that no longer exists. the open web has been replaced by walled gardens, discord servers, newsletters, private forums, & algorithmic feeds that are never exposed to search. worse, the visible parts of the web that google still indexes have been overrun by seo-optimized sludge, ai-generated spam, & paywalls.

their dna is fundamentally extractive. they never built a creator ecosystem because their whole game was to scrape, index, & serve ads against other people’s content.

the entire ecosystem slowly but surely shifted drastically—with llm’s anyone can organize anything so the mission breaks down.

[BURKOV @burkov]

I regularly receive direct messages from people wondering what's wrong with the so-called agents and agentic frameworks. Here's my answer.

The main topic of my PhD was agents and multi-agent systems. What they currently call "agents" (LLMs that were instructed to do something) aren't agents. LLMs hallucinate too much to be trusted with any important task, even if you have 100 "agents" to do the job and 100 to validate it. And even in this case, you can simply use an LLM directly, without using any "agentic" framework, and you will get the same deplorable result.

LLMs are only good under two conditions: 1) they are used on data similar to the Web data (which literally means the input must be some Web data) and 2) their output is always used as a recommendation to a human expert (which means that they cannot be programmed to work autonomously, as these framework creators want you to believe).

If you only apply agentic swarms under conditions 1) and 2), you will quickly realize you don't have much use cases and you don't need agentic swarms.

[Min Choi @minchoi]

It's only been just over a week since OpenAI dropped o3-mini.

And people are already doing crazy things with it!

10 wild examples:

1. Create visually stunning demo of the Big Bang Theory in three.js

[BURKOV @burkov]

The hardest part of reinforcement learning is choosing the right reward values. While 1 for success and 0 for failure seem easy for a human, it's also the hardest reward model to learn from because these rewards are only given at the end of the learning episode, which might consist of thousands of steps. In each step, the agent makes an action in an environment state and transitions to a different state. Propagating a 1 or a 0 across this long sequence of steps isn't an easy undertaking.

The agent would learn better from immediate rewards in each state after executing an action, but defining these immediate rewards requires a human to have a serious level of understanding of the environment dynamics and seems infeasible except for the simplest environments.

This is why reinforcement learning has been tough to put into practice, except for certain environments where 1/0 rewards are relatively easy to propagate, like in language models, some board games, or arcades.

[BURKOV @burkov]

To make it super clear, language models are awesome. Otherwise, I wouldn't spend 9 months of my life working hours after my day job on writing a book on them. What's wrong about them is not what they can do, but what the lying CEOs, VCs, and parasite influencers say they are or will be able to do.

This is what they are awesome at:

1. Giving answers that are more important now with some chance of error than perfect but tomorrow
2. Interactive problem-solving, where the user is an expert and could solve the problem alone, but it would take more time. This includes theorem proving, math problem solving, coding, and technical writing.
3. Converting it into a temporary "You are a classifier that can distinguish between these C classes" model that helps accelerate a complex system development and which will later be replaced with a real classifier.
4. "You act as an expert in domain A. Here's a document from domain A. Extract from it attributes B and C verbatim so that I can automatically locate them for verification."
5. Converting between programming languages, JSON, XML, YAML, or between different API specification formats.
6. "Improve my writing so that it fits in this context."
7. "Write 3 most important points of this long online article."
8. "Translate this text from language A to language B."
9. "Write code according to this specification so that all of my hidden tests pass."
10. "My code fails, here's the stack trace. Fix it."
11. "You are an expert in domain A. Generate examples of documents and their labels, to be validated by a human."
12. "Here's the solution (code, document) provided by a human. It does contain errors. Find them."
13. "Here's a scientific article. What does X in equation 2 represent and where does it come from?"
14. "Here's the code. What does function A do and why is this specific command used?"

Also, use cases where hallucination is a feature:

15. Storytelling, poetry, scriptwriting
16. Brainstorming and ideation
17. Roleplaying (in all senses)

All these use cases have been available since GPT-3.5, and nothing new was added. Only solution quality has been gradually improving without ever reaching perfection.

[Paul Graham @paulg]

I tried asking ChatGPT and here's what it said:

[Sarah Evans @SarahjevsEvans]

New painting
Atlantic Ocean from the West Coast of Ireland
acrylic on wood
16x24”

[Emily Anna @emily_anna1995]

Soothing...
#nature
#NaturePhotography

[Sam Yang @samdoesarts]

Working with some cool lighting scenarios #digitalart #DigitalArtist

[Francisco Ribeiro @fraveris]

Lavender Blues
A lavender field in Provence, France by Rae Andrews

[Francisco Ribeiro @fraveris]

Graham Gercken (Australia)

[Ⓜ️ⒶⓇⒸ ⚽️🏆 @VieNatureSavoir]

Good evening 🖤💫