gravynz

jacked and cracked

[gravynz @algravithm]

move 37

[Aaron Levie @levie]

It’s almost incomprehensible how wide the distance is between someone who’s working at the frontier with AI and basically everyone else right now.

The crazy thing is these lines are not converging, but only growing farther apart. If the curve of AI progress had leveled off, it would certainly have given most people and companies time to catch up or implement their workflows in a standardized way. The technology would have neutralized competitive advantage because everyone would be using it the same way.

But what’s happened is the opposite. The curve of AI progress is basically going vertical right now, which means if you had codified your processes 3-6 months ago you’re likely already behind again. Now imagine most companies that kicked into gear a year ago and thought that they were done. Even further behind now.

I’ve never seen a technology in my life and career that improves at this rate, which consequently means you need to upgrade your expectations at a similar rate. Our brains weren’t wired for this, and our organizational change management processes certainly weren’t either.

With any form of recursive self improvement, things just get wilder. It’s a time when you will need to be constantly reevaluating the things that didn’t work just a couple months ago to see if they now can be solved. AI native companies certainly have a strong advantage due to this right now, so now it comes down to can the bigger companies keep up. Wild times.

[Randall Balestriero @randall_balestr]

The Platonic Representation Hypothesis is now a Theorem ◼️
Train JEPAs separately, on any modalities: they provably learn the same representation up to one rotation.
The result? A 109M JEPA-Text model that aligns with vision models better than 72B LLMs.
w/ @ylecun 🧵

[Jayden Teoh @jayden_teoh_]

Multi-Token Prediction (MTP) can be done much better.

🚀 Turn out,you can recover the gradient for KL self-distillation to next-token prediction (NTP) for free without ever computing the KL. You just need a one-line code change:

h_ntp += h_mtp.detach() - h_mtp

Soft Multi-Token Prediction is pretty much a free lunch for MTP.

📜 Paper: https://t.co/K69tPSXe2k
💻 Code: https://t.co/K1G4CAPi7d
🤗 HuggingFace: https://t.co/IQ86GSJn55

[Jayden Teoh @jayden_teoh_]

Next-token prediction is myopic. What if transformers learn to predict their own next latent state?

🌠 We present 𝗡𝗲𝘅𝘁-𝗟𝗮𝘁𝗲𝗻𝘁 𝗣𝗿𝗲𝗱𝗶𝗰𝘁𝗶𝗼𝗻 (𝗡𝗲𝘅𝘁𝗟𝗮𝘁): a self-supervised learning method that teaches transformers to form compact world models for reasoning and planning. It also unlocks up to 3.3x faster inference via self-speculative decoding! 🚀

[Felix Draxler @FelixDrRelax]

This will change how LLMs are trained: Parallel Language Models predict several tokens in parallel at autoregressive quality. Add a parallel decoder at pretraining, get 5.7x speedup after posttraining.
🧐 → https://t.co/vXTeHmLOEe
ℹ️ → https://t.co/UuhnIBJZ6x
Read on: 🧵 1/4

[Yuda Song @yus167]

Foundation models are trained in stages: pretraining, midtraining, and post-training. But is this separation fundamental?
We introduce unified training: a single algorithm for the full pipeline. This gives
⚙️ Simpler infrastructure
🔬 A framework for better science
(1/n)

[Cursor @cursor_ai]

We identified a security incident affecting a modified Claude Code VS Code extension accessed through our Cursor IDE extension marketplace, impacting at most 5,000 users. This malicious version of the extension was not developed by Anthropic. It does not impact other extensions downloaded from the https://t.co/sJRGX5jLXW, Grok Bot, Grok or our other SpaceXSI products.

In response, we took down our extension marketplace this afternoon, worked with relevant providers to neutralize the malware, and have reinstated the marketplace after confirming there is no further risk.

If affected, please review instructions in this blog post to understand how to find the malicious package and establish a path to mitigate any potential impact. We are asking all customers update to Claude Code for VS Code version 2.1.314: https://t.co/ESqf7YEtHZ

[Alexi Gladstone @AlexiGlad]

Energy-Based Models can now scale better than Diffusion models.

Introducing TEM: Transformer Energy-based Models

TLDR:
• TEM outscales Diffusion without guidance
• First scalable contrastive Energy-Based Models
• Better generalization, test-time scaling, and OOD detection than Diffusion

🧵

[Ilya Razenshteyn @ilyaraz2]

I interviewed ~2 dozen researchers, engineers, research / engineer managers etc. asking them how they / their teams use coding agents in their work. Then I sprinkled it with my opinions and hot takes and wrote a substack post (link in the next tweet, I'd be delighted if you subscribe; of course, the post is free and any future post will be free too!) trying to triangulate the current reality of software engineering / industry research as opposed to what gets peddled on social media.

TL;DR (3. and 4. are the most important!):

1. Models, harnesses, workflows: don't bother with fancy workflows, they don't really work. Use stock Claude Code / Codex (or Copilot with Claude / GPT models if you are a Microsoft stan), perhaps with subagents in some very special cases. Write a design doc, then plan a change, then implement it, then merge. Wash, rinse, repeat. Don't get swayed by people online who are trying to sell you snake oil in the form of their new harness or workflow. None of this stuff works (in general).

2. Correctness/tests/reading or not reading code: it's OK not to read code if you do one of the following three things. a) One-off stuff / demos, which you need to work on exactly one input. b) Testable / verifiable stuff, e.g., GPU kernels. c) Stuff, where it's OK to be "directionally correct" (ML / data work / research code). In other cases, especially if you require any degree of maintainability / reliability or your code is customer-facing, reading code is still necessary. Reading tests and steering the model to write better tests is a good idea too. It's often good to use a different model to review the code that the other model wrote. Even if you approve the PRs without changing them, reading still helps for future more productive prompting.

3. Team dynamics: if we are not careful, agents can really undermine team dynamics. Two biggest problems are "slop infestation", where over time all PRs / emails / Slack messages end up being unreadable slop, and "overbuilding", where we are building and shipping useless stuff to get recognition or as a form of procrastination. Other, less severe but still important problems include: reduced communication, being secretive about ideas, decreased diversity of thinking, mode collapse of the ideas, trusting LLMs more than people.

4. Call for action: we really need to be careful with preserving team work and collaboration. We still need to "form packs to hunt mammoths", but with coding agents this becomes increasingly harder. Overall, many of the agents-induced problems stem in myopic short-term thinking of the sort "we have a couple of years left until our jobs are fully automated, so I can as well extract as much value as possible, everyone else be damned". While somewhat understandable, I think it’s important to resist these urges, if nothing else, for your own sanity’s sake.

[Full disclosure: until 2 months ago, I worked at OpenAI and my work directly contributed to Codex, I'm a big fan of coding agents and quite strongly AI-pilled, but there is little I hate more than fearmongering, gaslighting, lies and rage-bait that are ubiquitous in this space, unfortunately, hence spending time on all of this.]

[Alex Imas @alexolegimas]

I joined @GoogleDeepMind to develop a research program on the economics of AGI. Today, together with Andrew Koh (@andrewjkoh) and Shane Legg (@ShaneLegg), we are incredibly excited to release The Economics of Intelligence, the first in a series of projects from our group on how AGI will change the economy.

How do capability improvements transform the economy, and when? In this project, we define thresholds of economic capabilities, e.g., minimal AGI, based on how they transform the economy. We then track how the economy–market structure, employment, and wages–changes during the transition across capability regimes.

We have seen exponential increases in what AI agents can do but the broader economy seems almost remarkably unchanged. Part of the reason is that the solo AI agent is the wrong economic object; economic transformation is an equilibrium outcome where organizations compete. Looking at how intelligence functions within organizations, transformation will depend on when two key capability thresholds are crossed: competence (gains on individual skills, e.g., coding, math) and coordination (gains on ability of agents to cooperate and work together).

We show that what economic transformation looks like depends on whether AI becomes supercompetent, supercooperative, or both—the regime of artificial superintelligence (ASI). Tracking this transition shows why the real visible transformation may come from improvements in coordination—a capability most benchmarks don’t yet track. Past a certain threshold, AI capabilities can break the Coasian ceiling on firm size and lead to an economy that looks drastically different than the one we see today.

Two main takeaways:
1. To understand the path of transformation, we should be tracking agentic coordination more closely.
2. The impact of AI will likely show up in wages before market structure or employment.

We are really excited about this paper. Excited to hear people’s thoughts.

Link: https://t.co/9DTGSeKoSA

[Sasha Rush @srush_nlp]

I've been really fascinated by the DL Alchemy course (https://t.co/4aSUZbOEs3).

The decision to put the burden of theory construction on the student (and their claude) feels transformative and surprisingly challenging. Maybe knowledge in ML was never that critical.

[div (fp32) @divagr1925]

1/4

Meet Qwenputer: Qwen3.5-4B with a full RISC-V CPU inside its weights, using the model's own Gated DeltaNet layers as RAM.

It runs C and MicroPython in its forward passes, no external runtime.

On DeepMind math: base 55–90%, base + Python tool 95–100%, Qwenputer 97.5–100%.

[Albert Gu @_albertgu]

I've always believed that intelligence should come from simple, general building blocks that learn their own abstractions directly from raw data; in other words, the bitter lesson. I think we're finally seeing a real step towards AI that can learn from all data.

Language modeling on raw bytes is a microcosm of this ideal, and to me it's a central problem in AI: if we can't build LLMs that learn end-to-end from bytes, how will we handle the rest of the world's data that we understand even less?

Last year we introduced hierarchical networks (H-Nets) as a first step. Our new version finds up to 3-5x efficiency gains in controlled scaling law experiments, and at larger scale matches an open-source model trained on 11T tokens with ~10x less data and compute. The posts below walk through what we found and why we think it works.

[Herman Brunborg @hermanbrunborg]

I improved NanoGPT Speedrun from 39.9s → 6.7s (-83.2%), more than every record since 2025 combined (-81.4%)!

The web is full of rare, repeated content that the LM can’t memorize in time. I solve this with retrieval-augmented n-gram hints🧵

[Alex Tamkin @AlexTamkin]

New Anthropic research: How AI agents shape experts' judgment calls

When experts delegate work to an agent, do they still make the key decisions? We tested this in a randomized trial with 50 professional data analysts.

[Felix Rieseberg @felixrieseberg]

The coding interview is dead, at least for my team at Anthropic. I've seen enough to know that it no longer provides signal. I removed it from loops this month.

I have a spicy, optimistic take - I think there's an opportunity for junior engineers: https://t.co/PVvpQCbQtK

[Google DeepMind @GoogleDeepMind]

This is Genie 4, our most realistic and interactive world model yet. 🌐

Explore dynamic environments in real time to see how the scene responds, sounds, and how the people in it move, react, and speak.

[Anthropic @AnthropicAI]

Over the past few months, Anthropic scientists have engaged in deep research partnerships with six organizations developing new medicines. We've also begun drug discovery efforts of our own.

In doing so, we’ve learned much more about how Claude can support the many stages of drug development.

In this blog post, we share some early results from these efforts, and describe how this work fits in with our wider ambitions in the life sciences.

https://t.co/EVbxkdDTgJ

[Ryan Williams @rrwilliams.bsky.social @rrwilliams]

An internal model at OpenAI has produced a claimed refutation of ETH! Russell Impagliazzo, Mohan Paturi, and I have been asked to help organize an open and independent examination of this claim. 1/4

[gravynz @algravithm]

i wonder if its worth it to learn competitive programming even as a non CS students (statds) since i aimed to land an occupation as an ai engineer