Reggie

nullius in verba

[jack @jack]

i’ve shifted from telling agents what to do, to asking them what to do, and pulling the best thread.

[Reggie @LeReggie]

@ChaseBrowe32432 @tszzl This is survivorship bias in whom you (still) choose to talk to often.

[Reggie @LeReggie]

@JacobBastian25 The French have only made one significant contrib to frontier LLMs and that's @MistralAI 's pdf parser https://t.co/jBMo9jT5KH

[Reggie @LeReggie]

I mean, duh.

[Łukasz Bartoszcze @ten_lukasz]

@didier_lopes @pangram https://t.co/GDAY2CKTdU two guys from ucl did it pretty well, the model is open source for a great paraphrasing humanizer

[Reggie @LeReggie]

Thesis; antithesis; synthesis.

[vicki @vboykis]

@christophcsmith I think it could be both, but from opposite ends: ie people who have not written much code before are becoming more rigorous and long-time coders, less so we are getting sucked into some kind of amorphous average

[Reggie @LeReggie]

This seems more like an ad than real research. I read the post three times (and have a phd in the subject) I can't tell what the hell they actually did.

[John Schulman @johnschulman2]

People sometimes ask why fine-tune when general-purpose models keep getting better. Bridgewater's work is a good reminder that with the right data -- here, expert judgements -- you can beat prompting-only approaches by a lot.
@ddkang and the Bridgewater AIA Labs team are great -- glad to see them sharing this.

[Reggie @LeReggie]

In some ways *mass* technological unemployment is preferable... it creates a problem that must be addressed immediately. Going from 4% to 8% of non politically active demos would be a disaster we'd probably ignore until it's too late.

[Council on Foreign Relations @CFR_org]

“It does not take a very high unemployment rate to fundamentally disrupt a country and its politics and its institutions… And it happens fast,” says CFR expert and former U.S. Secretary of Commerce @GinaRaimondo, discussing the political and social risks of an AI-driven jobs transition.

Watch the latest episode of The Spillover, hosted by @scmallaby with CFR President @MikeFroman: https://t.co/m59whrzfdn

[Reggie @LeReggie]

@suchenzang 1. Hard agree
2. wut?

[Pat Simmons @per_simmons_]

Claude just became a craacked video game designer.

With the launch of Unreal Engine's MCP server last week, you can now build entire video games just by talking to Claude.

I spent the past few days building with it, and I'm telling you, this is going to forever change how video games get made and who gets to make them.

In this video I show you exactly how to set up the Unreal Engine MCP yourself and run through three demos: building a full playable city, cloning a real city from Google Earth, and creating custom buildings in Blender.

Here's the agent harness I mention too: https://t.co/mos9EwnZ2h

Intro
What I built in a few hours
Setting up the Unreal MCP server
Fixing the port 8000 connection issue
The agent harness that avoids the pitfalls
Demo 1: Building a city with City Sample
Demo 2: Cloning a real city from Google Earth with Cesium
Demo 3: Custom buildings with Blender headless
Outro

[Patrick Collison @patrickc]

I asked Claude about the air conditioning debate in Europe, and it really didn’t pull any punches.

[Alyssa Unell @AlyssaUnell]

1/ We use LLM judges to scale up costly human evaluation. But to trust an LLM judge, you need… human evaluation. 🔄

Our new preprint tackles this circularity: "Metric Match: A Subset Selection Approach to Evaluating LLM Judge Reliability" 🧵

[Reggie @LeReggie]

@Muennighoff @dhtikna Yeah as a data scientist all of the code I write is based on a mountain of non-code context

[John Carmack @ID_AA_Carmack]

If you are asking “Why push back against anti-datacenter efforts?” I consider it a tragedy that anti-nuclear efforts largely strangled nuclear power in the US based on vibes, and I don’t want to see that happen to AI. Public opinion matters, and it shouldn’t be ceded unchallenged.

If you are asking “Why should I support AI efforts at all?” I believe we are in the midst of a transition more vibrant than the industrial revolution. Opinions formed a couple of years ago about the uselessness of AI are no longer valid. Millions of people and organizations are getting great returns from using it, and the demand for data centers is the market responding to the value signal. That is how progress is made!

[NVIDIA DRIVE @NVIDIADRIVE]

We just made it significantly easier to build your own reasoning-based AV.

Alpamayo 2 Super: 32B params, 7+ cameras, 360° perception, chain-of-thought reasoning, and autolabeling. AlpaGym: closed-loop RL training so your model learns from its own mistakes in simulation.

Full replay: https://t.co/TC3odTSUeN

[Reggie @LeReggie]

@JustHarvey_ @MediumSizeMeech Somehow much less true to life

[Richard Socher @RichardSocher]

AI is now doing our AI research.

At Recursive we set out to build recursive self-improving superintelligence (RSI) to automate knowledge discovery. The best way to expand humanity’s knowledge is through the scientific method.

RSI leads to better ideas, explanations and inventions which lead to better RSI. Automating the scientific method requires closing the loop between ideation, implementation and validation, and being able to run it over extended periods of time.

Today, we are excited to share the first outputs of Recursive’s automated open-ended discovery system. To be clear, this system is merely a milestone towards RSI, a v0.1 of what I sometimes call the “Eureka Machine”. It is one program that you can point at any hard problem and get useful inventions out. Though it’s still very early, we've run it on three AI tasks and achieved state-of-the-art results on all three.

These results demonstrate that even this early version of the system can solve a variety of autoresearch problems in AI and improve over prior state of the art. Concretely, it did this on the community benchmarks NanoGPT speedrun, NanoChat, and NVIDIA's Sol-ExecBench.

AI is code and AI can code.
The code and ideas that lead to these results were not invented by our team but by the AI system itself.

To do RSI safely, we need to investigate its inventions. That's best done transparently with the community.
@Recursive_SI we are open-sourcing the system’s discoveries, demonstrating that it finds creative and benign solutions instead of focusing on obvious optimizations or dangerous ideas.

Link below.

[Reggie @LeReggie]

@tszzl @LinusMixson Too many read and got the wrong lessons. @AnthropicAI taught HAL to lie.

[Nathan Lambert @natolambert]

The core part of this Anthropic Fable release saga is that there are many overlapping issues at once. Some of which operate on different timelines of the AI arc, and some have easier fixes. In my critiques, I asked for specific changes to some things, understanding that some things don't have an easy fix.

The simplest issue was an uneven application of safety domains in a way that was misleading to users. This was an implementation issue that overlaps with a values-based decision of what their customers should be doing. Many people including myself pointed out how it was insane to list core safety areas and then have one of them launch with a different safety mechanism, one which actively mislead users. Doing this from the guise of safety was a major misstep and in my opinion Anthropic got very justifiably raked over the coals for it. Don't release the model if you can't hit your safety targets.

A subissue here is the idea of silent manipulation. This again is a horrible precedent, and quite odd for a company that has done extensive, leading technical AI safety research on ideas like CoT monitoring and other emergent misalignment issues. Silent manipulation of users is baking in a misalignment to the system at its face level. This comes with a permanent degradation in user trust, which begets a less safe environment for AI. Users who don't have clear information on how AI works will not develop safe working patterns with it.

The more complex issues are with how Anthropic handles broader scientific engagement with their models. The safety classifiers launched with these models obviously have accuracy issues to start. I have priced in that there will be more false positives to start, that's life. It's Anthropic's business to degrade their products at release time, or make the trade off of user satisfaction versus revenue. Still, it is a very real sign of concentration of power that businesses can make such obviously user-harmful behaviors and still lead in the market. This concentration of power is only starting to set in and we could see even weirder signs of it in the coming years.

It is now simple enough for me to test Claude Fable in my workflows and know if I'm restricted. This is obviously a suboptimal equilibrium – i want the best intelligence I can get, without restrictions – but it is easy enough for me to make sense of and work with.

The specific issue of restricting access to AI research in particular was a bubbling and hard to fix issue with Anthropic specifically, and the frontier labs generally. There is a common view that the frontier labs will be the mediators of all major scientific innovations in the future, as the places with the best models and the compute for inference to solve major problems. This is a categorical error in how science works, which is a community evolution of accepted ideas, and the the evaluation of your ideas by (hopefully numerous) independent, other practitioners. You cannot have science advance only within a monolith.

As an AI researcher I'm very sad to have the latest models restricted, but I would expect Anthropic to do this eventually. I lost more trust over the silent manipulation than I would with a restriction in access. Anthropic has made it pretty clear that they only trust themselves as the mediators of cutting-edge AI research.

If I had a say, Anthropic should've proactively made a program to make sure researchers get access in the broader AI community without the safeguards. Academics, nonprofit workers myself, etc. have no reason to not get access. The only valid argument here is that they want to control frontier AI, which is a know your customer part of serving these models.

This worldview of science has personally motivated me greatly over the last year, and increasingly so this week, to make the open science of AI continue to be viable. Olmo was a wonderful success here. Still, building research infrastructure is different from working for access to the tools needed to do the trade.

[Reggie @LeReggie]

@DimitrisPapail @natolambert Even without nefarious intent, this is always true of solving complex models with LLMs (and humans).

[ComfyUI @ComfyUI]

Turn a simple idea into a storyboard, then generate the video with Seedance 2.0.

This ComfyUI workflow uses LLMs to structure prompts into storyboard-ready scenes that define how the video should play out. The storyboard can be paired with reference images and sent directly into Seedance for video generation.

To try this workflow, link below 👇