Danny Rose

Founding AI Engineer @ Alpha Design AI. Researcher in UCSB NLP Group. Passionate about AI real-world applications for the benefit of humanity.

[Xin Eric Wang @xwang_lk]

Introducing 𝐆𝐫𝐨𝐮𝐩-𝐄𝐯𝐨𝐥𝐯𝐢𝐧𝐠 𝐀𝐠𝐞𝐧𝐭𝐬 (𝐆𝐄𝐀), a new paradigm for open-ended self-improvement in AI agents.

The core shift is simple but radical:
👉 𝐭𝐡𝐞 𝐮𝐧𝐢𝐭 𝐨𝐟 𝐞𝐯𝐨𝐥𝐮𝐭𝐢𝐨𝐧 𝐢𝐬 𝐧𝐨 𝐥𝐨𝐧𝐠𝐞𝐫 𝐚 𝐬𝐢𝐧𝐠𝐥𝐞 𝐚𝐠𝐞𝐧𝐭, 𝐛𝐮𝐭 𝐚 𝐠𝐫𝐨𝐮𝐩 𝐨𝐟 𝐚𝐠𝐞𝐧𝐭𝐬.

Why does this matter?
To answer that, we need to rewind.

Scientific breakthroughs share two core traits: open-endedness and cumulative progress.

Exploration never truly ends, and crucially, discoveries don’t vanish: they accumulate, compound, and unlock future advances.

AI progress should work the same way.
But today, it mostly doesn’t.

Most AI systems are built on fixed, human-designed architectures.

They can train.
They can optimize.
They can accumulate experience.

But they rarely escape the structural limits imposed at design time.

They cannot meaningfully reconfigure themselves, so progress remains bottlenecked by continuous human intervention.

To fix this, researchers turned to open-ended self-improving systems, often inspired by biological evolution.

The dominant pattern looks like this:
- select one parent agent
- refine it
- produce offspring
- repeat

Evolution unfolds as a chain or tree.

This sounds powerful, and it is good at generating diversity.

But there’s a hidden flaw.

Those evolutionary branches are isolated.

Agents explore independently, discoveries stay local, and most variants are short-lived. They add novelty, but rarely become stepping stones for long-term progress.

Exploration happens.
Accumulation doesn’t.

This leads to an uncomfortable question:
Why are we forcing AI agents to evolve like biological individuals?

AI agents aren’t constrained by reproduction, lineage, or genetics. They can directly share:
- trajectories
- tools
- workflows
- learned artifacts

They can aggregate complementary skills instantly.

So why not design evolution around that?

This is where Group-Evolving Agents (GEA) comes in.

GEA rethinks evolution from the ground up by treating a group of agents as the fundamental evolving entity.

Not a single parent.
Not isolated branches.

A collaborative, experience-sharing group.

Here’s how GEA works at a high level:

1️⃣ Parent Group Selection
At each iteration, GEA selects a parent group using a Performance–Novelty criterion — balancing strong performance with exploratory diversity.

2️⃣ Experience Aggregation
Instead of isolating agents, all members contribute their experiences into a shared pool.

3️⃣ Group Reproduction
Using this pooled experience, the parent group jointly produces a child group.

Exploration is no longer wasted — it’s consolidated.

This one change fixes a core failure mode of prior systems.

In classic tree-structured evolution:
- diversity explodes
- but useful discoveries die on isolated branches

In GEA:
- diversity is explicitly reused
- progress becomes cumulative
- improvements persist across generations

When evaluated on challenging coding benchmarks, the impact is clear:
- 71.0% success on SWE-bench Verified
- 88.3% on Polyglot
- This significantly outperforms prior SOTA self-evolving methods (56.7% and 68.3%).

But the more interesting part is why.

Analysis shows GEA:
- consolidates exploratory diversity instead of losing
it
- achieves stronger performance with the same number of agents
- is more robust to framework-level perturbations
- improves workflows and tools rather than overfitting to a specific model

These gains transfer consistently across GPT- and Claude-series models.

GEA performs meta-learning-based self-improvement with no human intervention.

And yet it:
- matches human-designed SOTA on SWE-bench Verified (years of efforts from our top AI researchers)
- vastly surpasses it on Polyglot

All through open-ended, group-level evolution.

The takeaway:
Open-ended progress doesn’t fail because exploration is hard.

It fails because discoveries don’t accumulate.

By evolving groups instead of individuals, and enabling explicit experience sharing, GEA turns exploration into something that finally compounds.

That’s the promise of Group-Evolving Agents.

[Danny Rose @DannyRose30]

@WilliamWangNLP Congratulations, this is an amazing accomplishment!

[Danny Rose @DannyRose30]

Thrilled to join Alpha Design in revolutionizing Chip Design!

[William Wang @WilliamWangNLP]

New career update: 🚀 Today, we’re thrilled to launch ChipAgents, our most ambitious project. The agentic AI chip design environment will allow engineers to iterate on your chip design & verification 10x faster by collaborating with ChipAgents in your favorite code editor. 🤖

[Vaishnavi H @vaishnavihima]

🚀 We are happy to officially release the VIP dataset! #videoChainofThought #videoreasoning #LLMs #EMNLP2023

Paper: https://t.co/e3xYj7Y5af
Dataset: https://t.co/1DZG0bEeJq
🧵1/6

[Alex Mei @alexmeigz]

🚨 RED ALERT! 🚨
Tired of expensive and intensive testing of your language model? It's time to automate red teaming with our latest research project accepted to #EMNLP2023: ASSERT -- a test suite for Automated Safety ScEnario Red Teaming! https://t.co/YtDkxc5yAS

🧵1/n #NLProc

[Raphael Schumann @RaphiRaph_]

What happens if you embody🚶GPT in Street View and provide it with navigation instructions?

✨🤖✨
VELMA: Verbalization Embodiment of LLM Agents for Vision and Language Navigation in Street View
✨🤖✨

💻 https://t.co/ZpWSHkVGra
📜 https://t.co/uZLtU2LXcD

[Sharon Levy @sharonlevy21]

🎉Excited to announce that I will be starting as an Assistant Professor of Computer Science at Rutgers University in Fall 2024! In the meantime, I will spend the next year at Johns Hopkins @jhuclsp as a postdoc with @mdredze!

[Danny Rose @DannyRose30]

We will soon release the Video Infilling and Prediction (VIP) inference-time dataset to evaluate video chain of thought abilities!

Blog: https://t.co/frCkibaMps

Outstanding co-authors: @vaishnavihima, @andy__ouyang, @RyanHe88, @alexmeigz, @yujielu_10, @m2saxon, @WilliamWangNLP

[Danny Rose @DannyRose30]

Large language models can help understand videos by thinking frame by frame.📽️💡

A video chain of thought creates structured and unstructured scene descriptions that bridge the gap between videos and LLMs! #VideoCOT #LLM

https://t.co/yQFuKlloV9 🧵3

[Danny Rose @DannyRose30]

Videos are a series of around 24 frames per second, rich in visual and semantic information. 📸📖

Creating scene descriptions of sparse key frames guide multimodal models to predict intermediate and future frames in a computationally-efficient way. 🎞️

[Alon Albalak @AlbalakAlon]

In this new preprint, we (@colinraffel @WilliamWangNLP) draw inpsiration from (1) exploration in RL and (2) data selection in supervised learning to develop algorithms that 🚀beat SOTA few-shot methods🚀

📜https://t.co/kJr7Lbb2dG
code: https://t.co/LpgrjxFXRO
1/4

[Danny Rose @DannyRose30]

Thank you for promoting our research in multimodal computer reasoning! See below for an official introduction of Visual Chain of Thought! https://t.co/WwkDK0Nsit

[Aran Komatsuzaki @arankomatsuzaki]

Visual Chain of Thought: Bridging Logical Gaps with Multimodal Infillings

Presents VCoT, a novel method that leverages chain of thought prompting with vision-language grounding to recursively bridge the logical gaps within sequential data

https://t.co/m4M5AoenEw

[Danny Rose @DannyRose30]

Thank you for sharing our work on multimodal computer reasoning! Our official introduction of Visual Chain of Thought is below! https://t.co/WwkDK0Nsit

[AK @_akhaliq]

Visual Chain of Thought: Bridging Logical Gaps with Multimodal Infillings

abs: https://t.co/DCGi0ixQT0

[Danny Rose @DannyRose30]

6/6 Stay tuned for our upcoming code and demo releases! 🚀

In the meantime, here’s a blog further explaining VCoT:
https://t.co/7u19WaJwLc

Amazing co-authors: @vaishnavihima, @andy__ouyang, @RyanHe88, @alexmeigz, @yujielu_10, @m2saxon, @WilliamWangNLP

https://t.co/rwpzufTy1S

[Danny Rose @DannyRose30]

5/6 We demonstrate VCoT’s potential using VIST (Visual Storytelling) and WikiHow datasets.

VCoT improves upon baselines and/or offers insights into computer reasoning, such as the high consistency low novelty of CoT, or the high novelty low consistency of CoI. ⬆️⬇️

[Danny Rose @DannyRose30]

4/6 VCoT can also enhance downstream sequential reasoning tasks like visual storytelling or instruction summarization. Check out this example summarizing an article: “How to Care for Burmese Cats”

[Danny Rose @DannyRose30]

3/6 Human evaluation demonstrates that VCoT generates more novel and consistent multimodal infillings than CoT (chain of thought on input text) plus CoI (chain of thought on input image captions).

[Danny Rose @DannyRose30]

2/6 Visual Chain of Thought (VCoT) mimics human reasoning by combining chain of thought with vision-language guidance/imagination.💡

VCoT pipeline:
-Unify sequential data into text-visual pairs
-Extract the sequence’s main focus
-Recursively generate novel/consistent infillings

[Danny Rose @DannyRose30]

Multimodal infillings can unlock the power of computer reasoning about sequential data. 🖥️🧠

We’re excited to announce that “Visual Chain of Thought: Bridging Logical Gaps with Multimodal Infillings” is available on #arxiv! #LLM #GPT #StableDiffusion #NLP #CV #VCOT 🧵1/6

[Yujie Lu @yujielu_10]

🚀Thrilled to release #TIP (Dual Text-Image Prompting), a #DALLE2 #StableDiffusion-2 enhanced #LLM that can generate coherent and authentic multimodal procedural plans toward a high-level goal. 🧵8

📜paper: https://t.co/ua2P5Bp1Dn
🔗data & code: https://t.co/YkIlPJjtgs