Sam Wiseman

NLP and ML things, formerly @duke_nlp.

[Reflection @reflection_ai]

Most approaches to “agentic AI” focus on post-training fixes.

In this conversation, member of our technical staff, @achowdhery argues the bottleneck is pre-training itself. Drawing on her work on PaLM and early Gemini, she explains why next-token prediction breaks down for long-horizon planning -- and how objectives, attention, and training data must evolve to support true agentic behavior.

[Sam Wiseman @_samwiseman]

@sebgehr Thanks!

[Sam Wiseman @_samwiseman]

Thrilled to share I’ve joined Reflection AI, just as it’s announced its Series B funding round! I joined because I’m excited about its ambitious plan to train its own open-weight frontier models, and because of its enormously talented people. Congrats to the team!

[Reflection @reflection_ai]

Today we're sharing the next phase of Reflection.

We're building frontier open intelligence accessible to all.

We've assembled an extraordinary AI team, built a frontier LLM training stack, and raised $2 billion.

Why Open Intelligence Matters

Technological and scientific progress is driven by values of openness and collaboration.

The internet, Linux, and the protocols and standards that underpin modern computing are all open. This isn't a coincidence. Open software is what gets forked, customized, and embedded into systems worldwide. It's what universities teach, what startups build on, what enterprises deploy.

Open science enables others to learn from the results, be inspired by them, interrogate them, and build upon them in order to push the frontier of human knowledge and scientific advancement. AI got to where it is today through scaling ideas (e.g. self-attention, next token prediction, reinforcement learning) that were shared and published openly.

Now AI is becoming the technology layer that everything else runs on top of. The systems that accelerate scientific research, enhance education, optimize energy usage, supercharge medical diagnoses, and run supply chains will all be built on AI infrastructure.

But the frontier is currently concentrated in closed labs. If this continues, a handful of entities will control the capital, compute, and talent required to build AI, creating a runaway dynamic that locks everyone else out. There's a narrow window to change this trajectory. We need to build open models so capable that they become the obvious choice for users and developers worldwide, ensuring the foundation of intelligence remains open and accessible rather than controlled by a few.

What We've Built

Over the last year, we've been preparing for this mission.

We’ve assembled a team who have pioneered breakthroughs including PaLM, Gemini, AlphaGo, AlphaCode, AlphaProof, and contributed to ChatGPT and Character AI, among many others.

We built something once thought possible only inside the world’s top labs: a large-scale LLM and reinforcement learning platform capable of training massive Mixture-of-Experts (MoEs) models at frontier scale. We saw the effectiveness of our approach first-hand when we applied it to the critical domain of autonomous coding. With this milestone unlocked, we're now bringing these methods to general agentic reasoning.

We've raised significant capital and identified a scalable commercial model that aligns with our open intelligence strategy, ensuring we can continue building and releasing frontier models sustainably. We are now scaling up to build open models that bring together large-scale pretraining and advanced reinforcement learning from the ground up.

Safety and Responsibility

Open intelligence also changes how we think about safety. It enables the broader community to participate in safety research and discourse, rather than leaving critical decisions to a few closed labs. Transparency allows independent researchers to identify risks, develop mitigations, and hold systems accountable in ways that closed development cannot.

But openness also requires confronting the challenges of capable models being widely accessible. We're investing in evaluations to assess capabilities and risks before release, security research to protect against misuse, and responsible deployment standards. We believe the answer to AI safety is not “security through obscurity” but rigorous science conducted in the open, where the global research community can contribute to solutions rather than a handful of companies making decisions behind closed doors.

Join Us

There is a window of opportunity today to build frontier open intelligence, but it is closing and this may be the last. If this mission resonates, join us.

[Sam Wiseman @_samwiseman]

@ponderousbs Thanks! Me too :)

[Sam Wiseman @_samwiseman]

@Jinwu76 Me too!!

[Sam Wiseman @_samwiseman]

@sanxing_chen Thanks!! :)

[Sam Wiseman @_samwiseman]

@brendanm0407 Same!

[Ruoming Pang @ruomingpang]

As Apple Intelligence is rolling out to our beta users today, we are proud to present a technical report on our Foundation Language Models that power these features on devices and cloud: https://t.co/TaAdd0fBOp. 🧵

[Daniel Paleka @dpaleka]

And this cool paper from 2021 which deserves more recognition https://t.co/uDEbU6Ppa8

[Wojciech Kryściński @iam_wkr]

🚨🚨Registration for the @coling2022 #CreativeSum Workshop Shared Tasks (books & tv/movie script summarization) is now open, the form is available here: https://t.co/FPGI2PCEVO

Registration closes July 1st.

#NLProc

[Wojciech Kryściński @iam_wkr]

🚨📑🤖🚨

Excited to announce that the Automatic Summarization for Creative Writing Workshop (#CreativeSumm) will take place at #COLING2022! For updates on the timeline check: https://t.co/yGJUOeDGU6

[Craig Thomson @ThomsonSoftware]

I keep seeing published or under-review papers in #NLProc data-to-text generation using the RotoWire dataset where problems of train/test contamination are not addressed. Evaluation are also poor. I created a simple post to explain both including a TL;DR https://t.co/AaP4c2VLhW

[Silvia Fabbi @silvia_fabbi]

#EMNLP2021livetweet #EMNLP2021 Generation by Splicing Together Nearest Neighbors - Sam Wiseman (@_samwiseman), Arturs Backurs (@ArtursBackurs), Karl Stratos (@karlstratos) 👉 https://t.co/hG4cvTI3cC

[Sam Wiseman @_samwiseman]

So we train a model to mimic this shortest sequence, which gives fairly interpretable derivations of text from neighbors, like the one at the top of the thread.

3/3

Also, hello twitter people 👋

[Sam Wiseman @_samwiseman]

Newish #EMNLP2021 work w/ Arturs Backurs & Karl Stratos: we try to generate text (in a data-to-text setting) by splicing together pieces of retrieved neighbor text.

Paper: https://t.co/M91S5sxQf5

1/3

[Sam Wiseman @_samwiseman]

The idea is to allow arbitrary span insertion/replacement (from neighbors) into a canvas. Turns out finding the *shortest* sequence of these splice ops that yields a sentence given some neighbors reduces to parsing under a WCFG.

2/3