Amin Moghaddam

[Amin Moghaddam @m30m_en]

A nice way to see SoTA on image generation. The three fingers dataset is my favorite.

[OpenRouter @OpenRouter]

New: visual benchmarks for understanding the capabilities of image models 🌅

See how every model performs across a variety of challenging prompts: https://t.co/P98jIE8dsn

[Amin Moghaddam @m30m_en]

Randomly wrote None instead of none in the middle of the sentence.

[Amin Moghaddam @m30m_en]

Someone should make an RL Env from all the tasks that @huskirl has created so far.

[Amin Moghaddam @m30m_en]

Today I learned about this list and it's awesome!
Lots of interesting people to follow from the list.

[Chris Barber @chrisbarber]

Talked with the head of data acquisition at a large ai lab: "we are already working with a lot of data vendors and wanted to see if you had any other suggestions"

"where did you find the ones you work with today?"

"mostly on pavlov’s list"

[Amin Moghaddam @m30m_en]

Actively switching between these 2:
the fear of AI becoming great super fast and rendering many positions redundant
the fear of AI not becoming great fast enough to show ROI and cause a massive recession

[Amin Moghaddam @m30m_en]

I spend a good amount of weekends closing tabs

[Amin Moghaddam @m30m_en]

Loved the intro. We are at the era of jagged intelligence.

[Izzy @isidoremiller]

Full details at https://t.co/XHloX3q2qf or read on for takeaways:

[Amin Moghaddam @m30m_en]

A short summary of OpenAI incidents

[Amin Moghaddam @m30m_en]

I wish I was a character in TinyStories dataset.

[Amin Moghaddam @m30m_en]

You vs the startup she tells you not to worry about.

[Jeff Dean @JeffDean]

We created a pitch deck to tell a handful of VC firms about us and what we were up to (a fun experience!). Here’s a few slides about our background and some of the things we’ve worked on from the pitch deck (it was fun putting together the list of people in our teams who have gone on to found a whole range of exciting companies). We are delighted to have selected @radicalvcfund
and @khoslaventures to lead our initial funding round, along with participation from @lightspeedvp, @kleinerperkins, Doerr Capital (@johndoerr), and Alphabet (@Google). We’ll be working with them to close our seed round over the next few weeks.

[Amin Moghaddam @m30m_en]

@abeirami @octonion Say it, and I will do the analysis

[Amin Moghaddam @m30m_en]

@rauchg @max_paperclips The model is trying to use the Goldratt classic principles to improve itself upon further iterations.

[Amin Moghaddam @m30m_en]

All these models are super eager to layoff humans😂

[DAIR.AI @dair_ai]

On benchmarking long-context agentic instruction following.

Agent benchmarks mostly reward reaching the answer. This new benchmark measures whether the agent reached it the permitted way, which is the question enterprise deployments care about.

If you ship skills files, policy documents, or long system prompts, you have been trusting that they actually bind agent behavior.

But how are you measuring all of this?

Surge AI built a benchmark to actually check this.

HANDBOOK.md places a standard operating procedure of 20 to 124 pages in context and grades whether it governed every action across an extended tool-use horizon. 65 tasks, five domains, ten fictional companies.

Each task runs in a self-contained company environment with a file workspace plus mock email, chat, calendar, issue-tracking, and commerce services exposed over MCP. Every task mutates one of ten base handbooks, altering the specific rules and thresholds that grading turns on, so memorization does not help.

Grading is fully deterministic and two-sided. 824 programmatic criteria check that required actions occurred and that prohibited actions did not.

Paper: https://t.co/qKA2z8R1rz

Learn to build effective AI agents in our academy: https://t.co/LRnpZN7L4c

[Amin Moghaddam @m30m_en]

@henryehrenberg How did you decide on which models to use for LLM-based rewards? Was there any differences between Sonnet and Opus in terms of grading or mainly just the cost?

[Amin Moghaddam @m30m_en]

Feeling blessed to work in the software industry. Great post

[Arvind Narayanan @random_walker]

To understand and empathize with how workers in many or most fields outside software experience advances in AI capabilities, I propose a little thought experiment. https://t.co/QZG24aRiBe

[Amin Moghaddam @m30m_en]

Thanks to all the public data available by @datacurve , I created the SlopSpotter:
https://t.co/qxkSg4VYxN

[Amin Moghaddam @m30m_en]

@ardabasarici @plainionist well, the issue is that they don't maintain them very well. After a while you have a ton of out-of-sync comments that just confuse the agent more.

[Amin Moghaddam @m30m_en]

I was eyeballing some coding benchmarks and trajectories, saw a commit from someone I knew from highschool. What a small world!

[Amin Moghaddam @m30m_en]

Kudos to @datacurve for open-sourcing all the tasks, patches, and trajectories. This makes research easier and more efficient.