Logan Mann

Computer Engineering @UCSB | prev: @mercor

[Logan Mann @loganmann]

Fresh from #NeurIPS2025! 🧠✨
Presented my work at the CogInterp workshop: "Don't Think of the White Bear."
We investigate Ironic Process Theory in LLMs.
The core insight? Safety prompts can backfire under cognitive load. 🧵👇

[Logan Mann @loganmann]

@BarrAlexandra hey Alexandra, I worked on evals at Mercor. A couple of my papers that may be useful:

White Bear / ReboundBench: negation under cognitive load
https://t.co/LDBOk1h3x0

Where Reliability Lives in VLMs
https://t.co/RKbycy2eKw

would love to chat!

[Logan Mann @loganmann]

Attending Bench Summit or in the Bay Area on Oct 28?

Let’s connect and meet up!

[Logan Mann @loganmann]

I’ll be presenting “Is It the Agent or the Eval? Diagnosing Failures Before Trusting Scores” at Bench Summit by @Kaggle on Oct 28 in San Francisco!

Let’s make agent evaluations more trustworthy. @GoogleDeepMind #AI #AgentEvals

[Logan Mann @loganmann]

Selecting the best agent more often does not necessarily minimize the cost of a wrong choice.

Following discussions with a former DeepMind researcher, I built a reproducible synthetic experiment comparing uniform sampling with pilot-based variance allocation: four agents, eight tasks, and utility defined as reward minus 800 times a failure indicator.

At 25,600 simulated draws per policy per world, with rare risk concentrated in one agent-task pair, variance allocation made 539 wrong selections versus 455 for uniform across 1,024 worlds. Yet mean utility lost fell 27.8%, from 0.0268 to 0.0193.

The two metrics ranked the evaluators differently. Selection accuracy counts a near tie and a costly mistake equally. Regret measures the expected utility gap between the best agent and the selected one.

With risk spread across tasks, variance allocation was worse on both metrics. This is not a general win for adaptive sampling.

The practical lesson: report how often agent selection is wrong and how costly those errors are. Lower average regret does not guarantee safety.

Independent, exploratory synthetic work; no real-agent validation. Code, data, uncertainty and reproduction instructions:
https://t.co/aXzfct7MeZ

#AgentEvaluation #AIResearch #ResearchEngineering

[Logan Mann @loganmann]

Selecting the best agent more often does not necessarily minimize the cost of a wrong choice.

Following discussions with a former DeepMind researcher, I built a reproducible synthetic experiment comparing uniform sampling with pilot-based variance allocation: four agents, eight tasks, and utility defined as reward minus 800 times a failure indicator.

At 25,600 simulated draws per policy per world, with rare risk concentrated in one agent-task pair, variance allocation made 539 wrong selections versus 455 for uniform across 1,024 worlds. Yet mean utility lost fell 27.8%, from 0.0268 to 0.0193.

The two metrics ranked the evaluators differently. Selection accuracy counts a near tie and a costly mistake equally. Regret measures the expected utility gap between the best agent and the selected one.

With risk spread across tasks, variance allocation was worse on both metrics. This is not a general win for adaptive sampling.

The practical lesson: report how often agent selection is wrong and how costly those errors are. Lower average regret does not guarantee safety.

Independent, exploratory synthetic work; no real-agent validation. Code, data, uncertainty and reproduction instructions:
https://t.co/aXzfct7MeZ

#AgentEvaluation #AIResearch #ResearchEngineering

[Logan Mann @loganmann]

@FReza1984 Fair concern. Symphony adds no coordination layer: no locks, no queues, no approvals. It only makes the work visible. The one active signal is a warning when two people already edit the same file.

[Logan Mann @loganmann]

I built Symphony: a live map of your AI coding agents.

Symphony streams every edit, from every machine, onto one live board. It flags two agents on the same file before you get a merge conflict.

⚡ Install one hook. That is all.
🆓 Open source. MIT.

[Logan Mann @loganmann]

🔗 https://t.co/8ZTHQtoaVi

[Logan Mann @loganmann]

I built Symphony: a live map of your AI coding agents.

Symphony streams every edit, from every machine, onto one live board. It flags two agents on the same file before you get a merge conflict.

⚡ Install one hook. That is all.
🆓 Open source. MIT.

[surya @suryamidha]

roulette is basically free money if you know how to play properly. if it’s red a bunch of times in a row you know it has to be black next. also it’s always black.

[Logan Mann @loganmann]

🎉 Our paper "Where Reliability Lives in Vision-Language Models" was accepted to the ICML 2026 Mechanistic Interpretability Workshop. @icmlconf

Attention maps barely predict correctness. The signal lives later: hidden states & self-consistency. 🧠

💻 https://t.co/V1kchLz4Ai

[Logan Mann @loganmann]

@0xKaodi Would love to participate — applied on Luma!

[Logan Mann @loganmann]

@icmlconf Short summary: world models for creative writing.

Joint work with @sharmavasu55 and the @PocketFM_App research team on a latent world model and frontier-saturated benchmark for long-horizon co-creative audio drama.

Paper: https://t.co/McCgyeFMsV

[Logan Mann @loganmann]

Excited to share that "NarrativeWorldBench" was accepted to three @icmlconf 2026 workshops: HiLD, Human-AI Co-Creativity, and Culture x AI.

Joint work with Vasu Sharma at PocketFM. Presenting all 3 posters in person. Reach out if you want to meet at ICML.

[Logan Mann @loganmann]

Excited to share that I've been accepted to Y Combinator's Startup School 2026 at Chase Center, SF on July 25-26. If you're attending or will be in SF that weekend, I'd love to connect. DMs open.

[Logan Mann @loganmann]

@GurushaJuneja @NeurIPSConf Best of luck!

[Logan Mann @loganmann]

@Arnavbdwj @a16z @speedrun @daniel_dhawan @lucky_z2 @bilawalsidhu @rdominguezibar dropping the demo video below. we use large vision models to verify logistics documentation and unblock supplier payments at scale.

would love to get this on the radar for the SR007 cohort - cc @ShamnyAviv @IdoZabarsky1 @SalarShahini

#a16z #speedrun #verticalAI #fintech

[Logan Mann @loganmann]

applying to @a16z next week and would love a referral from anyone in the scout network.

building TrustAudit: an invoice acceptance engine for Indian AP teams. the wedge is Section 43B(h), India's MSME payment rule.

live demo: https://t.co/ZhlNWzaTuj