David Andrews

Undergrad AI researcher @ Georgia Tech | LLM agents, RL, systems | @TheDuckAI

[David Andrews @broyojo42]

@notch claude code is pretty good, codex as well. if you want to try some open models, Kimi K3 and GLM 5.2 are excellent (you can even self-host them!). I wouldn't recommend vibecoding, you should review and inspect the code like it is from any colleague

[David Andrews @broyojo42]

@robertskmiles the magic of open source software

[David Andrews @broyojo42]

@zainhas opus and fable trajectories

[David Andrews @broyojo42]

@alexandr_wang where is it?

[David Andrews @broyojo42]

@neetu_arnold objective and quantitative benchmarks and evals must exist, otherwise there is no incentives for hill climbing

[David Andrews @broyojo42]

@jxnlco codex does not run autonomously or long enough, compared to claude. I will have to babysit more. also, cannot do something very creative or new compared to claude i feel, better on standard things

[David Andrews @broyojo42]

@_sholtodouglas I sometimes need more thorough response/stronger model, so I will reach for GPT 5.5 Pro Extended Thinking. Otherwise, I will be using Opus 4.7 Adaptive Thinking for 95% of tasks

[David Andrews @broyojo42]

i've noticed recently that gpt 5.5 pro uses thinking time much more adaptively, so it is pretty reasonble to use as my daily driver model. no longer need to wait 30 minutes for every query.

[David Andrews @broyojo42]

@Jonathan_Blow this is the worst it will ever be!

[David Andrews @broyojo42]

@willccbb

[Yixiong Hao @Yixiong_Hao]

We're launching an international, cross-sector Delphi study to establish consensus on conducting and reporting AI evaluations.

All critical infrastructure—from bridges and aircraft to pharmaceuticals—has agreed-upon, rigorous evaluation standards. AI systems will be at least as consequential, yet current practices are uneven, siloed, and hard to compare across organizations and contexts.

We need voices from frontier labs, auditors, academia, policymakers, civil society, and industry practitioners to create a shared reference.

[Yixiong Hao @Yixiong_Hao]

Research blog: Teaching Models to Dream of Better Monitors through Monitor Sensitive Training.

@sixers2772 , Kasey Corra, @ArchChaudhury , and I developed a post-training alignment method called Monitor Sensitive Training (MST) and validate it on toy tasks involving bias and sycophancy. We find that MST outperforms standard post training techniques when given flawed data.

MST augments each training sample with a monitor label that describe how evaluation is going to be applied. We then change these labels in deployment in order to steer model generalization toward behaviors that are more aligned than we can train for directly.

We would like to hear your feedback and ideas to apply MST to real world settings and tasks!

[David Andrews @broyojo42]

i am producing a monument to slop never seen before

[David Andrews @broyojo42]

my theory: alibaba is stopping open source releases, leading to the core qwen team resigning

[David Andrews @broyojo42]

claude being down is the new pentagon pizza tracker

[David Andrews @broyojo42]

just hit the Claude Max weekly limit after 3 days of non-stop running with Opus 4.6 + agent teams. about ~150k lines of code written, definitely starting to feel the agi

[David Andrews @broyojo42]

@natolambert @METR_Evals I believe that it will continue to scale to that length. maybe we need to measure tasks relative to those completed by teams of humans, and add up the total man-hours

[David Andrews @broyojo42]

i'm sure the RL setup for this is quite hard though, credit assignment can suffer especially

[David Andrews @broyojo42]

openai really needs to let codex recursively call itself and manage various subagents as a default, i want some more parallelism sometimes without having to manage them manually