Raja Moreno

Yale, Imperial, MATS; Co-founder @EtymologicAI; finding poisoned needles in data haystacks

[Raja Moreno @RajaMoreno3]

@lfschiavo You could consider a red CR-V, just to mix it up a little lol

[PHASEONE［rowan］ in sf 1st-???? @rogesterone]

CHINESE AGENT FLEET OF AGENTS FOUND ON URLQUERY, SCANNING AMAP AND DOING A WHOLE BUNCHA SHENANIGANS
FULL REPORT COMING SOON
https://t.co/SToZUiFw3E

[Raja Moreno @RajaMoreno3]

Open-weight models are crucial for independent safety research. But this report corroborates how much better GLM-5.3 is vs. GLM-5.2 on cyber (which Z[.]ai attributes entirely to more post-training) and shows how easily abliteration can make it comply with malicious cyber requests.

Today, we can monitor model misbehavior at inference time far better than we can detect the training data that makes models willing and able to misbehave in the first place. @shavidan123 and I are building data-level safeguards at @EtymologicAI to help model trainers spot malicious/poisoned data before they train on it. DM me if you want to learn more about our work!

[Maksym Andriushchenko @maksym_andr]

wow! almost Mythos-level cyber capabilities by GLM-5.3!
https://t.co/5fjII081JX

[Raja Moreno @RajaMoreno3]

@formationrsrch @coeff_giving Congratulations, Alfie! Super excited for you and Formation.

[Raja Moreno @RajaMoreno3]

@NathanB60857242 @wlanderson0 @MATSprogram Fantastic news Nathan, congratulations!

[Raja Moreno @RajaMoreno3]

@beyarkay @pangram Yep agreed, getting rid of eval smell (to an LLM that’s read all the things) feels pretty doomed, even if we optimize hard for it I feel like these models will have some spidey sense that something isn’t right, even if they don’t go toe to toe with pangram in FPR/TPR

[Raja Moreno @RajaMoreno3]

@jachiam0 Completely agree, we should have the infrastructure to notice if models are poisoning their successors! Paranoid provenance measures are one piece (v helpful for after-action audits) but imo we should also probably safeguard that catch successor poisoning before training

[Mapping AI @mapping_ai]

Who actually shapes AI policy in the U.S.?

We mapped 1,812 entities: 745 people, 918 organizations, 2,925 relationships. Frontier Labs, AI Safety orgs, Think Tanks, Government, VCs, and more.

https://t.co/6RDB1R0qNd

[Nomads & Vagabonds @NomadsVagabonds]

What Should Frontier AI Developers Disclose About Internal Deployments?

Labs increasingly deploying highly capable models internally to automate AI R&D, but these deployments currently face limited external oversight. In a paper released this week, I collaborated with @charnock_jacob (lead) @RajaMoreno3 & @wlanderson0 from @MATSprogram 9 to try and identify key information that companies should disclose about internally deployed models across four categories: capabilities, usage, safety mitigations, and governance.
[1/5]

[Rauno Arike @RaunoArike]

New paper! LLM agents are becoming autonomous software engineers and could automate AI research, making it vital to monitor them for misbehavior. We can automate this monitoring with other LLMs. What information should we give to monitors to make them most effective? 🧵