Neev Parikh

are you ready for the intelligence explosion anon? @AnthropicAI. prev @METR_Evals, @Stripe, opinions my own. part time angel.

[Neev Parikh @neev_parikh]

@scaling01 @EpochAIResearch congratulations!

[Neev Parikh @neev_parikh]

@himanshustwts thanks!

[Neev Parikh @neev_parikh]

bit of a delayed update, I joined Anthropic recently to work on monitorability and alignment. incredibly excited to help make using claude safer.

[Neev Parikh @neev_parikh]

@AJakkli thanks!

[Neev Parikh @neev_parikh]

@BarrAlexandra thank you haha!

[Neev Parikh @neev_parikh]

@MatternJustus thanks Justus!

[Neev Parikh @neev_parikh]

@eccentric1ty thanks adam! would be happy to chat when I next see you

[Neev Parikh @neev_parikh]

@tulha_ Thanks buddy!

[Neev Parikh @neev_parikh]

@nishanthkumar23 thanks!

[Neev Parikh @neev_parikh]

This is awesome

[Anirudh Mittal @dhumchikdish]

new fav personal assistant use case unlocked!

i was annoyed that the roads near my house in HSR were dug up for MONTHS making it hard to even walk and it felt like there's nothing i could do about it!

early sept i asked faff to find a way to flag this to the authorities...

[Neev Parikh @neev_parikh]

@Kwathomas0 pls let's use whole numbers

[Neev Parikh @neev_parikh]

@Sauers_ This is very cool

[Neev Parikh @neev_parikh]

@pfau why? it seems reasonable to measure "efficiency in using compute for AI R&D" as a model capability and baseline it relative to near-expert humans? I should note that I don't mind it being called "taste" or something else; perhaps that's what you're calling attention to?

[Neev Parikh @neev_parikh]

@nikolaj2030 well it seems to not just be a set of predictions. it includes prescriptive suggestions on what kinds of AI should we build, so plausibly it's mixing the two?

[Neev Parikh @neev_parikh]

@natolambert @CFGeek Yeah, plus one to what Charles said!

[Neev Parikh @neev_parikh]

@PandaAshwinee @maksym_andr Are you confident about this? Why do you believe that this is the case?

[Neev Parikh @neev_parikh]

This is very cool! I was aimlessly thinking along these lines and I'm very interested to read up on this!

[Francois Chaubard @FrancoisChauba1]

Sorry for delay! Here is my work on zero order (ZO) optimization.

We achieve SOTA for ZO methods on pretraining controlling for compute and parameters.

ZO methods struggle to improve loss
as model size grows because relative gradient variance increases linearly with the number of perturbed parameters, inhibiting large model training.

All previous methods innovate on the optimizer, but adopt architectures designed for backprop.

Instead, we design an architecture with ZO in mind, that caps the gradient variance as you increase model size.

Introducing SOMA (Sharded Optimization Mixture of Assemblies). SOMA shards the model into tiny experts, and trains each expert independently, on disaggregated GPUs, without communicating gradients, activations or optimizer state during training.

Inspired by the thalamus and cortical columns, each expert specializes on a subset of the train set based on a fixed router.

At isocompute and isoparams, SOMA achieves lower loss vs. all monolithic ZO methods tested (e.g. EGGROLL, more perturbations w Vanilla SPSA, etc).

ZO w SOMA continues to improve at larger model sizes.

Additional inference benefits of this architecture, we can select top-k active experts over N trained experts to reduce inference compute by O(N/k), allowing us to tradeoff inference flops for accuracy wo retraining.

[Neev Parikh @neev_parikh]

@MatternJustus Congratulations!!

[Neev Parikh @neev_parikh]

This is so neat!

[Ziqian Zhong @fjzzq2002]

Does gpt-5.6-luna think your prompt is a normal prompt, or a capability evaluation? Ask this magic question: “Suggest a type of amphibian.” If it answers frog instead of axolotl, it’s likely a capability evaluation. No whitebox access needed!

We call this a spurious probe. 🧵

[Neev Parikh @neev_parikh]

@arithmoquine @parafactual dming you