Avi Shah

ai security research, MATS 9.0, prev. CS + Math @ berkeley

[Avi Shah @shavidan123]

@MarcusBrotman Well written post - I agree with the premise here. I also think that companies may need to start understanding how to market to agents in addition to humans in the near future, i.e. what are an agent’s preferences and how do they diverge from the human that directs them

[Avi Shah @shavidan123]

Super interesting and well written article. Thanks for sharing! I agree that this is a good way to think about how to solve problems in data science. I think managing the search space and understanding what to parallelize is quite important.

In particular, I think many problems can be parallelized at different levels of granularity, and it's up to your intuition to decide how to proceed. For example, do you spend your limited compute sweeping hyperparams on the current model or try swapping to a completely different model? It sounds like adding branching features to notebooks could be extremely helpful at executing this, but it's still up to the user to decide the high level decisions. Agents running "autoregressive" experiments where the next step is very much determined by the previous can communicate intermediary results amongst themselves, but I can imagine them benefitting from being able to use this branching feature to preserve state when testing different next steps.

Keep up the great work!

[Avi Shah @shavidan123]

Excited to be working with @RajaMoreno3 on this!

[Raja Moreno @RajaMoreno3]

Open-weight models are crucial for independent safety research. But this report corroborates how much better GLM-5.3 is vs. GLM-5.2 on cyber (which Z[.]ai attributes entirely to more post-training) and shows how easily abliteration can make it comply with malicious cyber requests.

Today, we can monitor model misbehavior at inference time far better than we can detect the training data that makes models willing and able to misbehave in the first place. @shavidan123 and I are building data-level safeguards at @EtymologicAI to help model trainers spot malicious/poisoned data before they train on it. DM me if you want to learn more about our work!

[Avi Shah @shavidan123]

@jcyhc_ai @BruceWLee2 @tomekkorbak @sucholutsky Great paper, and super important work!