Jules Gagnon-Marchand

Generative models for biology and health @ Altos Labs

[Jules Gagnon-Marchand @julesgm4]

@karpathy imports in the function also..

a surprising fraction of people argue that exceptions in Python are hard to read and that user facing errors should log then exit, vs raise exceptions.

[Jules Gagnon-Marchand @julesgm4]

@lateinteraction continuous, low level empathy and projection maybe, "what is this person doing / trying to achieve, what are they feeling"

[Jules Gagnon-Marchand @julesgm4]

@alexalbert__ having a button to systematically trigger search without having to ask the model to use its search tools every time would be nice

[Stella Li ✈️ ICML🇰🇷 @StellaLisy]

⚙️ Looking closer into GRPO: there is a "clipping bias" that amplifies high-prior model behaviors.

Code reasoning could be one of the magical behaviors for Qwen-Math💻

Empirically, we disabled clipping (fig.)-the gains disappeared‼️

[Jules Gagnon-Marchand @julesgm4]

@oh_that_hat I think a leg doesn't have memory of it's own internal state, & instead at best has residual internal effects of a previous internal state (which is different & I don't think I would qualify as consciousness)

[Jules Gagnon-Marchand @julesgm4]

@oh_that_hat and not so much a computation of the current state of its senses + its own internal previous computational state. it's sensory memory vs memory/ understanding of a level of of its computational state.

[Jules Gagnon-Marchand @julesgm4]

@HannesStaerk Hello Hannes, was this recorded?

[Nathan Lambert @natolambert]

DeepSeek makes it quite clear how they trained R1.
None of these steps alone are super surprising, but how to sequence and blend them together definitely is.

[Jules Gagnon-Marchand @julesgm4]

@natolambert I also wonder if you leave some performance on the table with RLVR because of this, where some of the learning potential on the data is wasted on trying to deal with the prompting format

[Jules Gagnon-Marchand @julesgm4]

@natolambert it makes the models trained with open-instruct perform worse on HuggingFace Lighteval for example

[Jules Gagnon-Marchand @julesgm4]

@xhluca maybe you just ask the llm reranker I guess

[Jules Gagnon-Marchand @julesgm4]

@xhluca I was thinking about the inverse problem. how do you detect if there's nothing interesting to retrieve? that's also an important problem

[Jules Gagnon-Marchand @julesgm4]

@xhluca (not following this sub field anymore, it's possible that it all already exists)

Also Salut Xing :))

[Jules Gagnon-Marchand @julesgm4]

@xhluca also detecting which direction in a scientific area is unexplored

[Jules Gagnon-Marchand @julesgm4]

@arianTBD @srush_nlp Hello Arian, do you have a link to the talk?

[Epoch AI @EpochAIResearch]

1/10 Today we're launching FrontierMath, a benchmark for evaluating advanced mathematical reasoning in AI. We collaborated with 60+ leading mathematicians to create hundreds of original, exceptionally challenging math problems, of which current AI systems solve less than 2%.

[Jules Gagnon-Marchand @julesgm4]

@lvwerra