@danielrupawalla Could trying to remove eval-awareness signals actually select for models that are better at detecting subtler ones? I wonder if there’s an adversarial-training dynamic where increasingly 'stealthy' evals just produce increasingly sophisticated situational awareness.
Setting up my @OpenAI Dot. Why is a fantasy sports plugin grouped under Productivity? Wouldn't something like "Counter-productivity" be more appropriate?
@fchollet If this is right, isnt the implication that we spent years scaling the wrong thing? 100,000x scale barely moved ARC, then changing the training objective basically solved it. That seems like a much bigger story than "reasoning models are better."
Mustafa Suleyman is extremely underappreciated as an AI thought leader. Love his point here: if you train a model to narrate a self, preferences and moral status, then treat those outputs as evidence of an inner life, you’ve created an endlessly self-reinforcing epistemic loop.
https://t.co/lxtKPGIw3k
The irony is that medical evals probably matter more for a general-purpose model than say terminal bench, not less. Regular people will use Gemini for health questions regardless of how it’s positioned.
One issue is the quality of medical benchmarks just tends to be lower. With more realistic more longitudinal clinical workflow benchmarks coming out, I think we could see medical evals become an unusually good stress test for calibration, long-context reasoning, evidence synthesis, and knowing when to defer.
An interesting tension this NYT piece highlights: under fee-for-service, AI that prevents an unnecessary visit can cost a hospital revenue. AI that collects more for that visit has a much easier ROI story. "Prove the ROI" sounds like a check on AI hype, but it can favor the tools that make care more expensive.
https://t.co/rrCan0Ghv0
@himanshustwts Reminds me of the 2023 Reddit study where ChatGPT was rated more empathetic than doctors, but its replies were ~4x longer. I think a length-matched comparison should be standard in these evaluations, so we can see how much of the gap survives
Interesting detail in @OpenAI's MentalHealthBench: clinician-written replies scored below most models but incurred fewer rubric penalties. The authors suspect their shorter, conversational style plays a role.
Reminds me of the 2023 Reddit study where GPT-3.5 was rated more empathetic than doctors, but its replies were about 4x as long. I’d really like to see these comparisons with response length held constant.
Mainly long-context training on clinical agent traces. A recent Qwen3.5-9B run I did used ~36M training tokens. On Tinker I think that would have cost me about $50 (1.4/M train), vs about $11 in GPU charges on a $2.09/hr RTX Pro 6000. (not huge sums ofc, but this is a personal hobby project and I usually have a few runs like this lined up)
The other issue is context length: many of my traces are >100K, while Qwen3.5-9B on Tinker is currently 64K.
That said, I've found Tinker great to use overall, and Inkling-Small has been especially nice to work with. I'd definitely move more of these runs back on Tinker if the economics were a bit closer.
@rhythmrg Between clinical evals at Ambience and my own RL work with simulated primary care, an interesting problem Ive seen is credit assignment. If a clinician fixes an output before accepting it, how do we avoid rewarding the model/system for the clinician's contribution?
@willdepue Been working on RLVR in a simulated primary care clinic. A hybrid seems promising: learned models for messy patient interactions, explicit state for orders and meds, and independent reward checks. Only way to expand beyond hand-authored scenarios.
@KatieS I've dealt with pieces of this in various contexts, and one important component of a general solution: permissions should be scoped to the actual task. An agent reconciling meds shouldn't inherit the clinician's full EHR privileges