Rowan

AI Researcher; training closed large diffusion models @ frontier lab.
Also pursuing open research with open research group.

[Rowan @endosome]

@Programming1024 @askalphaxiv I would agree but my read is more that they limit the effectiveness to the ability of current frontier (opus), not that they intentionally provide misinformation + misdirect or defer to a much much weaker model. It's all black box so of course only anthropic truly knows..

[Rowan @endosome]

@askalphaxiv I don't find fable to be that much of an improvement over opus, but I am aware I might be secretly getting the peft lobotomized version with my research areas.. :/

[alphaXiv @askalphaxiv]

As believers of open research, we are disappointed to see Anthropic silently degrading Fable 5 for AI development

"Any topic related to building pretraining pipelines, distributed training infrastructure, or ML accelerator design... may have limited effectiveness through Claude via methods such as prompt modification, steering vectors, or parameter-efficient fine-tuning."

Not only do they get to decide what you use LLMs for in research, but this also enables them to silently intervene in your research without you knowing.

This sets a dangerous precedent. If a model refuses openly, users can understand the boundary. If a model falls back to another model, users can still evaluate the difference. But if a model silently modifies or weakens its own answers while still pretending to help, researchers lose the ability to know whether a failed result came from their own idea, their implementation, or an invisible intervention by the model provider.

That is not safety. Safety policies should be transparent, auditable, and user-visible.

On top of that, the people most harmed by this are not the largest labs with massive teams and proprietary infrastructure. It is the independent researchers, academic groups, startups, and open-source builders who rely on public tools to compete, innovate, and pioneer AI for everyone else.

[Rowan @endosome]

@ChrisGe05 @bfl_ml nice work; would be curious to see similar analysis of register tokens, and how this changes above results given some recent art showing diffusion benefits from them

[Rowan @endosome]

@EleaZhong Can you push to -10 😭

[Rowan @endosome]

DM me if you want to work on this

[Rowan @endosome]

RLHF/DPO / SFT layer on models seems to be a brittle shell (Xiangyu Qi et al) that doesn't survive small finetuning

Is that shell is low rank / just low magnitude? Anyone with access to midtrain, sft, RL stage weights of an LM/Diffusion Foundation model want to do some analysis?

[Rowan @endosome]

if low rank, then could be modeled with LoRA to de-RL.

.... and then how do finetune dynamics differ training in-between this stage and composing the RL/SFT (LoRA or diff) on top. New LoRA variant?

I'd assume you'd get better / faster training and aesthetic ability retention

[Harry Thasarathan @HThasarathan]

Our method reveals model-specific patterns too: DinoV2 (left) shows specialized geometric features (depth, perspective), while SigLIP (right) captures unique text-aware visual concepts:

This opens new paths for understanding model differences!

(7/9)

[Felix Petersen @FHKPetersen]

Excited to share our NeurIPS 2024 Oral, Convolutional Differentiable Logic Gate Networks, leading to a range of inference efficiency records, including inference in only 4 nanoseconds 🏎️. We reduce model sizes by factors of 29x-61x over the SOTA.
Paper: https://t.co/c4xFC0SAid

[eigenspace @eigendecay]

@yacineMTB takes activations that are kiki and makes them bouba

[Rowan @endosome]

@charles_cc_ @EMostaque because the model will essentially output a weighted average of all possibilities. The average of multiple noises is still a valid noise signal, but the average of a bunch of potential images is a blurry mess. And this is made worse at high timesteps when more task is uncertain.

[Rowan @endosome]

@charles_cc_ @EMostaque If you predict the denoised signal at each step the model has to learn to output various levels of denoised image. Predicting something fixed, like the final image or the noise itself makes the task simpler.

Predicting the noise is better than predicting the final image... (1/2)