Smells Like ML

Building #ExperimentOps @remyxai
Experiment orchestration for AI teams

#XO #BeAnExperimenter #WhatsNext

[Smells Like ML @smellslikeml]

In that benchmark comparison, do you even have the sample size to compare two models, or are you making decisions based on statistical noise?

2605.30315v1 offers a simple test
Outrider added it to our fork of lm-evaluation-harness

Install Outrider: https://t.co/apCnaQRZAW

[Smells Like ML @smellslikeml]

pip install remyxai → discover research and turn it into reviewable PRs
https://t.co/UppiG8G98C

[Smells Like ML @smellslikeml]

In that benchmark comparison, do you even have the sample size to compare two models, or are you making decisions based on statistical noise?

2605.30315v1 offers a simple test
Outrider added it to our fork of lm-evaluation-harness

Install Outrider: https://t.co/apCnaQRZAW

[Smells Like ML @smellslikeml]

@artists_voyage Especially insidious for segment-level error analysis

You slice into subgroups to improve over a baseline, the CIs widen with each cut, and decisions made off the point estimates end up chasing noise.

[Smells Like ML @smellslikeml]

Now @remyxai Outrider is on @github Marketplace!

Schedule Claude to implement core methods from the most relevant papers for your repo using Github Actions!

https://t.co/JqE1DKegF5

[Smells Like ML @smellslikeml]

'CrossView Suite' introduces CrossViewBench, focusing on explicit alignment mechanisms and object-level consistency across views offers a strong framework for evaluating the fidelity of VQASynth's synthetic data and improving the robustness of the generated spatial questions.

[Smells Like ML @smellslikeml]

Outrider is the GitHub Action that matches new arXiv methods to your repo and drafts the PR

https://t.co/apCnaQRZAW

[Smells Like ML @smellslikeml]

@remyxai @AnthropicAI Coming soon: evals on every PR. Your benchmark suite + datasets run against the diff, results linked to PR before setting ready for review.

Design partner pilot opens soon — DM if interested.

[Smells Like ML @smellslikeml]

Under the hood: arXiv → @RemyxAI ranks weekly against your team's commit history → @AnthropicAI Claude Code drafts the integration → Outrider opens a draft PR with tests

[Smells Like ML @smellslikeml]

https://t.co/pTmRDMPBsb

[Smells Like ML @smellslikeml]

When code evolves, developers signal an implicit preference for the new over the old.

Scale that analysis across many repos and patterns emerge.

Taste is learnable too, even if OAI hasn't figured out selection yet.

[Greg Brockman @gdb]

taste is a new core skill

[Smells Like ML @smellslikeml]

Check out the dataset for VQASynth
https://t.co/KgXhkjHsjO

[Smells Like ML @smellslikeml]

https://t.co/LmEDVeZGHF

[Smells Like ML @smellslikeml]

Uses a Gaussian Process to learn contributor preferences implicitly from repo merge history

Next, I applying the GP to synthesize a larger volume of preference data to help finetune an open-weight coding model with DPO and LoRA.

https://t.co/8bqDhtqdTx

[Katarzyna (Kasia) Kobalczyk @kasia_kobalczyk]

Another paper accepted! #ICML2026 here we go 🇰🇷

[Smells Like ML @smellslikeml]

The space of possible improvements to your AI model is large while evaluation is costly

LILO learns efficiently from decision maker's preferences, balancing exploration and exploitation in a principled way w/ Bayesian Optimization

https://t.co/rpW3323Col

[Smells Like ML @smellslikeml]

used it to rank 11 arXiv papers by alignment, with spatial-reasoning benchmark and novel-view papers topping the list