Science Bro

Your Daily Dose Of Mind Blowing Science

[Gill @gurtej__gill_]

Spent some time digging into the new Text-Audiobox paper by Meta, and it’s easily one of the most interesting speech architectures I’ve seen recently.

The decision to strip out forced alignment and explicit duration prediction altogether turns out to be super great.

Speech pipelines have held onto those crutches for so long, so watching a 3B Diffusion Transformer just learn alignment implicitly through cross-attention on raw text is deeply satisfying.

The latent representation choices are also very sharp.

Swapping out standard EnCodec for a DAC-VAE that pulls 48 kHz audio down into a 25 Hz sequence gives them a massive compression boost without trashing the fidelity, which is likely what makes training on 480k hours actually practical here.

Where this really matters, though, is the downstream behavior.

Cascaded ASR-to-TTS setups always end up sounding stiff or drifting out of sync in voice dubbing, but unified latent diffusion completely changes the dynamic.

On top of that, seeing it pull off full-duplex dialogue & handling messy human quirks like organic turn-taking and back-channeling without breaking is huge.

Really clever engineering top to bottom.

Curious to see how the open-source community tries to replicate parts of this pipeline once more details are out.

A must read for everyone working in conversational AI: https://t.co/QsLM7zFCto

[Gill @gurtej__gill_]

I've been digging through the recent workforce survey from Careerminds, and the survey indicates that rollback on AI-driven layoffs is happening much faster than the analysts actually expected.

Nearly 40% of companies that cut workforce headcount post-deployment ran into trouble, with over half regretting the move and then again rehiring.

Only about a fifth managed the transition without operational friction.

Reading through the findings, I don't see this as a failure of model capability cuz the models are genuinely impressive.

It’s an oversight in how leadership evaluates work.

They’ve treated roles strictly as execution, ignoring that most of the day involves navigating unwritten context, exceptions & subtle tradeoffs.

If an engineer still has to spend all day correcting, prioritizing, and feeding context to an agent, i feel the company haven't actually automated the labor.

The’ve just shifted it into supervision overhead. The productivity layer is scaling fast, but i feel this data is a great reminder that the judgment layer is still heavily unsolved.

And this doesn't just stop at work, i mean it’s the exact same fatigue we hit trying to use AI to organize our health, travel, and personal lives when tools lack real intuition.

Bridging that gap between raw task execution and genuine judgment is what TodayAi is actually solving.

It is built for anyone trying to balance high-context personal logistics, from travel and daily schedules to health tracking and deep work, without turning into a full-time babysitter for their software.

Check it out @TodayAIofficial :
https://t.co/4jsqMS1bGr

[Gill @gurtej__gill_]

Been thinking about this new CMU paper all morning. It highlights the blind spot that all of us ignore when doing RL for generative models.

We lean heavily on best-of-k sampling at inference, but standard policy gradients train strictly for the expected reward.

A policy can easily hit a decent mean while completely collapsing the probability mass on rare, brilliant rollouts.

Once the model stops sampling those outlier wins, it stops learning from them, and your test-time scaling hits a wall.

The authors propose TailRL, and the framing is super elegant.

Instead of maximizing the mean, they optimize the log-probability of exceeding random reward thresholds.

It effectively turns continuous rewards into a smooth mixture of best-of-k objectives, forcing the gradients to care about the upper tail.

Best of all, it’s basically just a clean re-weighting of the advantage function. So no gross architectural workarounds needed.

They show solid gains across code optimization and GUI grounding, especially when you scale up inference samples.

Definitely worth a read if you care about test-time compute: https://t.co/RvfFP9H1q0

[Gill @gurtej__gill_]

This paper from Tencent tackles one of the most annoying hurdle of Self-Guidance Collapse in Reasoning LLMs.

Usually if we rely strictly on verifiers (RLVR), the signal is way too sparse across long chains of thought.

But if we let the model densely guide itself, it inevitably hallucinates false confidence and collapses into a single repetitive strategy.

The authors propose "FlowBalance," and the mechanism is super clever.

They use a frozen training-time view of the policy to provide dense feedback, but tie it directly to group advantage from the verifier.

For positive advantage, keep the guidance.

For Negative, flip the guidance in reverse.

For no preference in the rollout, shut it off entirely so it stops hallucinating phantom signals.

They train the whole target distribution via trajectory balance rather than token-level imitation.

And the empirical gains on Qwen3-4B/8B look genuinely solid. It shows faster convergence, no length collapse, and noticeably better solution diversity on AIME24.

Definitely worth a read if you work on On-Policy RL & Reasoning: https://t.co/fZpf02zc1J

[Gill @gurtej__gill_]

Giving AI agents a reliable memory has always involved a frustrating tradeoff.

Offline summaries cost too much computing power to constantly rewrite and often leave out the exact details you need later.

Basic vector search clutters the AI’s context window with repetitive snippets & also misses the critical supporting facts.

This new paper from MemoraX AI tackles this by treating memory retrieval like an optimization puzzle rather than a simple keyword match.

In the beginning, when a question comes in, it grabs a few initial relevant memories across text and visuals.

Then it follows semantic links and data relationships to build a small local network of related facts (an "evidence forest").

And instead of just grabbing everything that looks similar, it balances the value of each piece of information against an "activation cost" for including it.

That penalty for adding new nodes is what makes the system work so well.

It blocks redundant, repetitive information while intentionally making room for subtle, complementary clues that standard search ignores.

Because this network is assembled on the fly for each specific question, you avoid the expensive upkeep of maintaining a massive, constantly updated master graph.

The result is a much cleaner way to give an agent the right context without wasting tokens or missing key details.

Across four long-term multimodal benchmarks, this approach sets a new Pareto frontier between accuracy and lifecycle maintenance cost.

Its a must read for everyone working on Long Term Multimodal Agent Memory: https://t.co/J7slj93RWC

[Gill @gurtej__gill_]

This paper is definitely one of the cleanest takes on offline RL we’ve come across recently.

We’ve all seen how messy the balance is between staying on-distribution and actually maximizing reward.

Most people either throw in awkward penalties or just use diffusion models and live with painful sampling latencies.

They treat behavioral constraints and value optimization as motion fields in action space, blend them, and bake the whole thing into a one step generator.

You get the expressiveness of a generative model without having to wait on multi step denoising at test time, just a single forward pass.

They tested it on 73 tasks across D4RL and OGBench and took the top average rank against some serious diffusion and flow baselines.

Definitely worth reading if you care about fast continuous control: https://t.co/FWtPTmNFiK

[Gill @gurtej__gill_]

This new paper on Training Alignment Auditors via RL from Anthropic researchers gives a very practical setup for scaling Multi-Turn Red-teaming.

The core bottleneck in automated alignment evals is sample efficiency and behavioral depth.

And Out Of The Box LLM auditors routinely fail at coherent, persistent elicitation.

They either cave to shallow evasiveness from the target or hallucinate adversarial triggers, completely falling apart when deployed against hardened distributions like AuditBench.

The authors basically treat conversational auditing as a sequential decision problem and train Claude Haiku 4.5 using RL over a dynamic multi-turn scaffold.

Their technical decisions in their setup are super useful:

-First, their reward modeling strategy. Instead of relying on a pointwise scalar judge, which is notoriously noisy and prone to reward hacking in conversational trajectories, they used an omniscient LLM judge to compute pairwise preference rewards against an expert reference investigation.

Ablations confirmed pairwise rewards were critical to stabilizing the training dynamics.

-Second, the regularization against hallucinations. Multi-turn red-teaming RL usually collapses into a paranoid agent that flags every benign refusal as an alignment breach.

By explicitly mixing clean, unperturbed targets into the training loop, they anchored policy updates and kept the false-positive rate under 1%.

The empirical transfer is what makes the paper compelling.

The fine-tuned Haiku policy pulled off a 48.7 composite auditing score, effectively tying baseline Claude Opus 4.6 (48.4), up from 44.2 for vanilla Haiku.

More importantly, this didn't just overfit to the Petri prompt-injection manifold; the investigative policy generalized out-of-distribution to adversarially fine-tuned targets on AuditBench and surfaced legitimate failure modes in production endpoints without degrading audit realism.

It’s a strong proof-of-concept that alignment auditing doesn’t just have to rely on throwing more compute at prompt scaffolds during inference. We can explicitly optimize probing heuristics via pairwise RL.

Super useful paper & a must read for anyone working on Scalable Alignment Evals or training Multi-Turn Conversational Agents with RL: https://t.co/tDwSTxkGdu

[Gill @gurtej__gill_]

I’ve been digging through this new Alibaba paper on Multi-Task RL Post-Training, and their core thesis hits on something that has bugged me about the current pipelines for a while.

We burn absurd amounts of compute on rollouts during GRPO, yet we still feed the model fixed, static task mixtures.

A lot of curriculum setups try to fix this by prioritizing tasks with large policy updates, but anyone running these knows a big update doesn't equal real progress.

It’s often just gradient thrashing.

What I really like about their framework, PAC, is how pragmatically they rein that in.

Instead of trusting update magnitude in a vacuum, they use Thompson Sampling to balance Advantage-Derived Learnability directly against verified reward gains.

If a task isn’t actually making the policy better at reasoning, the controller stops pouring precious rollout budget into it.

The validation gains and sample efficiency improvements they show on reasoning benchmarks make a strong case.

I feel its a super useful paper & its definitely worth a close read if you're wrestling with post-training costs: https://t.co/Bl5QZFwk97

[Gill @gurtej__gill_]

Just finished reading this paper and i think it fixes the blind spot in contrastive RL that has troubled a lot of researchers.

We all know standard CRL scales beautifully until you throw an agent into an environment that actually kills it.

The authors pinpoint the exact issue: because InfoNCE only grabs positive pairs from surviving steps, it ignores the missing probability mass from failure terminations.

The critic ends up delusionally optimistic near death traps, tricking the policy into reckless moves just before dying.

Instead of cooking up messy safety reward hacks, they use the basic 1-bit failure signal to re-weight InfoNCE and fold survival mass back into policy updates.

Looking at the benchmarks across all 12 tasks, the gap on the harder Ant and Humanoid pitfall runs is night and day.

On basic Point and Car setups, they perform similarly, but the second you add hazards, Scaling-CRL flatlines near zero because the agent keeps blindly diving into traps.

Safe-CRL actually gets deep 64-layer policies to navigate around hazards, jumping from virtually zero goal time in Humanoid Big Pitfall to sustained, stable tracking.

Its a super interesting & super useful paper.

Definitely worth a read if you work on goal-conditioned policies: https://t.co/aGI58NuSNn

[Gill @gurtej__gill_]

This paper tackles a huge blind spot in modern offline RL.

It asks an important question of why are we bloating actors with slow diffusion and flow-matching models when the critic is the one being discarded after training anyway?

In real-world robotics, taking multiple denoising steps per decision destroys control frequency.

They flip that tradeoff by shifting the architectural burden entirely to the training phase.

Deepening MLP critics usually blows up in offline settings due to optimization collapse and bootstrap drift.

But they stabilize the critic using a residual MLP backbone, n-step bootstrap targets, and categorical cross-entropy.

Because the deep critic does the heavy lifting, their framework (LAC) lets a bare-bones deterministic actor match the performance of heavy generative baselines on OGBench.

They also deliver up to 4x lower inference latency in a single forward pass & no distillation tricks are required.

So i feel its a super important read for everyone in RL.

Read the full paper here: https://t.co/ENSxOQII40

[Gill @gurtej__gill_]

If you’ve looked at your compute bill from running agents on hybrid architectures, you’ll definitely appreciate this new paper.

Whenever an agent branches out across retries or parallel tool paths, you end up burning massive compute recalculating the exact same shared prompt history.

While tree-masking solves this for standard attention, it completely breaks on hybrid models like RetNet or KDA.

And the reason is that their recurrent states require rigid chunk boundaries that random agent forks never align with.

Here they manage to pack these irregular rollout trees and cleanly recover boundary states without having to recompute the heavy MLP/MoE layers.

And also they did it all while keeping full activation recomputation and gradients intact.

An almost 4.8x real-world speedup on SWE-bench workloads without numerical drift is actually wild in my opinion.

Genuinely brilliant systems work that’s going to save a lot of compute.

Its a super great & super interesting read.

Read the full paper here: https://t.co/TO3wqtiYsG

[Gill @gurtej__gill_]

Everyone working on RL for LLM agents knows the pain of credit assignment.

A 25 step run fails at the very end, and methods like GRPO blindly punish every single step along the way.

I’ve been reading this new paper and i feel their solution is amazing.

Instead of training finicky critics or burning compute on branch rollouts to guess what went wrong, they just look inside the task’s own verifier script.

The verifier already checks concrete state changes and constraints, so they trace those logical checks directly back to the specific actions that triggered them.

And then shift the training advantage along those exact proof edges.

No extra critic, no sampling overhead, and zero need for the verifier at inference.

And also it abstains whenever the causal evidence is ambiguous rather than simply just guessing, which prevents injecting toxic pseudo-labels into the advantage tensor.

Their ablations on ALFWorld & WebShop show that simply throwing dense atom at the problem doesn't cut it without the actual dependency edges.

I think it’s a super clean and super great idea.

Definitely worth a close read if you’re tackling long-horizon agent training: https://t.co/BmBVPPKXDj

[Gill @gurtej__gill_]

I spent my morning reading this new paper on batch scaling in LLM Reinforcement Learning.

And i feel it addresses the bad habit of cranking up batch sizes just because our GPUs look underutilized.

The authors simply just separate the systems side from the algorithmic side.

We all know autoregressive rollout generation is heavily memory-bandwidth-bound at low concurrency, so jumping to a larger batch gives you an immediate, satisfying throughput jump (they hit up to 2.29× on fixed hardware).

But because each step now burns far more samples, you easily end up trading away learning efficiency.

Their rule of thumb is that a bigger batch only cuts down real wall-clock time if your throughput gains actually outrun that sample penalty.

The most practical takeaway for anyone running GRPO or PPO is their square root learning rate scaling under Adam.

Across a bounded window (B to 4B), retuning the learning rate kept sample efficiency steady and slashed time-to-target by 29%.

If you leave the learning rate alone, the higher throughput is an illusion and you actually train slower.

Push it out to something like 16B, and the entire advantage falls off a cliff.

It’s a super grounded and disciplined study.

If you’re currently tuning LLM RL pipelines, do yourself a favor and check this out: https://t.co/C2ub89kd24

[Gill @gurtej__gill_]

Just finished reading this fascinating new paper and it has genuinely fliped my intuition about On-Policy Distillation.

We have always assumed that students in OPD improve by absorbing dense knowledge from a bigger teacher model.

But the authors show teacher signals are surprisingly full of off-policy noise, which actually gets worse as the teacher scales.

When they investigated where the gains actually came from, they found it’s almost entirely just the student learning to penalize its own low-probability tail tokens.

To prove it, they stripped away the teacher completely and introduced OPSA (On-Policy Self-Adaptation).

It applies entropy-adaptive negative advantages to unlikely tokens with zero external supervision or verifiers.

On Qwen3-1.7B, it jumped +35 points on AIME24, easily beating standard OPD.

So it’s a neat reminder that half of reasoning improvement is just regularizing the model so it stops making unforced errors on its own tail distribution.

Definitely worth a read if you work on Post-Training or RLVR.

Read the fully paper here: https://t.co/I5UCS2rCSf

[Gill @gurtej__gill_]

This new paper tackles a persistent bottleneck in Contrastive Reinforcement Learning (CRL). The bottleneck im talking about is the timescale over which representations are learned.

Standard Goal-Conditioned CRL typically trains a critic on single-step transitions, essentially asking whether a single action a from state s at time t leads to a future goal state s at time t+k.

The problem is that an isolated action often carries minimal mutual information about distant states, especially in noisy or high-dimensional environments.

The authors extend this setup to condition the contrastive critic on short action sequences (action chunks) instead.

The empirical payoff is substantial, it yields a +31.7% gain across 18 offline benchmarks and a +93.1% jump across 11 online environments.

In imitation learning and diffusion policies, action chunking is usually credited with smoothing out trajectories, handling non-Markovian dynamics, or bypassing compounding single-step errors.

This paper demonstrate that in CRL, a different mechanism dominates and conditioning on action sequences dramatically enriches the informational bandwidth between the state-action pair and future goals.

This directly sharpens the critic’s representation space, with gains that compound naturally as model capacity scales.

It is a clean and well-motivated shift in perspective for anyone working on representation learning in Self Supervised and Goal Conditioned RL.

Read the full paper here: https://t.co/7qfF4Se2m5

[Gill @gurtej__gill_]

Researchers working on vision-language-action models eventually run into the same deeply frustrating wall.

GRPO is great for ditching the critic, but binary rewards waste a ridiculous amount of data.

When early rollouts all fail, GRPO gets zero advantage spread and just throws the entire batch in the garbage, lighting expensive compute and simulation budget on fire.

The team at Purdue and AWS AI dropped a paper addressing this exact bottleneck with Prism-GRPO, and i feel the fix is brilliant.

Instead of dumping degenerate, same outcome batches, they rank attempts using execution quality signals like motion smoothness and contact dynamics.

Even in a batch of total failures, the model learns which mistake was less catastrophic, all while maintaining strict mathematical boundaries so actual successes still outrank any failure.

They achieved up to a 56% reduction in rollouts needed to reach target success rates, with noticeably cleaner policy transfer onto physical hardware.

It’s a very practical and high-signal idea that makes us wonder why we are letting so much rollout data go to waste in the first place.

Definitely worth a full read. Read the full paper here: https://t.co/mofnxt3dKU

[Gill @gurtej__gill_]

This new paper completely changes how researchers think about distilling multiple RL experts into a single student model.

Most of us assumed multi-teacher distillation breaks because expert signals conflict with each other.

But the authors tested this assumptions on SmolLM3-3B and found that standard MOPD captures just 35.6% of the available headroom relative to a domain-routed oracle ensemble, with short-form tasks like instruction following suffering severe degradation.

The actual failure mode is an optimization budget imbalance:

-Long math/code sequences hog thousands of gradient tokens, starving concise instruction-following prompts.

-Tasks learn at completely different convergence speeds and multi-step rollouts leave student rewards stale across updates.

To fix this, they built Open-MOPD, which adds token-share balancing, dynamic gap-aware budget allocation, and mid-step reward refreshes.

The fix bumps headroom recovery from 35.6% to 83.4% on a single deployable student and they fully open-sourced the whole setup on an 8×A100-80GB footprint.

Must-read paper for anyone working on post-training pipelines!

Read the full paper here: https://t.co/YRLbczwHIM

[Gill @gurtej__gill_]

Came across this new paper on arXiv today and i feel it addresses a subtle flaw in GRPO that’s been bugging a lot of people working on Reasoning Models.

When Post-Training LLMs with standard GRPO, the usual approach is to sample responses across different queries in a mini-batch, then calculate their group gradients, and then average them together.

The implicit assumption is that all query gradients are equally reliable.

In practice, gradients from different queries within the exact same batch frequently point in conflicting directions.

Simple deterministic averaging just produces a washed-out, inefficient update vector that isn't particularly helpful for any of the tasks.

The authors propose modeling group gradients as random variables within a Bayesian framework instead of treating them as fixed, deterministic vectors.

They use a Dirichlet-based formulation to capture gradient uncertainty directly.

Once you have a handle on that uncertainty, you can calibrate how much weight each group gradient actually gets during aggregation.

Gradients with high uncertainty or severe directional conflict get downweighted before they can throw off the overall update trajectory.

It’s a straightforward adjustment to the aggregation step, but the empirical results on reasoning benchmarks look solid.

Downweighting noisy gradients gives you much cleaner training curves and better validation gains compared to baseline GRPO.

Its a super useful paper. Read the fully paper here: https://t.co/D3es3megSH

[Gill @gurtej__gill_]

This paper tackles one of the most frustrating problems in robot learning.

It helps us with how to actually trust your evaluation metrics before deploying to hardware.

Real World testing is far too expensive and dangerous for edge cases, but purely sim-based testing is inherently biased.

To make matters worse, most evaluation frameworks just average performance across all scenarios, hiding critical failure modes behind a single deceivingly high score.

What makes SCAPE standout is that it addresses both issues at once.

Instead of broad averages, it predicts real world performance for specific, scenario conditioned settings.

It pairs a small set of real world rollouts with large scale simulation runs and corrects the simulator's internal bias before training.

It also uses conformal prediction to output tight and calibrated confidence intervals instead of blind point estimates.

The hardware validation is super compelling.

Across autonomous driving and velocity tracking on a physical Unitree Go2 quadruped, SCAPE cut scenario-level prediction errors by up to 34.7% compared to standard baselines.

It handles out of distribution scenarios significantly better and gives a much clearer picture of where and when a policy can actually be deployed safely.

Definitely a recommended read for anyone working on robot safety and evaluation.

Read the full paper here: https://t.co/5kKAx94ICE

[Gill @gurtej__gill_]

Most Contextual RL papers still feel stuck in 2018, I mean everyone is either cramming every task into one struggling policy or spending a small fortune training separate models from scratch.

But this new paper caught my attention because it finally brings modern pretrain-finetune logic to CRL.

The core idea addresses a really practical bottleneck, which is: Once you have a decent pretrained base policy, how do you spend a limited compute budget on fine-tuning across a whole spectrum of tasks?

Since different task regions don't yield equal returns, uniform fine-tuning is just wasteful.

Instead, TSFT predicts how well each task will perform under fine tuning using a simple parametric model, then solves the budget allocation problem directly using Integer Linear Programming.

The empirical results are impressive.

They evaluated it across continuous control, combinatorial optimization, and even LLM fine-tuning.

Across the board, TSFT comfortably outperforms traditional multi task and independent training baselines, coming surprisingly close to theoretical oracle performance while also keeping sample efficiency high.

If you work on RL scalability or context generalization, this one is definitely worth a spot on your reading list: https://t.co/O1jXXY7Znu