1/ As agentic tasks grow longer and rollouts become more expensive, the choice between critic-based PPO and critic-free GRPO becomes increasingly important.
Critics can reduce gradient variance, allowing models to learn from fewer trajectories, but they also introduce additional compute costs and potential bias from bootstrapping.
Our new research @ScaleAILabs compares PPO and GRPO under a shared actor objective across 1,200+ training runs in control tasks, short-horizon math, and long-horizon terminal agents. We studied how three factors influence this tradeoff: batch size, critic reliance, and task complexity. 🧵
2026 is the year agents learn when to ask for help
Our research from @scale_AI introduces Long Horizon Augmented Workflows (LHAW), a synthetic data generation pipeline for creating underspecification on ANY dataset (yes, you can try this at home!) and evaluating how agents act🧵
1/ Creating specialized smaller models for your domain-specific agents is clearly the future, that’s why at @Scale_AI we’ve been prepping for the movement.
I’m excited to share the recent advancements we’ve made doing RL for Enterprises!
https://t.co/DlJ85oQ0EI
Congratulations to @Cisco for launching AI Defense today! We worked with Cisco to build the guardrail models that power this end-to-end solution to safeguard enterprise AI transformation.
https://t.co/YkqnSDfpu7
The latest research from Scale shows how enterprises can strategically generate synthetic AI training data with limited budgets for fine-tuning LLMs
https://t.co/Xp9tzog9yI
The @scale_AI team put a spotlight⚡️on our work "Pushing Mixture of Experts to the Limit" https://t.co/nKlJvCgsim and showcase the impact on LLaMA-2.
Really nice blog post + working implementation. 🔥
https://t.co/6lzQDna1Az
*WARNING*: Using ChatGPT to write professional documents can result in severely unfair outcomes!
More details in our #EMNLP2023 Findings paper:
"Kelly is a Warm Person, Joseph is a Role Model": Gender Biases in LLM-Generated Reference Letters
Arxiv Link: https://t.co/ortwhi3ntX