Weihao Zeng

PhD Student @hkust @hkustNLP | General Agent @deepseek_ai | LLM Agent | Ex-intern @Kimi_Moonshot @MSFTResearch | Author of SimpleRL, Toolathlon, LOCA-bench

[Weihao Zeng @AndrewZeng17]

🧠 Long-running agents quietly fail as context grows.

Even with 100K–1M token windows, reliability degrades — plans drift, constraints are forgotten, exploration collapses.

We introduce LOCA-bench, a benchmark designed specifically for long-context, long-horizon agents.

Github: https://t.co/YK2xcni0wK
Paper: https://t.co/TgtRHuOijo

[Weihao Zeng @AndrewZeng17]

@sivil_taram thanks!!!🥰

[Weihao Zeng @AndrewZeng17]

Try our model on general agent tasks —making intelligence more accessible to everyone 🥳🥳🥳

[DeepSeek @deepseek_ai]

🚀 DeepSeek-V4-Flash Official API is now LIVE in public beta!

🔷 We’ve massively upgraded its Agent capabilities—benchmark scores are now far surpassing the V4-Pro-Preview. Check out the massive performance leap below! 👇
🔷 The official V4-Flash now natively supports the Responses API format and is fully adapted for Codex!

Check out the configuration details in our official API docs: https://t.co/smCwQZMeiq

[Weihao Zeng @AndrewZeng17]

Hello Guohao, our HKUST NLP Group also has another project, LOCA-bench, which previously implemented a large number of MCP servers, including Calendar, Canvas, Email, Google Cloud, Google Sheets, Snowflake, and WooCommerce, among others.

Feel free to use them😀

Link: https://t.co/XyBPi1EvIC

Paper: https://t.co/AOPb9cv7IU

[Weihao Zeng @AndrewZeng17]

Our LOCA-becch now support Sandbox mode，a client-server architecture that separates concerns for better debugging, isolation, and scalability！

Check it here: https://t.co/RL39wYfi2w

[Weihao Zeng @AndrewZeng17]

🧠 Long-running agents quietly fail as context grows.

Even with 100K–1M token windows, reliability degrades — plans drift, constraints are forgotten, exploration collapses.

We introduce LOCA-bench, a benchmark designed specifically for long-context, long-horizon agents.

Github: https://t.co/YK2xcni0wK
Paper: https://t.co/TgtRHuOijo

[Weihao Zeng @AndrewZeng17]

@kakakbibibi Great bro👏👏👏

[Weihao Zeng @AndrewZeng17]

Add Kimi-K2.5 and GLM-5 results

Very pleased to see that the newly released models have made progress on LOCA-bench compared to their previous versions 👏👏👏

[Weihao Zeng @AndrewZeng17]

🧠 Long-running agents quietly fail as context grows.

Even with 100K–1M token windows, reliability degrades — plans drift, constraints are forgotten, exploration collapses.

We introduce LOCA-bench, a benchmark designed specifically for long-context, long-horizon agents.

Github: https://t.co/YK2xcni0wK
Paper: https://t.co/TgtRHuOijo

[Weihao Zeng @AndrewZeng17]

🙌🙌🙌

[Yuzhen Huang @yuzhenh17]

Long context window ≠ long-term reliability.
As context grows, agents:
• forget constraints
• stop exploring
• hallucinate after correct tool use
We introduce LOCA-bench, a benchmark for long-context, long-horizon agents.
Scale the environment length — accuracy drops sharply.

[Junxian He @junxian_he]

As long context becomes the main bottleneck for agents, existing benchmarks focused on long-context modeling still rely on single-step info retrieval/aggregation.

We introduce LOCA-bench, which controllably grows context length (up to infinitely long!) via manipulating env states. LOCA-bench can assess lang agents in various context sizes on the same task query: https://t.co/kyFLqGQ2v3

[Weihao Zeng @AndrewZeng17]

Huge thanks to everyone that contributed to LOCA-bench: @yuzhenh17 @junxian_he

[Weihao Zeng @AndrewZeng17]

🧠 Long-running agents quietly fail as context grows.

Even with 100K–1M token windows, reliability degrades — plans drift, constraints are forgotten, exploration collapses.

We introduce LOCA-bench, a benchmark designed specifically for long-context, long-horizon agents.

Github: https://t.co/YK2xcni0wK
Paper: https://t.co/TgtRHuOijo

[Weihao Zeng @AndrewZeng17]

🛠️ One standout: programmatic tool calling.

Across models, it consistently improves success rates and shortens trajectories by reducing intermediate context bloat.

LOCA-bench is open-sourced as a platform to study and stress-test long-running agents in realistic settings.
If you care about long-context agents, this benchmark is for you.

[Wei Liu @WeiLiu99]

Writing GPU kernels is the perfect playground for Reinforcement Learning

Why? 1️⃣ Verifiable Objectives: Code either runs or crashes; it’s either fast or slow. 2️⃣ Iterative Nature: It naturally fits multi-turn refinement, just like human experts profile and optimize step-by-step.

But making it actually work is notoriously hard. RL models love to "cheat" (reward hacking) or "slack off" (lazy optimization). 😈😴

We finally crack this.

Introducing Dr. Kernel: Reinforcement Learning Done Right for Triton Kernel Generations.

With just 14B parameters, our model achieves performance competitive with, and even surpassing, frontier models like GPT-5 & Claude-4.5-Sonnet on KernelBench

To solve these challenges, we:
🔹 Built KERNELGYM: A robust environment that handles crashes & detects hacking.
🔹 Propose TRLOO: An unbiased estimator for multi-turn RL.
🔹 Overcome "Lazy Optimization" via Stability (MRS) & Objective (PR/PRS) alignment.
🔹 Achieve massive gains via Sequential Test-Time Scaling (STTS).

[Junxian He @junxian_he]

Gemini-3-Pro improves Gemini-2.5-Pro from 10.5% to 36.4% on Toolathlon! Only one step away from Claude-4.5-Sonnet now, very impressive

[Weihao Zeng @AndrewZeng17]

Honored to be part of building Tool Decathlon (Toolathlon) 🛠️✨

Every component — from the user prompts to preprocessing and evaluation — has been meticulously crafted and rigorously verified.

Toolathlon aims to expose where top proprietary models fall short on diverse, complex, and realistic agentic tasks — and also highlight the gap between open-source and proprietary systems.

Our goal: to spark insights that drive the next evolution of agentic models. 😄💪

[Junxian He @junxian_he]

🚀We are excited to introduce the Tool Decathlon (Toolathlon), a benchmark for language agents on diverse, complex, and realistic tool use.

⭐️32 applications and 600+ tools based on real-world software environments
⭐️Execution-based, reliable evaluation
⭐️Realistic, covering daily and professional scenarios

Toolathlon reveals significant shortcomings of SOTA LLMs in realistic tool-use tasks, where Claude Sonnet 4.5 achieves 38.6% success rate. It also indicates a clear gap between open-source and leading proprietary models.

Check our blog: https://t.co/dkvtu54xE6
Github: https://t.co/CkF6BqeuC9
Paper: https://t.co/TygC61Mebu

🧵⬇️