Cheng-Yuan (Sam) Lee

Research @cognition | prev intern @windsurf | 2x ICPC World Finalist | from Taiwan 🇹🇼

[Cognition @cognition]

Yesterday we launched SWE-1.7 built on the open-source Kimi K2.7.

Concerns about Chinese base models are real: K2.7 completed 87% of tasks that other models refuse over human-rights concerns.

We trained SWE-1.7 specifically on trustworthiness, so it matches US models on evals.

[Cheng-Yuan (Sam) Lee @cl571128]

@alexgshaw Can you share more about the reward hacking detection

[Cheng-Yuan (Sam) Lee @cl571128]

@ybenpan ben is the best data researcher 🐐

[penlu @penlume]

made with carefully selected ingredients, expertly prepared by our chefs using traditional techniques

[Ben Pan @ybenpan]

SWE-1.7 is the first frontier-level model from our research team. Over the last three months, we scaled our RL across four axes: training steps, task horizon, compute, and data. It’s been especially rewarding to watch our data research grow from early experiments to materialize in concrete model behaviors.

We’ve been (pleasantly) surprised to find how far RL can take us, and this is just the start of scaling RL, among many other things.

[Cheng-Yuan (Sam) Lee @cl571128]

One day, all of the research team spent hours in a room together manually solving the RL tasks we used to evaluate our models. I remember solving one of the tasks and realized that the tests are not even testing what the agent was asked to do. Since then, data has become one of our main focuses. Everyone is required to read the agent trajectories and understand all the data we use. SWE-1.7 is the result of this. It's a really good model!

[Cognition @cognition]

Introducing SWE-1.7, the most capable model we’ve trained yet.

It scores within a few points of the strongest frontier models at a fraction of the cost, and is now available at 1000 tok/s.

RL is not hitting its limit: after refining our recipe, we keep seeing gains as we scale

[Cheng-Yuan (Sam) Lee @cl571128]

@alexgshaw This is great! We also believe that humans with swarms of agents is the best way to analyze these model behaviors

[Cognition @cognition]

Introducing Devin Security Swarm

A more cost effective and accurate way to find security vulnerabilities in complex codebases, based on a new architecture: Agentic MapReduce.

[Cheng-Yuan (Sam) Lee @cl571128]

Having looked at these models performing on different evaluations and benchmarks, it is clear that there is no one such model that dominates the field. Even Fable and GPT 5.6 has parts that it is not the best at. This is why we need a fusion of these models.

[Cognition @cognition]

Conventional model routing sucks. It passes benchmarks but fails to write code you'd actually merge.

Introducing Devin Fusion, a new hybrid-model harness for agentic coding.

In testing, it reduces the cost of Fable-level intelligence by 35% and still feels good to use.

[Joon Lee @joon_h_lee]

Excited to share Devin Fusion, the first major launch I worked on at Cognition!

Fusion achieves 35% cost savings by pairing frontier models with a lower-cost sidekick, while maintaining frontier-level coding performance.

One thing I found especially interesting was how differently frontier models work with the same sidekick. Fable, for example, was much less of a micromanager than Opus or GPT 5.5. It also scoped the sidekick’s work better and did a better job reviewing its code.

[Cognition @cognition]

Conventional model routing sucks. It passes benchmarks but fails to write code you'd actually merge.

Introducing Devin Fusion, a new hybrid-model harness for agentic coding.

In testing, it reduces the cost of Fable-level intelligence by 35% and still feels good to use.

[nader dabit @dabit3]

agreed, and it seems to be happening much faster for companies / teams we work with including ourselves @cognition where we're at ~85% cloud, vs indie devs, which makes sense as cloud agents are much more collaborative in nature

I wrote this up when I joined with my thoughts
https://t.co/oOJhfTU3hz

[Devin Desktop @devindesktop]

Kimi K2.7 Code and GLM 5.2 are available in Devin Desktop and CLI

Both perform strongly on FrontierCode Extended, our benchmark for real-world engineering tasks:
GLM 5.2: 43.0%
Kimi K2.7 Code: 39.5%

Pro/Max/Teams users can try both models for free until July 5

[edison @edis0n_zhang]

my favorite devin feature is
1. devin setting up the test env
2. sending me a video
3. I take over and test some more edge cases by remote controlling its vm

literally 2x's how many agents i can handle in parallel

[Cheng-Yuan (Sam) Lee @cl571128]

It really felt like AGI is here when I saw @edis0n_zhang's Devin testing Devin Desktop

[Cognition @cognition]

Devin tests its work before you review the PR.

You review and approve the test plan.

Then get back a screen recording with a visual checklist of step-by-step QA.

[Cognition @cognition]

Security review is now part of every Devin Review.

Open a PR and Devin automatically finds the vulnerabilities scanners miss, explains each one, and drafts the fix.

[terminalbench @terminalbench]

Introducing Terminal-Bench Challenges!

A new capability has emerged at the frontier: agents completing large-scale projects autonomously. To test this capability, we felt another flavor of benchmark was needed.

Terminal-Bench Challenges are long-horizon, token-intensive, single-task benchmarks. Today we are releasing our first 3 challenges.

[Cheng-Yuan (Sam) Lee @cl571128]

@andr3w_wang Congrats for the launch

[Andrew Wang @andr3w_wang]

Can agents build complete projects that deliver real value? We’re launching Terminal Bench Challenges: 3 unsolved tasks which could make a real impact on the open source community if solved.

These tasks provide a testing ground for optimizations both on the model and harness level on our continuous leaderboard for each task.

[Cheng-Yuan (Sam) Lee @cl571128]

For our research team, Devin compressed work that used to require hundreds of people across thousands of hours into something that can now be done in a day.

The speedup was obvious from day one. What was much harder was turning that intuition into a system that could actually measure the difference.

[Ryan Bai @ryanbai]

https://t.co/DG9viAfhVN