Concept

[Concept @conceptdotdev]

We launched https://t.co/5Ykerfvdky to measure how well agents understand specific companies.

Read the results here:

[Phil Chen @philhchen]

Today we’re releasing Company Agent Bench, a benchmark for company assistant products.

We compare Dots, Grok Bot, Claude Tag, and Devin on how well an agent understands its company.

We also found that API models with a simple harness perform better than these products (1/6)🧵

[TK Kong @tkkong]

Today, we’re launching https://t.co/WuxyRB4yn0

@OpenAI Dots placed #1 across agent products while @AnthropicAI Fable 5.2 placed #1 overall via API.

Our goal is to measure how well agents understand a specific company.

Check out the results and findings below!

Shoutout @joshzqin for spearheading this project

[Phil Chen @philhchen]

Today we’re releasing Company Agent Bench, a benchmark for company assistant products.

We compare Dots, Grok Bot, Claude Tag, and Devin on how well an agent understands its company.

We also found that API models with a simple harness perform better than these products (1/6)🧵

[TK Kong @tkkong]

I think there's an interesting tension in the market rn.

Agents only really started to work about a year ago with Opus 4.5. Many vertical AI companies since have gone exponential in growth (just chart out the valuation and revenue numbers YTD).

At the same time, when talking to enterprises, they aren't feeling a truly MAGICAL wow that agents have delivered significant business impact.

So what's the actual gap between agents that can solve millennium prizes and delivering real business impact?

My cofounder Phil who was a researcher at OAI/GDM has been thinking deeply about this problem and talks through where agents fail today and the next frontier research problems to close this gap.

[Phil Chen @philhchen]

https://t.co/4sIXMCw6zY

[TK Kong @tkkong]

The Frontier or Nowhere

We’re hiring engineers. Come work with me & @philhchen

[TK Kong @tkkong]

Gemini 4 Argon reaches Astra-level performance on multimodal data annotation. Significant improvement compared to Gemini 3.8. Argon also better at identifying correct actions in egocentric videos.

[Concept @conceptdotdev]

@jasontheutopian @MeckaAI @tkkong @philhchen Incredible model, great to partner

[Concept @conceptdotdev]

@joshavata @tkkong @philhchen Great to collab on this!

[Concept @conceptdotdev]

@MeckaAI @tkkong Let's go! Great to partner with the team

[Phil Chen @philhchen]

Frontier models still struggle to accurately caption egocentric videos. Dense video captioning remains a task where a specialized model, our collaboration with @MeckaAI, outperforms general-purpose foundation models. Many interesting problems still to solve in robotics.

[Concept @conceptdotdev]

Sharing our first research post with @MeckaAI

[TK Kong @tkkong]

Today, we're sharing results from our partnership with @MeckaAI to see if frontier models can label egocentric videos for robot learning.

We had five frontier models caption 200 clips with human grading.

GPT-6 Astra which performed the best got 43.5% of clips fully correct.

[TK Kong @tkkong]

We’re hiring for designers & engineers. Come work with me & @philhchen

[TK Kong @tkkong]

https://t.co/6NeTJEdb2q

[TK Kong @tkkong]

We're opening up early access.

Phil and I have been working with teams to ship internal AI apps and agents.

We’re applying our learnings from Ramp Labs, OpenAI, and DeepMind to help operations teams scale faster.

DM if you're interested in trying it out!

[Phil Chen @philhchen]

https://t.co/oMoEAYkqGl