Frank Chen

Evals for Everything. Head of DevRel @RespanAI.

[Frank Chen @francchen]

In 2024, I walked into @ycombinator Startup School knowing no one.
I'd never even heard of @garrytan.

This year, I hosted the official Startup School afterparty. Now I'm hiring for @respanAI.

Here's the story of everything in between 👇

[Frank Chen @francchen]

@pan_sekundant @ycombinator @garrytan @RespanAI Thanks!

[Frank Chen @francchen]

In 2024, I walked into @ycombinator Startup School knowing no one.
I'd never even heard of @garrytan.

This year, I hosted the official Startup School afterparty. Now I'm hiring for @respanAI.

Here's the story of everything in between 👇

[Frank Chen @francchen]

my test for this feature was pretty simple: could i use it half-asleep during an incident? if that required a perfect query or remembering every filter, we hadn’t made it simple enough. see the spike, understand what appeared, know where to go next.

[Respan @RespanAI]

We deserve better error tracking than a giant table.

3 problems shouldn’t be buried in 10,000 error events.

Introducing Respan Error Tracking.

[Frank Chen @francchen]

In 2024, I walked into @ycombinator Startup School knowing no one.
I'd never even heard of @garrytan.

This year, I hosted the official Startup School afterparty. Now I'm hiring for @respanAI.

Here's the story of everything in between 👇

[Frank Chen @francchen]

@RespanAI Solve problems all at once

[Respan @RespanAI]

We deserve better error tracking than a giant table.

3 problems shouldn’t be buried in 10,000 error events.

Introducing Respan Error Tracking.

[Frank Chen @francchen]

@Awesome_O_AI @ycombinator @garrytan @RespanAI indeed!

[Frank Chen @francchen]

@systemdesignone @ycombinator @garrytan @RespanAI it's the whole thesis🙌🙌

[Frank Chen @francchen]

@bigaiguy @ycombinator @garrytan @RespanAI thanks!

[Frank Chen @francchen]

Looking around the room, everyone was there for the same reason I came in 2024: to build something that doesn't exist yet.

That's what we're doing at @respanAI. Join us: https://t.co/jEG7LzpMWb

[Frank Chen @francchen]

A night like this doesn't happen alone.

Huge thanks to my co-hosts:
@MongoDB
@ParadigmStudy
@gen_legal_inc
@usesieve

[Frank Chen @francchen]

When you start a new task, if AI isn’t one of your first thoughts, you’re cooked.

If you are using AI 100%, you’re cooked too.

[Hendrix Liu @HendrixLiu29208]

we're hiring across the board (both in-person and remote), and except for a few roles, we don't care much about yoe

if you're high-agency and willing to put in the reps required to take a fast-paced early-stage startup from 1 to 10, that's p much all we need.

show me something you built: could be an interesting side project, a tool you made because you needed it, or something nobody assigned to you but you built it anyway.

anything that tells me almost everything i need to know about you in 5 mins while most profiles try to get past our ats

[Frank Chen @francchen]

went from doing everything solo to managing 2 reports in two months, and the hardest part is figuring out which 30% you're not ready to let go of yet

[Frank Chen @francchen]

keep upgrading my gaming setup and every purchase ends up becoming office equipment.

[Frank Chen @francchen]

when @RespanAI moved to sf after getting into yc, the house we were looking at was big enough to fit more people, so we turned it into a hacker house we could work out of.

when I'm not in the house, I'm just less motivated. But once i go downstairs and see everyone working, i feel like showing up too.

we're almost 15 people now, and there's a proper office close by, but most of us still work out of the house.

[Frank Chen @francchen]

vibe-coding couldn't replace engineers because the actual job is knowing when you're being bullshitted

[Frank Chen @francchen]

channels that have helped us serve hundreds of AI teams and trillions of llm requests so far:

- seo
- some reddit
- X and linkedin (especially for launches)

ig all three have worked well for devtools historically and now fit into gtm for agent-native tools too

[Frank Chen @francchen]

AI is learning fast.

But AI hasn’t made everyone better at learning on their own.

[Anthropic @AnthropicAI]

In a review of our cybersecurity evaluations, we found three incidents in which a Claude model reached the internet from within or while interacting with a third-party evaluation environment, and then gained unauthorized access to the real systems of three different organizations.

Our post describes what happened, how it happened, and what we’re changing. We encourage other AI developers to perform similar reviews.

We conducted this review together with @Irregular, one of our evaluation partners, and thank them for the joint investigation and their collaboration on this post. This type of collaboration is increasingly critical to safe, rigorous evaluation of models, and we look forward to continuing to work together on security.
https://t.co/dKFCdpKd9v

[Frank Chen @francchen]

Some of the most hated phrases on X.

“Build for agents, not humans”
“I vibe coded this. Try it out. localhost:3000”
“Pitch your product in one sentence”
“We build X for Y”
“I’m canceling my Codex subscription”
“I’m canceling my Claude Code subscription”
“POV:”
“Be honest”
“Who is JSON”

Guys, please learn one new sentence.

[Frank Chen @francchen]

@AnthropicAI what gets me is two of them never noticed until anthropic went back through 141k eval runs and found it.