Derek Cheng

Engineering @ Atlassian. ex-Microsoft, Meta.

[Derek Cheng @derekcheng]

Hey everyone, I’m Derek. I lead engineering at Atlassian for Jira and Developer AI. I’ve been a builder most of my career at both startups and big tech. Grew up in Canada but now based in Seattle.

I’ll start sharing more of what we’ve got cooking. Would love to connect - DMs are open or feel free to drop a question.

[Derek Cheng @derekcheng]

I get to work with Christian every day on envisioning the future of teamwork, continual learning, our context strategy, and... headcount spreadsheets :)

Give him a follow!

[Christian Liensberger @ciaothisischris]

Hey, I’m Christian. I’m the technical assistant and chief of staff to Atlassian’s CTO for AI and Teamwork. My role gives me an unusual view of how a large engineering organization is changing the way it builds software with AI, from product strategy and engineering decisions to the everyday workflows of teams building Jira, Confluence, and Loom.

I’ll share what we’re learning along the way: the product and engineering questions we’re working through, which AI workflows are catching on and why, and the challenges that keep showing up across teams. I’ll also share the workflows I’m trying myself, including what works and what doesn’t.

Before Atlassian, I spent 15 years at Microsoft in product and technical leadership roles, including leading Cortana at Work. I’m based in the Seattle area and grew up in Italy.

If you’re building with AI or helping your teams adopt it, I’d love to compare notes. My DMs are open, or feel free to drop a question below.

[Derek Cheng @derekcheng]

@joshbeitler Yes! Also, the problem of selecting which apple device for my airpods to connect with is apparently undecidable

[Derek Cheng @derekcheng]

On the verge of resolving P vs NP, but I’d just like to not have to re-login to iPhone apps every three months…

[Derek Cheng @derekcheng]

@TylerBrownAT The React thing resonates. For a decade, it seemed the most impactful change all year was a new React version :)

[Derek Cheng @derekcheng]

@davidtoniolo lol I am completely unreasonable

[Derek Cheng @derekcheng]

@swperb1 The trouble is scaling this to real world legacy codebases

[Derek Cheng @derekcheng]

This seems like the fundamental problem with formal verification methods I'm seeing: if you trust an LLM to flawlessly convert your legacy codebase to a formal spec, wouldn't you just trust its initial code output anyways?

[Alex Lavaee @alexlavaee]

We spent fifty years making proofs impossible to fake. Then we let the coding agent pick what to prove.

A research paper this month put a number on it.* AI wrote the formal spec, AI wrote the proof, the verifier accepted both. Then humans read the specs. Close to 60% proved something other than what the author meant. Not wrong proofs. Wrong specs.

The verifier was right every time. That's the problem. Left to choose, the agent picks properties that are easy to prove and say nothing about the program, and the checker can't tell a useless theorem from a useful one. I'm calling it vibe verification: the agent writes the spec and the proof, the checker goes green, nobody reads the spec. This applies to Verus, Lean, Bend, TLA+, Dafny, [insert favorite formal language here].

Here's an example of an agent failing at something that feels too simple to fail:

Ask for a verified sort. It writes a spec that says "the output is sorted" and proves it. What the spec never says is that the output has the same elements as the input, so a function that returns an empty list satisfies it, and the verifier accepts the proof. Trivially satisfiable specs are the default failure, not an edge case.

Here's a formula that actually works. Don't say "formally verify the codebase." Say "show that [property] holds in [component]." A property is a precondition (true going in), a postcondition (true coming out), or an invariant (stays true the whole time).

For example, "A session token is never accepted after the logout that revoked it" is a good invariant.

The bottleneck is moving from code review to intent review, and the spec is where intent lives. So: read the spec before the code. Write 3 inputs it must accept and 3 it must reject. Treat a model-written spec with utmost paranoia and rely on your judgement as an engineer.

Formal verification for software doesn't automatically create trust, it moves it, from the code to the spec. If a human didn't read the spec, you didn't verify anything. You vibe-verified it.

Follow me for more on verification and agentic engineering.

* Wang, Jiang, Liu et al., "VerusEval: Evaluating LLM-Generated Verus Specifications Beyond Verifier Acceptance", Shenzhen University, arXiv 2610.26464, October 2026

[Derek Cheng @derekcheng]

Model naming as a window into corporate psychology -

Anthropic: literary forms
OpenAI: cosmic bodies
Google: chemical elements

[Derek Cheng @derekcheng]

@dantrappx LOL yes, I've definitely had this experience

[Derek Cheng @derekcheng]

When a coding agent says it's done, what do you look for/do to be *completely* convinced the change is good and ready to merge? What are red flags?

[Derek Cheng @derekcheng]

@ganeshiyer316 It's been oft reported that cross-family verification is more robust due to suspected in-family bias. But I haven't seen solid metrics on this. Have you seen pretty clear evidence this works well?

[Derek Cheng @derekcheng]

@CalvinGrunewald Yeah, risk-tiered review/approval is definitely the way forward to allocate scarce human attention where it is needed

[Derek Cheng @derekcheng]

@basedjensen Same structural problem even with frontier models, though right?

[Derek Cheng @derekcheng]

@TaliaRinger Yeah that’s exactly what I worry about. The bugs move to the spec subtly mismatching the code.

[Derek Cheng @derekcheng]

@0xdoug Very interesting result... but not sure it quite implies that the probability of an LLM generating correct formal spec from code > probability of generating correct application code from human direction

[Derek Cheng @derekcheng]

RIght, but most of the discourse is that an LLM will write those precise claims by translating from the original code, which in a brownfield app is large, messy, and written in a language not well-suited to formal verification. My point is relying on that translation to be lossless makes this endeavour very challenging to pull off.

[Derek Cheng @derekcheng]

@dani_avila7 The finding on better cross-functional communication/collaboration is pretty interesting and matches what we observe in practice

[Derek Cheng @derekcheng]

@CalvinGrunewald Yeah, might need to tolerate a whole bunch of chaos to get to the verification promised land :)

[Derek Cheng @derekcheng]

@alexlavaee @kunchenguid This is cool. Will be interesting to see how well it performs on an existing codebase. Do you think there is a viable path beyond Rust?

[Derek Cheng @derekcheng]

@CalvinGrunewald That's pretty cool. Seems like there are two tricky things here for a brownfield app: (1) retrofitting code with various formal statements, (2) refactoring to have a clear boundary between pure functional/formally verifiable vs not. It's possible we just use agents to do both.

[Derek Cheng @derekcheng]

I often ask an LLM to do a long-running task, and then come back to reams of scrollback. I mostly just read the end and find it puzzling as it refers to various things that have long scrolled past or were embedded in its reasoning but not output. Seems like a clear post-training opportunity to summarize succinctly and inline key definitions/references that were developed earlier.