Michael Bleigh

Building the servers for serverless @Firebase. Formerly founder @divshot. Web platform maximalist.

[Michael Bleigh @mbleigh]

@nicolaygerold Is this still true if the model wrote the skill after examining trajectories of failed tasks attempting to do the thing the skill covers?

[Michael Bleigh @mbleigh]

I hate when I think about a problem for a really long time but can't navigate the tradeoff space to a clear ideal outcome. It's not fun to defend a position that feels "least bad" instead of "good".

[Michael Bleigh @mbleigh]

There needs to be a benchmark that mixes common acronyms with a few undefined fictional ones to test whether the model hallucinates the acronym's expanded definition.

I see this even on frontier models, even when they have the tools available to search for and define the acronym properly.

[Michael Bleigh @mbleigh]

@laurenzlong Have multiple rounds of the interview taking place simultaneously via chat. 😁

[Michael Bleigh @mbleigh]

@rockwotj Just curious, how do you account for cache when doing model routing? Switching models mid-task (especially frequently) is gonna substantially increase uncached token spend.

[Michael Bleigh @mbleigh]

Yep, and the software with the fewest security issues and performance problems is the software that never shipped and has no users!

The thing I don't like is pretending that either side of a tradeoff space is more virtuous than the other. We should acknowledging that our job as engineers is to navigate a complex tradeoff between velocity, security, usability, and performance and do what is best for the product being shipped at that specific point in time.

Sometimes that means combing over LoC with a fine-toothed comb, other times it means letting the slop cannon loose. We are accountable to use all the tools at our disposal, and use them appropriately.

[Michael Bleigh @mbleigh]

Software dev is wildly fluid compared to other engineering disciplines. The ability to rapidly iterate alters the fundamental approach of the practice.

Dismissals of AI-maxxing oft include a judgment of the work being done as "not serious", but this misunderstands software.

[Michael Bleigh @mbleigh]

This relies on four assumptions: that there is a bug in the code, that you would catch it in a manual review, that the bug is serious enough to cause significant user frustration, and that said user frustration is worse for the business than moving slowly.

Could be true, but far from guaranteed. We need not pretend that we ever shipped bug free software in the first place. Most bugs can be fixed forward with minimal user impact.

Being overly precious about shipping code is just as much of a failure mode as being overly sloppy. The job is to balance the tradeoffs for maximum business success.

[Michael Bleigh @mbleigh]

@remi_rousselet A number of people are indeed asking to review every line of code, though.

[Michael Bleigh @mbleigh]

@remi_rousselet But we can't assume that our methods for making sure the job is done correctly should be the same as they always were.

Reviewing every line of code creates a bottleneck that eliminates large portions of the AI productivity gain. We need to build new ways to gain confidence.

[Michael Bleigh @mbleigh]

AI is the single largest source of iteration acceleration the industry has ever experienced. We are only beginning to tap into what that really means.

An industry defined by iteration velocity can't help but be redefined by AI. And so it has been and will be.

[Michael Bleigh @mbleigh]

Faster iteration means more nimble responses to market demands. Software teams can focus on features one week, performance the next, and reliability the week after that depending on criticality of each relative to product market fit and overall business maturity.

[Michael Bleigh @mbleigh]

@pamelafox No, but largely because building effectively without writing code is only a year old which means that person could only have at most a year's experience.

Two years from now? Yeah, probably.

[Michael Bleigh @mbleigh]

@Lethain Part of the criteria is no high priority issues detected, but is that before or after the change has already gone through an agent review pass?

[Michael Bleigh @mbleigh]

In Pluribus, an alien intelligence usurps overnight and dissolves all previous divisions, establishing a semi-utopic society with regretted casualties along the way and a small group who refuse to partake of the new order.

So it's about AI and the software industry, basically.

[Michael Bleigh @mbleigh]

The age of ticket-closer engineering where you just take well-defined tasks and turn them into functioning code and cash a paycheck is definitely over.

But something newer and, imo, much more rewarding is taking its place.

[Michael Bleigh @mbleigh]

I feel like the more powerful agents get the more valuable software engineering experience and expertise becomes.

Building software is now about providing concise, properly scoped goals and verification criteria. The people best suited to do that? Expert software engineers.

[Michael Bleigh @mbleigh]

I used to worry about the junior to senior pipeline in an AI world and I'm still not sure exactly how it will work but I'm more convinced now that it will work.

It'll be a different set of skills but the same hard-won wisdom can still be acquired by other means, I think.

[Michael Bleigh @mbleigh]

@rockwotj I'm surprised Opus is the ugliest, generally been finding Opus to have better visual taste.