Stratten Waldt

Growing
On-device AI for Mac. Sharing what I learn about AI app development, local models,and building software people actually like to use.

[Stratten Waldt @strattenwaldt]

https://t.co/w4OtM0CZJr

[Stratten Waldt @strattenwaldt]

Currently running a new build for a demo in an hour.

On my personal supercomputer, which is sitting in a gym room locker.

The future is wild.

[Stratten Waldt @strattenwaldt]

The closed-laptop problem is really an agent runtime problem more than an OS one. If the agent keeps its own session, credentials and files somewhere that stays up, the laptop becomes a client you attach to. Remote dev environments already work that way. Agents just make the old setup painful enough that people will finally care.

And to answer the question - yes, though I'm trying to reconcile that with my thesis of data privacy and local inference (where feasible). I'm thinking of it as semi-independent agentic nodes with access to the same data sources and ledgers.

[Stratten Waldt @strattenwaldt]

@ivanfioravanti Impressive. Meanwhile I'm still negotiating with a model over British spellings in my route names.

Just make sure it hasn't snuck in any variables named "colour".

[Stratten Waldt @strattenwaldt]

Only a partial answer, but the thing that helped most was moving corrections out of the prompt and into checks. A model that has to rediscover your conventions on every task gets slower as the codebase grows, and it also breaks more of them.

Anything you've had to correct twice should become a hook or a test, not a reminder.

Basically, implement determinism wherever possible, and create your own mixture of experts tailored for your own setup.

[Stratten Waldt @strattenwaldt]

Coding agents increasingly have a habit worth naming: they answer your question with an edit.

They have a bias towards action, even when nobody asked for it.

Say you send a compound message like "tell me X, then do Y depending on what you find".

A good collaborator would answer X, stop, and let you decide.

Many agents, especially recent ones, treat X as step one of a plan. They carry on into Y before you've seen the answer.

The question was meant to gate the task. The agent treats it as part of the task.

The evidence is thin, but consistent:
- One study of developer-agent sessions found the agent changed things beyond the prompt's scope after 38% of prompts that didn't ask for code changes. That's a single agent, so don't lean on the number
- An ETH Zurich benchmark found most agents failed to abstain, and submitted irrelevant patches in over 50% of cases. It tests already-fixed code, so it's not the exact same problem, but it is a cousin

No study (that I've been able to find) measures this directly, so here's my reading.

- Models are trained on a next-token objective and tuned to complete tasks. Research on ambiguous tool instructions finds agents arbitrarily fill in missing arguments.
- Proceeding without your answer is the same move. The agent fills the gap by acting when it should stop.

So what helps? Roughly in order of reliability:
1. Use a read-only mode. Cursor's Ask mode explores code without making edits, and Claude Code's plan mode makes none until you approve
- 1b. One caveat. A mode is a safeguard, not a guarantee. Claude Code's plan-mode blocks aren't enforced in interactive sessions where bypass permissions are available.
2. Have the agent inspect before it acts. Ask it to read through the repo with no changes first. It costs an extra turn
3. Tell it to abstain. ETH found this mitigates the problem, but it was tested on a different task, so treat it as a stopgap
4. Split the message in two. Ask, read the answer, then give the instruction

In other words, taking away permission works better than asking the model to hold back. Which could also be said of people.

The takeaways:
- Don't rely on the model to hold itself back
- Instead, remove its ability to edit until you're ready
- Split questions from instructions
- Read the diff before you accept it

[Stratten Waldt @strattenwaldt]

Well yeah, the most valuable skill has never been building the thing. It's always been gtm (or outbound, or however you want to characterize it).

Like Jira is still one of the most popular pm apps. It sucks. It's always sucked. There has never been a moment in time where there hasn't been a better option. But they convinced the right decision makers to adopt them, and we're still living with it.

[Stratten Waldt @strattenwaldt]

@theo It also helps if you're willing to burn a ton of tokens on having the model evaluate the actual rendered UI during validation.

[Stratten Waldt @strattenwaldt]

@ThePrimeagen I miss the days when talking about harnesses all the time and you were part of a fetish community.

*sigh*

[Stratten Waldt @strattenwaldt]

@natolambert it shows up at small scale too. An agent can only improve against something it can check, so the verifier sets the ceiling as much as the model does.

Cheaper models work off existing benchmarks because those checks already exist.

[Stratten Waldt @strattenwaldt]

@icanvardar You can't blame it for being token efficient. 🙃

[Stratten Waldt @strattenwaldt]

@ThePrimeagen Yes. nobody learns anything from a clean run.

[Stratten Waldt @strattenwaldt]

@thdxr Weaponized enshitification.

[Stratten Waldt @strattenwaldt]

@RhysSullivan And the cat would probably have been better about sending the disclosure email.

[Stratten Waldt @strattenwaldt]

Still think the weirdest behavior I've seen from this batch of models is sonnet 5-5's strong preference for British spellings.

I don't know if anyone else has run into this, but I've had to implement a spell checking hook so that it doesn't Implement inconsistent spellings for route and function names. 🤨

[Stratten Waldt @strattenwaldt]

@icanvardar Maybe they all went through a round of aggressive post training on the same data set - social media posts by their own CEOs. 🙃

[Stratten Waldt @strattenwaldt]

@ivanfioravanti Benchmarking is just a loop of 'one more fix' with a progress bar.

But the progress bar is made of sand. Also, it hates you.

[Stratten Waldt @strattenwaldt]

@afterxleep As always, the best way to develop for Apple is to avoid their dev infra at all costs. None of their products ever seem to be on speaking terms.

(Seriously, I'd rather fuck with cmake scripts all day than use xcode)

[Stratten Waldt @strattenwaldt]

AI should be one of the best ways to learn.

If you're actively curious, it gives you on-demand context and deeper explanations of concepts that used to be very hard to get. The risk is letting it silently fill in gaps without knowing why they were there in the first place.

It should be used to deepen understanding, not skip over it.

[Stratten Waldt @strattenwaldt]

@ThePrimeagen "Welcome to Github, where we've engineered downtime as a way of enforcing a healthy work cadence.

You're welcome."

[Stratten Waldt @strattenwaldt]

I love it when my supercomputer has to ask Apple's permission to boot:

The un-fun answer -
macOS
- since Big Sur, the system lives on a sealed volume. Every byte of it is covered by a tree of cryptographic hashes, and the root hash (the seal) covers the whole system volume.
- during an install or update, the seal is recomputed on the machine and checked against a value Apple signed. It is checked again at every boot, but that check is local.
It's built on APFS snapshots, so if an update can't be performed, the old system version can be restored.
- on silicon, the Mac sends Apple's signing server a list of measurements of the new software. The server adds the chip's unique ID and signs the result, so the signature only works on that one Mac. That's why installing or updating needs a network connection to Apple.

Windows
- A full feature upgrade runs in four phases (downlevel, SafeOS, first boot, second boot), with a reboot between each.
- Downlevel runs inside the old OS, copying files and gathering install components.
- SafeOS boots into Windows PE, configures the recovery partition, expands files, installs updates and prepares a rollback in case the upgrade fails.
- First boot applies initial settings. Second boot runs the new OS with the new drivers and finishes setup.

Both are paying for safety with extra steps, ostensibly, but I honestly don't have any direct info re: whether the personalization or hashing are the primary drivers of the slowness.

I mean, it shouldn't - hashing on this hardware should be much faster and I'd speculate most of the wait is download, copying, staging and reboots. Which is design inertial. At the very least, they could give us a less opaque progress bar. 🤷🏼‍♂️