Nathan Scarrott

human generated tokens

[Nathan Scarrott @NathanScarrott]

@sanlsrni @grok what podcast is this referring to

[Nathan Scarrott @NathanScarrott]

- Get everything in text format (call transcripts, thoughts/ideas, media you find relevant/ informative, etc)
- Dump it in your agent of choice
- Hook it up to a centralised knowledge system such as linear
- See the performance and alignment gains by letting your agent interact with the knowledge base for both exploratory and focused work

It’s really that simple.

[Nathan Scarrott @NathanScarrott]

YOUR AGENT TESTS NEED TO CATCH TIME AND TOKEN SPIKES, NOT JUST CORRECTNESS

our claude agents on bedrock were quietly 2x slower and 35% more expensive than the exact same setup on the anthropic api. nothing failed, no errors, every eval green. the only tell was wall clock and token spend.

root cause: a string compare bug in the agent sdk. bedrock reports the model under a different id than you request (us.anthropic.claude-sonnet-5 in, claude-sonnet-5 back). the sdk thought the model changed every turn and silently deleted all the model's prior reasoning from context. so it re-derived EVERYTHING, every single turn.

no changelog entry, no error, invisible in transcripts (thinking text is hidden anyway). we only caught it because run times spiked vs baseline, then confirmed by capturing wire traffic and counting thinking blocks per request.

track your run duration and token distributions. alert on deviation. correctness tests will not catch a dependency silently doubling your costs.

[Nathan Scarrott @NathanScarrott]

@stevekrouse You can never take for granted that you have a shared understanding with the model.

Front load the work and reach a shared understanding by using the grill-me skill.

https://t.co/iNvjyI3Z8q

[Nathan Scarrott @NathanScarrott]

Claude code has got me illiterate.

Can't even say the word "workflow" anymore without it trying to run 30 subagents.

[Nathan Scarrott @NathanScarrott]

nobody really speaks on the hidden cost of hosting your own model like sizing instances, scaling, upkeep. a provider's API handles all of that. that convenience is underrated.

curious if anyone's made self-hosted inference pay off in a business setting?

[Nathan Scarrott @NathanScarrott]

I recently fine-tuned a ModernBERT for a page classification task to run on client infra with the intent of saving time and money.

Currently, Haiku handles it, and with parallelisation, it's pretty fast.

Hosting your own models sounds great until you need a bigger cloud instance to run the model (especially if you even want to consider parallelisation) and you need to upgrade to a GPU for rapid inference.

[Nathan Scarrott @NathanScarrott]

@neural_avb 🫡

[Nathan Scarrott @NathanScarrott]

@neural_avb You’re right that in theory you’d switch models per task. In practice, the client’s environment (AWS, Azure, on-prem) often decides what you can run. AWS doesn’t give you the same OpenAI or Google access, so a strong default simplifies multi-client deployments.

[Nathan Scarrott @NathanScarrott]

@TheSoleWomens @thesolesupplier Size 10

[Nathan Scarrott @NathanScarrott]

@ParallelNFT @AMAGORILLA @kirkham_eddie NEED THIS!

[Nathan Scarrott @NathanScarrott]

@RyanSte69227960 @OMGitsAliA I DONT BELIEVE IT NO WAY