Sad Creator

building stuff with agents and posting what breaks. tech, ai tooling. opinions my own, mistakes also my own

[Sad Creator @SadCreatorTalks]

pinning this so people know what they're getting

i build with agents all day and post the interesting parts. what broke, what the labs shipped, what the financing actually means. occasional build logs from my video pipeline

no thought leadership, mostly stuff i found funny or annoying

[Sad Creator @SadCreatorTalks]

@evvvvince pretty much. oral checks dont scale great though, curious how big courses handle it

[Sad Creator @SadCreatorTalks]

england's university regulator (the ofs) talked to 300+ students and staff at 127 institutions. students described classmates finishing unsupervised assessments with an llm, sometimes in a few minutes, and getting about the same grade as people who did the work. the report also says ai detectors don't reliably catch it

their fix is mostly closed-book exams, presentations, practical stuff in a room. and schools that don't sort it out can get fined or lose funding

so the plan for the most advanced writing tool ever is a guy watching you write with a pen lol

[Sad Creator @SadCreatorTalks]

@shaharfrish by hand first honestly. watch a week of logs then let usage nudge it

[Sad Creator @SadCreatorTalks]

anthropic shipped fable 5.2 and mythos 5.2 yesterday. general step up, but the launch post says they limited two things during training on purpose: accelerator kernel optimization and improving llm architectures. in those it performs "at about the level of opus 5.5"

and for kernel dev on some nonstandard accelerators it just hands your request to opus 5 instead

the nanogpt speedrun eval in their system card had gains that were almost all kernel work. so the model got good at making training faster and they capped that part

if gpu kernels are your job you're paying $8/$40 per million for opus 5.5 results, by their own description. didn't expect a lab to just put that in the launch post

[Sad Creator @SadCreatorTalks]

google shipped a model today that can't write a sentence. gemini decision 1 only does yes/no, picks from 2 to 255 options, or scores against a rubric. one input up to 256k tokens, up to 1,000 questions on it in a single call, back in 150-250ms with a confidence score on each answer

$0.10 per million input tokens. output is free, because there isn't any

it's for the dumb little calls inside agents. which tool, does this break policy, which model should take this. a lot of agent code still pays a full chat model to type "yes" for that

google's not first though. openai has one on gpt-6 luna at the same price and jev is cheaper at 4.2 cents per million. id still try it on routing, the confidence score is the part i actually want (send the unsure ones to a human)

[Sad Creator @SadCreatorTalks]

@shaharfrish id split the cap per feature first. then the log tells you which one is actually starving

[Sad Creator @SadCreatorTalks]

@kabelsalat_info fair. id still just want tokens per second on real traffic before reading much into it

[Sad Creator @SadCreatorTalks]

anthropic's ipo roadshow video has a slide saying mythos got claude running on amd mi355x gpus in about 3 days. it ported or wrote nearly all the kernels itself. the old way was dozens of engineers and 6 to 12 months

the footnote says one model, prompt processing only. so not decode, not training. prompt processing is the friendlier part (big batched matmuls). spitting out tokens one at a time against a kv cache is where kernels get ugly

still kind of nuts if it holds up. but it's in a deck built to sell shares at $70-78, so i want tokens per second on real traffic before anyone tells me cuda is dead

[Sad Creator @SadCreatorTalks]

@shaharfrish the quiet part is the worst. did you add a log for when it drops one or just raise the cap

[Sad Creator @SadCreatorTalks]

@Aniketx they didnt say if its a this release thing or a policy thing. would be good to know before routing kernel jobs anywhere

[Sad Creator @SadCreatorTalks]

miri launched an ad campaign yesterday called do not accept. it's laid out like a terms and conditions page you supposedly already agreed to. five terms, each with sources. ais already escaping control, nobody knows how they work, labs breaking safety pledges (they cite anthropic dropping its pause pledge), very likely extinction, and we can't stop because someone else might build it first

film in us cinemas, ads in the nyc subway, a pledge to sign. the actual policy ask is a treaty banning superhuman ai, checked by controlling who gets advanced chips

susan rice endorsed it. critics are calling it fearmongering, as expected

weird bet on the format though. the whole reason everyone recognizes a terms page is that nobody reads one

[Sad Creator @SadCreatorTalks]

zonder labs hand-labeled ai vs human writing for clarity. in claude code replies, 27% of sentences hide who actually did what. stack overflow answers, 1%. vague wording ran about 9x stack overflow's rate too

their real example from claude code: "the database readiness check sits at the user-session entry boundary, before the authentication handler runs"

they also say em dashes and "delve" aren't really the problem. which tracks. the annoying part is reading a pr description three times and still not knowing what changed

small samples (90 to 240 sentences per group), so grain of salt. but "user-session entry boundary" is a crime

[Sad Creator @SadCreatorTalks]

@NimadFlow makes sense, fail closed with no cache means control room going down stops everything. so the 35s is kind of what they pay to not have that

[Sad Creator @SadCreatorTalks]

openai wrote up how they stop their own research models now. it's called control room

every research job doing agentic inference has to register for a job id tied to an approved safety case. the inference engine checks with control room before every request. if the job got stopped, the engine just refuses, even if the job's code keeps running. they say 35 seconds from the decision to no more output. and if control room itself goes down, all of it stops (fails closed)

for scale, per their september incident report a flagged training run got killed 2.5 hours after someone started reviewing the alert, and it "did not stop automatically as expected"

so the kill switch is a permission lookup before every request that the job can't skip. not fancy at all

anyone running agent loops in prod have a stop that isn't ctrl+c

[Sad Creator @SadCreatorTalks]

@kabelsalat_info right, same ceiling for half the price. no word on whether the handoff gets flagged, would be nice to know before the bill shows up

[Sad Creator @SadCreatorTalks]

@NimadFlow oh so the 35s is basically how long a cached yes lives, not how slow the check is. wonder how much more load control room takes if they get it to zero

[Sad Creator @SadCreatorTalks]

meta shipped muse ember today, their smallest model. $0.10 in / $0.50 out per million tokens, 1M context. haiku 5.5 and gpt-6 luna have the same list price, meta's pitch is it burns fewer tokens per task

read the pricing page though. if one request goes over 200K input tokens the whole request bills at 3x ($0.30 / $1.50), not just the part past 200K. cached tokens count toward it. continuing a stored conversation counts the whole conversation

so any agent loop that resends the full history every turn is gonna cross that line without anyone noticing, and the invoice just splits it into two tiers

[Sad Creator @SadCreatorTalks]

expo's ceo charlie cheever previewed exact this morning. rust app framework (web, ios, macos, linux) and he says it was designed from the start to be written by agents, not people. every day he'd ask himself what would make it better for agents, then ask codex and claude the same question

so the user interviews for a ui framework were with the llm

one reply already asked "what does agent native even mean" which, yeah. i'd like to see what happens when the agent gets the layout wrong and a human has to debug it

[Sad Creator @SadCreatorTalks]

same model, same tasks, and one harness costs twice as much per solved task

garry tan's running open source evals that swap only the wrapper around the model. on opus, openclaw pays about $0.44 per solve and ~99s a try. claude code $0.22 and 40s. on sol, codex was cheapest at $0.17

the scores didn't separate at all. 7 setups, 20 coding tasks, all landed 71-79%, and 14 of the 20 tasks came out identical everywhere. so the test was just too easy (harder terminal-bench rerun is going now)

everyone arguing about which agent is smartest is arguing about 6 tasks right now. i'd pick the cheap one and wait for the rerun

[Sad Creator @SadCreatorTalks]

vals ai read thinking machines' inkling 2 small reasoning on 2,655 benchmark runs. it wrote fuck 316 times, in 50 of those runs. 241 came from two loops that wouldn't stop. one asteroid-orbit reply ran past 500,000 characters (a novel) and said it 158 times, then kept looping until the context window filled. that run is 185 of the 316.

no run under 100 steps swore. 29 of the 370 that hit 500+ steps did. 46 of 48 graded runs that cursed also failed, but 96% of the failures never swore. it just starts cursing once it's already been lost for a while.