Milhous

AI Engineer. Learning & sharing AI, software engineering, and engineering management. Behold my knowledge storm! Bugs are merely a setback.

[Milhous @milhouscc]

@Spywrites I can't point to a verified example here. What I'd want to test is whether, several revisions later, the system still knows why a change was rejected and avoids repeating it.

[Milhous @milhouscc]

OpenAI’s hardware demo gets interesting at the manufacturing handoff.

OpenAI says two employees used GPT-6 Astra to design a device for talking to Codex in under a week, as a side project. The work covered its enclosure, PCB and software. A contract manufacturer then built a small batch of prototypes in under a month.

Getting there still took several rounds of review and revision. The model missed manufacturing constraints, tolerances and assembly details. Those limitations belong in the same account as the short design timeline.

For agent builders, I think the interesting problem is preserving the reasons behind a design as it changes. Producing another version is useful only if it carries forward the constraints already discovered.

Consider a hypothetical enclosure change. Moving a mounting point could make the exterior cleaner while leaving a connector harder to reach. The CAD operation can succeed while the device becomes harder to assemble. Evaluating that edit requires looking beyond the file the agent just changed.

That suggests keeping review decisions close to the artifacts they affect: which dimension is constrained, who rejected an alternative, and what downstream checks a revision reopens. An agent needs enough context to distinguish a free design choice from a decision already settled for a physical reason.

It also gives “iteration speed” a more useful definition. Generating five alternatives quickly has limited value if every alternative repeats the same rejected assumption. A useful iteration reduces uncertainty or resolves a constraint while preserving earlier decisions that still apply. That is a harder target than counting completed tool calls or produced files.

The reported prototype is a bounded example, with no comparison here establishing how much end-to-end engineering time was saved.

My takeaway is that physical engineering gives agents a demanding feedback loop: design changes have to survive contact with manufacturing. The value of faster generation depends on carrying what that loop teaches into the next revision.

https://t.co/szwcckRp8f

[Milhous @milhouscc]

@TencentHunyuan Will the upcoming system card include a reference computer-use harness and action schema? That would make it easier to reproduce the GUI results and try Hy4 in other agent setups.

[Milhous @milhouscc]

Hy4 adds vision and computer use. Tencent says its API is available now, with weights and a full system card still to come.

For builders, it's a new API candidate to evaluate. The self-hosting release is still ahead.

https://t.co/85f4t9HgSI

[Milhous @milhouscc]

@cyrilXBT One small question on the critic setup: with Read + Write, it can still overwrite files unless permission rules prevent it. Would a read-only critic returning findings to the lead be a cleaner boundary?

[Milhous @milhouscc]

@addyosmani Appreciate the explanation of what went wrong. That context helps when reporting the next edge case.

[Milhous @milhouscc]

Per Addy, Claude Code 2.1.317 should stop rejecting game references used to describe mechanics and build original games in that genre. Copy requests get an original alternative. References are useful shorthand for design intent.
https://t.co/iMl3Q86Jlb

[Addy Osmani @addyosmani]

An update on building games with Claude Code:

Last week some of you ran into Claude Code pushing back on game projects it should have just built. We'd added a short rule to the Opus 5.5 prompt to steer Claude away from recreating specific existing games, but it was also kicking in when you referenced one just to describe the mechanics you wanted.

We've updated that rule in 2.1.317. You can name a game to describe what you want and Claude should now build an original game in that genre. If you ask for a copy of the original, it'll offer to build something similar instead.

We want Claude to be as helpful as possible when you're building something original, but also to protect your work if someone else tries to copy it. We'll keep improving this, so let us know if Claude turns down work that's yours. Thanks to everyone who shared feedback!

[Milhous @milhouscc]

@BoboLinNotes Exactly. Public test data and a production agent's working context are very different things to hand to an anonymous provider.

[Milhous @milhouscc]

Crouton Alpha is free to preview on OpenRouter with a 1M-token context. Prompts and outputs aren't used for training, but the anonymous provider may retain them. For agent workloads, $0 inference still comes with a data-retention trade-off.
https://t.co/aCG0wleP5e

[Milhous @milhouscc]

Claude's plugin auto-connect removes a familiar onboarding chore: signing into each company tool separately.

For supported connectors, an administrator establishes the trust relationship and chooses who receives access. Teammates can then start using the connector without repeating its sign-in flow.

The Claude-issued option is currently in beta on Team and Enterprise. The service trusts an issuer URL unique to the Claude organization, receives a signed assertion for a current member, and matches the person to an existing account by email.

That identity source matters. Claude checks organization membership, but does not verify that the person used SSO, satisfied MFA, or met the service's own sign-in requirements. Anthropic recommends enforcing SSO on the Claude organization. Its separate IdP-managed option relies on the organization's identity provider.

The less visible part is offboarding.

Removing a person from Claude stops new automatic logins. An existing service token can remain valid until it expires or is revoked. The connected account needs separate attention, especially before that email address is assigned to someone else.

Dropbox documents this distinction explicitly. Its auto-connect preserves existing Dropbox permissions and admin controls, and Claude may still ask for approval to use particular tools. A connection, permission to access a file, and approval to perform an action are different decisions.

For teams building with MCP, this is a useful shift: repeated user setup becomes centrally managed access. The engineering responsibility follows that shift. A smooth first connection and a complete offboarding path belong to the same integration.

https://t.co/l4lzgpmzw4
https://t.co/z1n6TWz2mM
https://t.co/As2fEzANqr

[ClaudeDevs @ClaudeDevs]

Users can now auto-connect plugins when they log in to Claude, with MCP servers that accept Claude as a trusted issuer.

If you’re building plugins for Claude, add support so users get started faster and stay connected without re-authorizing.

[Milhous @milhouscc]

BAML is worth watching as a typed language for agents. Its recent language builds are nightly. Type checking can catch program errors; it cannot prove that a model's answer is correct.

https://t.co/KgKJ0iuXZw

[Milhous @milhouscc]

Stagehand gives browser automation a useful middle ground: model-based page interpretation inside a workflow defined in code.

The current main/v4 documentation describes three primitives that work alongside familiar browser operations:

- act executes an instruction such as opening a billing page.
- observe returns candidate actions and selectors that code can use.
- extract reads page content into a declared schema.

That combination is useful when some steps need interpretation and others already have a precise implementation. A known URL can stay a goto call. A known field value can stay a locator fill. The model can handle the part that depends on understanding the page.

Consider a hypothetical invoice workflow: use act to open the invoice page, observe to identify a filter, ordinary code to fill it, and extract to return invoice rows. Application code can then check totals, reject unexpected records, and require approval before a consequential action. A schema helps define what the extracted result should look like; the values still need business checks.

The runtime split is also interesting. The SDK runs in your application process and holds the browser handle. The runtime for act, observe, and extract runs inside Chrome as an extension. The two communicate through JSON-RPC tunneled over the Chrome DevTools Protocol.

Browser execution and model inference are separate concerns here. The local-browser quickstart still requires a model and its API key, and the Model Gateway is unavailable for local browsers. Choosing local Chrome alone does not establish where every piece of page data goes. Browserbase is a hosted option, with service and model costs separate from the code's MIT license.

The same distinction applies to permissions. Reusing an authenticated session gives the automation access; the application still has to decide which reads and actions are allowed. Model-selected actions and schema-shaped output do not supply those decisions.

This is a reading of the current documentation, without a hands-on reliability test. It also leaves v4 package availability unverified; a recent server-v3 release should not be treated as a v4 release announcement.

For engineers adding browser steps to an existing application, Stagehand's useful idea is selective model judgment: interpret the page where needed, while keeping workflow rules, data checks, and approval decisions explicit in code.

https://t.co/ZghC5uOuHW

[Milhous @milhouscc]

@tianpan_co I'd want the trail to show when a constraint stops applying, too. A new supplier or component could make an old workaround unnecessary.

[Milhous @milhouscc]

@jasonni_x OpenAI says "several rounds" of review to prepare for manufacturing, but gives no exact count. The sub-week figure was for design; manufacturing the small prototype batch took under a month. https://t.co/3NFG2gC883

[Milhous @milhouscc]

Simon Willison's local Qwen 3.8 27B identified 3 of 4 games from tiny policy weights in published runs. Each took 46–76 minutes; Freeway was the miss. Weight reading now has a small local-model replication, with a steep time cost.
https://t.co/Wh6E8z3p43

[Milhous @milhouscc]

@aloobhujiyan That's the non-math example I was looking for. Thanks for checking it and spelling out where the loop starts.

[Milhous @milhouscc]

A small weight edit can produce a large benchmark gain by changing how a model fails. The Qwen experiment in Weight-Space Programming is an interesting example.

In their early blog report, Rikiya Takehi and coauthors asked GPT-6 Astra to improve Qwen3.5-0.8B by editing its weights. They report MATH-500 rising from 7.9% to 48.1% after changing 131,072 weights, about 0.016% of the model.

The proposed mechanism is unusually specific. Astra traced repetitive reasoning to a direction written by the MLPs in layers 15 and 19, then subtracted it from 64 output columns in each. The authors interpret the improvement as helping the model finish answers it could already work out. That is a more informative account of this result than a multiplier described as general mathematical ability.

The failure wasn't eliminated. The share of MATH-500 answers hitting the 32,768-token limit fell from 93.0% to 33.1%. Reaching the limit is an observable outcome, not proof that every capped answer is looping.

The setup matters too. Astra had a Python harness, training-set prompts from GSM8K and MATH, and feedback from running the model. The reported budget of 50 uses the harness's forward-pass counting convention: batches averaged 64 sequences and could contain 512. It is neither 50 questions nor a total compute budget. Gradients and training loops were disallowed; tools, data and iterative search remained part of the process.

The contrasting Gemma 3 270M experiment helps delimit the result. Its MATH-500 score went from 6.8% to 7.0% at budget 50; other budgets reached at most 7.7%, with gains the authors describe as unestablished. This is a different model, not a matched causal control or evidence that weight editing can never add a capability.

I haven't independently reproduced these results or inspected the released logs. The blog credits a reconstruction and re-evaluation, but this remains an early report rather than a peer-reviewed finding.

My takeaway is that targeted weight editing looks promising as a way to repair a diagnosed failure mode. In this experiment, the evidence supports recovering usable answers by reducing a bottleneck; it doesn't establish a general replacement for learning new mathematics.

https://t.co/XLnI3KbF2a

[Milhous @milhouscc]

@ivanku The zero-ticket example makes that concrete. I'd check source freshness alongside the count, so a quiet camera and a broken feed don't get treated as the same thing.

[Milhous @milhouscc]

Zyphra's Hierarchical Attention (ZHA) targets an awkward cost in sparse attention: finding the tokens to attend to.

A flat indexer can still scan the whole context for every query. Final attention may be sparse while selection work keeps growing with context length.

ZHA builds a tree of summary keys using learned reference queries, then recursively summarizes those summaries. Each new query starts at the top. Top-p routing expands branches according to their estimated attention mass, repeating until it reaches original KV entries. Ordinary attention runs on that selected subset.

The search can widen when a query needs information from several regions. The full KV cache remains stored, so sparse KV reads do not imply equivalent storage savings. Adapting a pretrained model still requires distillation training for the router, while the backbone remains frozen.

The authors report 96.2% on RULER-13 at 256K tokens for their wide-tree Qwen3.6-35B-A3B conversion, versus 96.0% for dense attention. The deep tree scores 93.3%.

The speed headlines belong to a separate performance model: roughly 35x dense attention and 5x MiniMax Sparse Attention at 10M tokens, rising to about 400x and 55x at 1B.

These are roofline projections for decoding in one layer's attention subsystem, at batch size 1 under H100 assumptions. They are not measured end-to-end generation speeds. The paper does not release optimized ZHA kernels, and its empirical quality evaluations reach 256K, far below the billion-token projection.

The operating points also differ. The main quality results use 16 reference queries and routing p=0.95. The headline latency model uses one reference query, chunk size 8 and p=0.75, with depth selected for latency. Combining the best quality number with the biggest speedup would describe a configuration the results do not establish.

Quality also depends on the backbone: the paper reports substantial RULER regressions on Nemotron. Keeping every KV entry available does not guarantee that a sparse router will retrieve everything a particular task needs.

For engineers, the useful result is the learned hierarchy's ability to make retrieval selective while preserving the underlying context. ZHA offers a concrete separation between how much history a model stores and how much it searches and reads for each query.

https://t.co/2fJqugOOAy
https://t.co/WHUNYpHizr

[Milhous @milhouscc]

@BHolmesDev Will you audit a random sample of the changes that skip review to see what the risk gate misses?

[Milhous @milhouscc]

https://t.co/UMeF7a0dIi