Strong take.
The verifiability point is the real unlock, not speed, not token cost.
The open question: how do you verify adapted skills, ones that diverged at runtime? Packages don't hot-patch themselves against a moving target. The verification loop has to follow the skill.
[2/n] What makes a web-agent skill useful?
A useful skill should be reusable, robust, and able to generalize beyond the exact instance it came from. Users rarely repeat the same request with identical parameters, so a skill that only works for one user, one page pattern, or one lucky path is correct but narrow.
Generalization exists on a spectrum: new parameters, new page patterns, related task templates, and eventually even new websites. Even when a skill does not transfer perfectly, parts of its workflow should still reduce the exploration needed.
Our current experiments focus on held-out instances within the same task templates. We merge multiple successful solves into one skill so it can support more parameters and page patterns. When direct reuse is insufficient, the agent can adapt the skill at runtime, and successful adaptations can later broaden it.