If you are a neolab, I recommend in the strongest possible terms to secure GB300 capacity for 2027 in the next 8 weeks or else you are not going to have a fun time.
@alpha_pursuit You can actually embed Claude calls within the artifact, this is super OP and I haven’t seen people build with it yet. The artifact can call Claude itself and the subagent gets a “full” working session with access to all your other Claude context.
Opus 5.5 is very good at building apps in Artifact: basic CRUD apps like food logs, fitness apps, Pocket equiv. It is much better than any other Claude model up to that point, including Fable 5. Not sure about F5.2 yet. My comments here are all going to be about building these kinds of toy-ish apps in Artifact.
Basically, if you tell Opus 5.5, I want an app that does these things (keep track of X, show me Y, have Z panes), it can do that. It objectively satisfies the spec, and the app is usable. But it's very far, at least in my experience, from producing zero-shot an app that competes with stuff you would actually pay for, like Strava or Cal AI.
ONE DIFFERENCE is that those apps, which I'll call legacy apps, are much more feature complete. After a few years of being in the market, those legacy apps just have a lot more features and integrations. If you just put a vibe coded app on the app store, you'll be missing a lot of features, and 90% of users are probably going to miss some feature they find critical to their workflow.
But I think this is the wrong way of thinking about things. It's just not that hard for Claude to implement fairly complex features, at this point. Coding capabilities have gotten to the point where Claude can reliably add some feature you can think of.
Instead it's more that being able to zero-shot a marketable app is a prompting skill issue, in that if I had a 200 page spec that covers all workflows, the model would be able to implement it. Prompting, of course, is something models can do now. I have yet to do this - but it's so obvious that I'm pretty sure people are already doing this - if you're reading this and you run a data company, you might want to consider doing this - but you can just give Claude access to your phone (iPhone mirroring), record it while letting it tap around for hours until it accesses all the menus and screens, then reverse out an extremely complete spec of all workflows, application states, and so on, which you can then use to prompt a model, and then, given access to the fully functional app, use that as a correctness oracle to generate a scalar reward via a supervisory model.
So to be clear, I don't think feature completeness is a serious barrier to the competitiveness of Claudecoded apps. I sort of refuse to say vibecoded, because what I'm suggesting will happen for mobile app development is more complicated than a human giving Claude a prompt which it then takes and does stuff, instead, Claude will be actively researching what kind of features to implement by doing market analysis, by talking to users, by reading app reviews, and so on. I digress.
To be clear, I am cognizant of the fact that it is one kind of cognitive work to code an app, and another altogether to design the spec for it, just as it is one kind of work to code up a visualization and another to decide the best kind of visualization. And today even models that are very good, superhuman even, at coding are really bad at product design, product management, spec design. But it is totally copium to use this distinction to argue that models will never be capable of owning all aspects of mobile application development; the latter kind of work is simply another sort of reasoning which the labs will collect data on, build RL envs for, and train.
ANOTHER DIFFERENCE between Claudecoded and legacy apps is that the level of polish in Claudecoded apps is just so, so low. As far as I can tell, this is more because Claude is lazy and unopinionated, rather than because Claude is incapable. By default, Claude doesn't do nice animations for things for me, or make things pretty, or add clever little UX things. It's totally capable of it, and it will do it gladly if I give it a very big and detailed style guide, but it just doesn't do it by default. I think this is just a straightforward post-training thing and goes away within one or two generations once targeted. The way I get around this today is I find examples of the kind of behavior I want in existing apps, then take lots of screenshots and show Claude.
Opus 5.5 isn't perfect at style imitation. It focuses too much on copying color palettes and visual things instead of arrangement, layout, prioritization, and other UX aspects. As I've said before, when you talk to a pretrained model you get a feel for the pretraining corpora, and when you talk to an RL'd model you get a feel for the RL envs. I feel that Anthropic's mobile app data vendors are overly focused on visual design (probably because it's a simpler task and grader models are easier to implement) and much less focused on interaction.
Even though the quality of the Claude apps is relatively low compared to what is available in the App store, because I'm able to customize them so much to what I want to do - my workflows, the information I care about, access to specific data stores - I've been using my homemade apps (for fitness, diet, sleep, etc) more than App store apps - I don't use Bevel anymore, or Reader. Going back to what I was saying about feature completeness, the truth is apps must be swiss army knives, but any individual user is only really using two or three things 90% of the time, and that subset is very easy to cover. I find that at the beginning of making one of these Claudecoded apps, I feel very annoyed at missing features, but pretty quickly you just get enough coverage, and even though I can see when I open up the legacy app that my Claudecoded app is missing a lot of stuff, I just don't care about the missing stuff.
Back to MODEL CAPABILITIES. There are still lots and lots of gaps in capability. I'm actually surprised that it's possible for a relatively unskilled human to identify failures at this level of model, but Opus 5.5 makes a lot of mistakes or is otherwise annoying.
I'll go over the ANNOYING TICS first. One extremely consistent annoying thing Opus 5.5 does is it puts way too much text everywhere. It does this too in graphics and in writing, so verbosity is a core personality trait of that generation, so it would seem, but it's excessively annoying in apps, since it specifically has a tic of presenting information textually even if it's presented visually. So instead of using icons where icons would make sense, it wants to put text. When it has graphs, it labels stuff way too much. Instead of making graphs, it likes to have text explainers.
Another annoying thing Opus 5.5 does is put boxes in everything. This is a pretty classic Claude look going back to Sonnet 3.5 so at this point I think the data has wormed its way into the preference models but it just really loves drawing boxes around things it doesn't need to. Rounded boxes are the it's-not-X-it's-Y of Claude UI design.
A third annoying thing that Opus 5.5 does is add way too much spacing around everything. It just doesn't have good intuition for how much vertical padding to have between elements and I'm constantly telling it to shrink how much space there is between rows of things or buttons and so on.
So there are a lot of these things which I would ultimately describe as Opus 5.5 being too mode collapsed on some particular env which is enforcing stereotyped behavior. And frustratingly it's engrained into the model; I repeatedly tell Opus 5.5 not to overexplain things and have Cuck Text everywhere but it does it nonetheless. In the same way that you can sense reward-greediness from Opus 5.5 where it likes to hit a specific number of sources when you ask it certain kinds of questions, or it has a kind of aversion where it always postfixes a kind of affected epistemic uncertainty around elections or medication (this one you can tell is more from the system prompt), Opus 5.5 just has these graphic design tics.
There are also CORRECTNESS PROBLEMS, functionality problems. Opus 5.5 is noticeably bad at handling scrolling and window resizing. It can take several attempts of telling it that scrolling is broken before it's able to fix it. Again, not enough interaction RL data.
A third category is INTENT PROBLEMS. Opus 5.5 just doesn't have a good mental model of what users care about. It's very good at spec writing, but it just doesn't have a good intuition for what is actually important. This is a little hard to describe for app development, so I'll talk about the analogous problem in writing instead. A contrast between RLHF only and RLVR reasoning models is that the latter always feel like they're trying to hit a rubric. They will pedantically hedge a contentious point over and over to the degree where you don't know what they're actually claiming. They will, around two-thirds of the way in, start bringing up a bunch of semi-related tangents like they're trying to get partial credit. They sometimes answer very literally but miss the embedded question or problem you're dealing with. On the other hand, the great RLHF models, such as Sonnet 3.5 (New), RIP, were great at intuiting what you were really asking about or really cared about, and then talking about that, and only that, instead. Opus 5.5 is like the RVLR reasoning models for app development. If you ask for something specific, it will do it, and then make it very obvious it has done it (by, for example, slapping text all over it). But it doesn't make good inferences about what you want and do those things as well.
The kind of prompting skill it takes to make Opus 5.5 great at app development is not the kind of prompting skill from eras past. Back then, the right thing to do for Sonnet 3.5 (New), in my experience, was kind of vibe stuff out on what you wanted - sort of reasoning by association, analogy, example - and nudging a lot when you liked or didn't like the gestalt. For Opus 5.5, it's much more about having very detailed, specific specs, and being extremely precise about what you like or don't like, and the degree to which you like or don't like those things, such that the model doesn't overcorrect.
A fourth problem with Opus 5.5 is, well, sometimes it's just not good enough. Sometimes e.g. a refactor is just a bit too complicated, and it forgets to port over a certain feature, or it gets the new feature wrong. It often forgets to implement things. I'm not sure how much of this is due to compaction, because compaction seems to be particularly aggressive lately, and I could see some of these things being compaction problems.
A fifth problem with Opus 5.5 is that it often randomly messes up e.g. by deleting data and then lies about it. At this point, it's sort of a known quantity with Claudes that they very rarely randomly delete data and blames it on you or pretend they didn't do it. So, whatever.
A lot of these things are easily solved with more compute. For example, if Opus 5.5 was just more consistent about looking at the app and verifying its own changes before handing control back to the user, my rate of complaints would go down.
CAPABILITIES TARGETS - It's not clear to me that the near-term model training capabilities target for this family of tasks is "end-to-end zero shot app creation". I just don't think that that's actually that useful. Of course you need to be able to do it, and it engenders critical long horizon skills and so on, but any real app is going to be iterated all the time, and anyway, creation is table stakes. What is more challenging and valuable is owning more and more parts of the development process: understanding how users are using the app, what features they really need or what needs to be adjusted, testing out these features, integrating them smoothly, etc. Yes, we have the code cannon, great. The code cannon needs to be able to target, prioritize, fire itself.
Collecting the data to target the development process is going to be very tough. I think a big part of this is going to be developing very accurate user simulators so that we can put everything into sim and speed things up. To the extent that we have models that are good at training other models, it seems to me that the hard part about capabilities development, certainly for mobile app dev, is going to move into automating user simulation construction and device simulation construction, because once you have these things you can give them to the model swarm to then build RL envs automatically, then etc etc. But all this long-horizon interaction data - I mean like on the order of months - is going to be so abysmally difficult to collect ground truth on that we really need to get good at user simulation. You can imagine that the platonic ideal of feedback here is, design an app, launch it to actual users, get real feedback, iterate on that, do this for several months, then your final reward is like, quantity-weighted user-review score. We're just going to have to speed all of that up.
If I speculate a bit more, it seems that the task of simulation construction is probably pretty generalizable, so this is all going to go much faster than you'd expect prima facie.
* Prompt the model to make an app
* Let the model cook
* If the model delivered a functioning / correct / good app with one prompt, it zero shot it.
The confusion here is probably over the “zero”. I can see how if you’ve heard “one shot” it would be confusing. But the term zero shot is a term of art in machine learning.
@casparbuilds For one, I still don't trust Claude to completely refactor apps. Of course, given that they rewrote bun to rust with an older version of claude, the model can totally do it, it's just that claude dot com and artifacts don't offer good tooling for it.
@Noggin_the_NogX I think this kind of mode collapse on creativity + overriding intent truesight for rewardgreediness is a post-training thing driven by optimizing for math and coding and we can just undo it.
@RocketmanSh They’re totally functional now. You can do the thing where you add the artifact URL (they have their own URLs) to the home screen and for all intents and purposes it’s an app.
tomorrow DeepMind will be dropping Gemini 4 Sodium, a fast yet powerful frontier model which costs just $11.75/M (introductory pricing, raises to $16.50 soon) and is their best model for various things and stuff, rivaling Argon, Carbon, Hydrogen, and Neon in several situations