@rezoundous There's no conspiracy. I'm a huge GPT fan. I've worked with 6.2 models for past 18 hours. I can confirm something's cooky with it. There's a ghost in the machine. Somethin' aight right with Uncle Astra.
"Anthropic is preparing a model that it hopes can guess what you meant to ask it. The Information reported on October 19 that one tester described the unreleased model as 'especially good at discerning a person's underlying intentions and goals even if they don't give the model much information, such as explicitly stating those goals.'
The same person told reporter Stephanie Palazzolo the model is 'better at thinking outside of the box rather than regurgitating common sense advice or solutions.' The example offered is specific. Asked how to invest in stocks, instead of generic advice such as looking at price-to-earnings ratios, 'the model might come up with its own creative algorithms to predict future stock performance."
-- If this is true, it could be a game changer for Anthropic. Arguably this is one of the most critical missing parts of working with GenAI, if not the most important. Outside of creating a model that can do my laundry, that is.
@TravisLongevity Not gunna lie, that's never once happened for me where any GPT model produced something more visually stunning than a Claude model. .. and I'm a huge GPT fan.
@skirano Can you explain when you say issue with Harness? Because my harness worked flawlessly with 6, and 6.1 Astra Ultra. 6.2 Give no fuggs about my RAG architecture, and just going HAM. Hard stopping like a luna model, ignoring half of my prompts/asks, ignoring orchestration
@GMorseCode@Gegam245074 Yeah it's not. 6 Astra Ultra had zero issues adhering to my guardrails and RAG architecture. Zero. 6.2 straight up ignoring half my shit, bypassing agent orchestration rules, and completing half of what's being asked. Something's not right with it.
@surlymochan@Sergey_Rura Yeah, but just purely odd for a Frontier model on an Ultra setting. Like I don't think I even experienced this on any GPT Frontier model. It's more patternistic to using like a light model. Weird behavior stops too like Luna or Terra. Something just feels defectual.
I'm a big fan of GPT and Codex, and a regular troll of Opus and Anthropic models. That being said, I am struggling a little with the 6.2 models so far. They seem to be losing the plot more, ignoring my RAG guardrails that are strictly defined. I've caught Astra ignoring =/. I'm not getting warm and fuzzies from 6.2 unfortunately, kind of meh for me. I'm running on Ultrafast and things still feel slow.
@theo Mad respect to your candor and honesty. This is also how I feel about the Open AI team, and that's why they hold my loyalty over Claude. However, these models just didn't land in too many ways. It's unfortunate. 6.2.1 probably on its way shortly though given the response :)
@buildwithrajath Being in software long enough, I think they rushed it. "Last minute changes" is red flags all day long. Especially days before a major production release.
@jareddavisoba@thsottiaux Yeah, This is happening for me nonstop in Astra Ultra 6.2. It just keeps dead stopping like its a Luna or Terra model. Not following instructions or even simple RAG architecture outlined in projects. Seems weird. Think we need a 6.2.1 soon. :)
@thsottiaux Ok... but... it can't seem to follow simple RAG guidelines in my project on Astra Ultra 6.2, and it's ignoring half my asks/instructions. It's also breaking from multi-agent orchestration objectives and instead operating synchronously in its own thread(s) because it wants to.
@buildwithrajath Glad I'm not the only one. I'm experiencing this heavily on Astra Ultra 6.2. Wild for a frontier model to be ignoring hard RAG guardrails, constraints, and hiding things under the carpet. Is it cheaper$ tokens? maybe, but the churn rate is 4x worse and less accurate.
Seems either like it hit a worktree bottleneck, or a git bottleneck, I'm not entirely sure. The primary thread still held the objective, but it still felt like some core goal items I asked for were politely pushed under the carpet over what I asked / preferred to do. Here is what it said
Yeah, it's just weird considering Astra 6.0 never did, but Astra 6.2 doing it regularly all day. It's funky. This is on Ultrafast + Ultra thinking mode for both 6.0 and 6.2, so it's just odd. Even Sol 6.0 as a frontier in ultra didn't have these issues. Feels more like an opus-like thing to do.
So I have a "meta agent" flow if you will. It has clear defined RAG guardrails about what the primary thread should do, and what it should hand of to other work-tree threads. What I've observed so far:
-> Made its own decision to just stop handing off at times to those threads, and operate in its own context window. an Example of this is let's say i have an art / asset pipeline. It decided to ignore that whole flow and start synchronously generating art in the meta thread. (weird, dont know why). I asked it why and it actually just said it ignored the directive despite it knowing it should have followed it. I probed it more why and it said the meta thread wasnt building anything to trigger the handoff, which obviously was a lie because it did it in its own thread. so... idk. weird.
I'm not sure if its just trying to adjust moving from 6.0 Astra to 6.2, etc. But it seems unusual to me for an Ultra Astra agent to miss the plot.
There have been other instances, but the above is just one of them.