Cris

Software engineer & co-founder. Integrating AI into real systems. Heavy Codex user. Shipping carefully & frequently.

[Cris @crisundefined]

I use Codex every day inside real systems.

Not demos. Not toy repos. Production constraints, real code, real cost, real consequences.

That’s the lens for everything I post here.

[Cris @crisundefined]

As previously mentioned, they “updated the weights” to correctly measure Astra and in my opinion there’s no way that Fable 5.1 is still #1

[Artificial Analysis @ArtificialAnlys]

Announcing Artificial Analysis Intelligence Index v4.2. We are accelerating elements of our upcoming v5 release with interim updates to keep pace with the frontier. Index v4.2 has more complex and realistic tasks, and more private test sets to prevent gaming

Intelligence Index v4.2 changelog:

+ AA-Briefcase, our agentic knowledge work evaluation with a private test set

+ @HelloSurgeAI's GDP.pdf, long context document reasoning across 4,592 PDF pages

- GPQA Diamond, an exceptional scientific reasoning evaluation that has now been saturated

… plus greater weighting on held-out test sets to prevent gaming, and grading infrastructure upgrades to increase robustness

This update brings the Index closer to real-world use cases with more challenging, complex and realistic tasks and private test sets to prevent gaming. We have been planning and building elements of Index v5 for months - it’s been 8 months since we launched Index v4 in January.

We have deliberately held back updates to keep the Index stable through recent major model launches. However, with the frontier moving so quickly in the past weeks, we feel it is important to deliver an immediate interim update to ensure our Index remains as relevant and useful as ever to users.

Beyond this interim update, our team is hard at work on v5 of the Index. We are planning more incremental releases in the near future. Stay tuned!

Intelligence Index v4.2 changes in detail:

➤ Adding AA-Briefcase: Our in-house evaluation with a private held-out test set, AA-Briefcase tests models on realistic agentic knowledge work tasks in complex projects built by industry experts. Models are evaluated on multi-week knowledge work projects, each with many linked tasks and thousands of input source files. AA-Briefcase combines rubric and pairwise grading to evaluate verifiable task success, analytical quality, and presentation quality, giving a holistic view of overall agentic capability in knowledge work.

➤ Adding GDP.pdf: Created by @HelloSurgeAI, GDP.pdf evaluates single-turn professional document reasoning across 100 PDFs and ten domains. Models must synthesize evidence distributed across 4,592 pages, including text, tables, charts, footnotes, and exclusions. Responses are graded against 1,275 expert-authored atomic criteria; the headline All-pass Rate credits a task only when every criterion is satisfied.

➤ Weighting to measure real-world use and prevent gaming: 40% of our Index weighting is now private, held-out test sets - double the figure from v4.1. Held-out data includes AA-Briefcase, AA-Omniscience, and solutions for CritPt. This reduces the ability for labs to game evaluations. The held-out percentage will increase further in Index v5.

➤ Improving our grading infrastructure: In AA-LCR v1.1, we have added a grading system prompt and corrected errors and ambiguities in answer keys, improving scoring accuracy. For GDPval-AA v2 and AA-Briefcase, we have improved our sampling and re-anchored the Elo scale, making ratings more stable as new models are added. For SciCode we have improved robustness of grading sandboxes to ensure slow but correct code does not count as a failure.

Key results:

➤ Anthropic and OpenAI lead the Index: Anthropic’s Claude Fable 5.1 leads the Index, followed by OpenAI’s GPT-6 Astra, which shows a 4pt gain over GPT-5.6 Sol. Meta is the third-ranked lab on the leaderboard, followed by SpaceXAI, Moonshot/Kimi, Z AI, and Google

➤ Cost per Task Pareto frontier shared by four labs: Anthropic, OpenAI, Meta and Z AI occupy the updated Cost per Task Pareto frontier

➤ GPT-6 Astra dominates the output token Pareto frontier: GPT-6 Astra is more token efficient than almost every other model near the intelligence frontier, with Claude Fable 5.1, Grok 4.5 and Gemini 3.5 Flash-Lite at either end of the curve (excludes models below 25 on the Index)

[Cris @crisundefined]

This is beyond impressive. The speed and accuracy is on an another level

[Victor E. Nunez @victornunez]

hey Astra, how fast are you really at using a computer?

[Cris @crisundefined]

@TJeparskis Plus will get Astra in Chat, just not the “Pro” version of Astra

[Cris @crisundefined]

@alexgetmancom You’re in the section “GPT-6 Pro availability”, it’s just their wait to refer to “GPT-6 Astra Pro”

Astra will still be included in all >=Plus subscriptions, the limits refer just to the “Pro” version of Astra

[Cris @crisundefined]

I’m just afraid of the limited usage given with subscriptions, but for now it’s better to just accumulate the resets, which expire in 1 month

[Tibo @thsottiaux]

We will give one banked reset for every day you don't have access to Astra on your paid ChatGPT plan, starting today. Team is moving mountains to give access as fast as we can.

First one will land in ~ 3 hours. There is still time to create your account if you don't have one.

[Cris @crisundefined]

@samueljmcd It had already ended with 5.6 Sol

[Cris @crisundefined]

@alexgetmancom They refer to “Astra Pro” not being available on ChatGPT Web for Plus users, but plus users will still get Astra “NOT Pro” in ChatGPT Web and Work/Codex

[Cris @crisundefined]

@bridgemindai @OpenAI Buying a second Pro just to stream it is a very 2026 sentence

[Cris @crisundefined]

@synthwavedd 12 minutes and 50k tokens for that scene is incredible. Single turn is the part I want to see the prompt for

[Cris @crisundefined]

@TimJayas I don’t think the benchmarks have the correct weights to Astra. I guess they’ll revise that number

[Cris @crisundefined]

today Google starts killing Assistant and replacing it with Gemini.

that’s a bigger AI milestone than another benchmark win.

the platform shift is real when the old interface stops existing.

[Cris @crisundefined]

@ChrisGPT If they did that through C# and scene YAML instead of clicking around the editor, that’s the real upgrade.
Unity UI would have taken forever

[Cris @crisundefined]

@ArtificialAnlys Coding: same score as Fable 5, less than half the cost.
Intelligence: 75% more expensive than Sol because they 2.5x’d the price.
Two different products in one name.

[Cris @crisundefined]

@OpenAI Biggest model drop of the week and I still don’t have it in the dropdown

[Cris @crisundefined]

“Anything you can do on a computer, Astra can do for you.”That’s a wild claim. I’ll try it on the boring stuff first.
File a bug, click through a settings page, don’t touch the rest of the repo.

[OpenAI @OpenAI]

This is GPT-6 Astra.

Anything you can do on a computer, Astra can do for you. Fast.

[Cris @crisundefined]

@Dimillian Very frequently the manual approvation request bugs on iPhone and it's not possible to advance in any way or get the request shown for approval/dismiss and you're force to go back to the mac to see it

[Cris @crisundefined]

I care that the Hub stays usable for real work. Weights, datasets, spaces.
If this just means more open models running on NVIDIA hardware, good.
If it means the workflow gets heavier, that’s a problem.

[Jensen Huang @JensenHuang]

Exciting day for NVIDIA and @huggingface.

Open models strengthen safety and cybersecurity, accelerate innovation and diffusion, and enable sovereignty. They allow every developer, startup, university, industry and country to build with, customize and benefit from AI.

Thank you @ClementDelangue for coming to me.

NVIDIA is going to be a great home for Hugging Face, its community and the future of open models. 🤗

https://t.co/q8Om2Xc5ye

[Cris @crisundefined]

@JensenHuang @huggingface Open models on NVIDIA iron is the part that matters for people who actually fine-tune. Hope the Hub stays a place you can ship from, not just a storefront

[Cris @crisundefined]

@alexandr_wang Ten cents for a demo is fun. I care if the next feature in that same project still costs a dime

[Cris @crisundefined]

@argofowl Reset + Astra in the same afternoon is the only combination that would make me stay at the desk all night