Labib Chowdhury

ML | NLP | Vision | MLOps

[Labib Chowdhury @Labib747]

@zackpaid FACE

[Labib Chowdhury @Labib747]

@CanavanCathal @Rames_Jusso @HeyGenDev @HeyGen Wow

[HyperFrames @HyperFrames_]

we added a lot of features to the new HyperFrames Studio

here are 9 tips to help you get the most out of it 👇

[Matt Pocock @mattpocockuk]

mattpocock/skills v1.3 is out!

- /pr (new) writes easy-to-read PR bodies, showing hard evidence that the changes work and assessing merge risk
- /implement-spec (new) takes in a spec and tickets, and implements them with subagents
- CONTEXT.md renamed to GLOSSARY.md
- /retro (new) reviews recent coding agent sessions and suggests repo improvements

/retro, especially, feels like a huge upgrade. Enjoy!

[Labib Chowdhury @Labib747]

@Hackwithseries 🙋🏼‍♂️

[Daron Acemoglu @DAcemogluMIT]

Third question on AI.

A question that also remains unasked is whether the AI boom can continue without leading to a massive increase in inequality.

A recent paper by Stijn Van Nieuwerburgh runs the numbers on how much revenue the AI industry needs to generate to recover its massive investment (summary and a link to the paper can be found here: https://t.co/QIGcoObcVc).

Van Nieuwerburgh’s arithmetic should make us more concerned. AI investments will average about 3.6% of GDP annually between 2025 and 2032. Van Nieuwerburgh calculates that, using a 10% rate of return, the industry would need to generate annual revenues of about $3.7 trillion by 2032 to recover these costs (growing from its current levels of about $200 billion or so). That is significantly more than 10% of current US national income, and will likely remain around 10% of national income by 2032, even if GDP growth rose from its current level.

A large fraction of this revenue will go to capital income. That means a massive increase in the share of capital in national income, which has already risen substantially over the last 25 years or so – now standing at an all-time high of about 47% (https://t.co/IuYldl173r).

Capital income is much more unequally distributed than labor income, so a massive increase in the capital share of national income will translate into a very sizable surge in inequality.

The rise in inequality may not stop with the capital share. My work with Pascual Restrepo documents that (automation-driven) increases in the capital share of national income are typically associated with rising labor income inequality as well (see, for example, https://t.co/2D9KUL3QfM). The same may happen in the next several years, boosting inequality further.

What is missing from our current debate is any discussion of a fundamental dilemma these numbers pose: can the AI boom avoid both an economically costly crash and a huge increase in inequality?
If the industry reaches these revenues, inequality surges. If the industry does not become profitable, a crash, with substantial costs in terms of lost output and jobs, becomes likely.

My assessment would be that the industry is unlikely to reach levels of revenue Van Nieuwerburgh calculates. First, diffusion has been and will likely continue to be slow. Second, competition from open-weight models, which are getting better, will limit how much proprietary models can charge. Third, despite important advances, I still believe that AI models will not be able to automate entire occupations anytime soon, thus limiting their value to businesses as cost-saving devices.

Whether this leads to a crash or not is more complicated and will depend on whether various AI companies are bailed out and what kind of support they receive.

Nevertheless, even if revenues fall short of these gargantuan amounts and we avoid a dramatic surge in inequality, I expect that the diffusion of AI will push up inequality between capital and labor and within labor.

If inequality does surge, a further question becomes central: can our democracy survive such astronomical levels of inequality?

[Andrej Karpathy @karpathy]

We'll be spending a lot more time trying to understand the outputs of language models. A few thoughts, tips & tricks:

Writing. Something I've had success with: Ask your LLM to explain something in ASD-STE100, it's a controlled language specification originally developed for aerospace maintenance documentation. LLMs well-versed in this language and it comes with heavy constraints on clean writing style that I often find a lot more readable. Sometimes I've tried to soften it a bit e.g. ask for "80% of the way to ASD-STE100" because the spec is quite stringent. But even better:

Diagrams / images. Instead of writing, ask your LLM to create a diagram. These can be a lot easier to process, parse, and understand. But even better:

Web pages. Ask for output "in HTML" to get a beautiful, interactive webpage. LLMs are getting really good at frontend and can create beautiful experiences, animations, etc. But even better:

Explainer videos. The output format I am most bullish on is fully custom / bespoke explainer videos generated on any arbitrary topic. Experiment with things like "Create a 3b1b style video explainer on X. Use my ElevenLabs API key for audio narration". (you'd need an API key for the latter or you can ask your LLM to find you decent free alternatives that use your local compute). This is actually starting to work!

In summary:
- As LLMs get better, they will do more and more of the legwork autonomously, and a lot more of our work will rise up the abstractions into oversight and understanding.
- Luckily, LLMs can help here too because as intelligence and code are increasingly abundant, you can ask for large, custom, discardable software artifacts (e.g. web apps, video explainers) that would have never made sense to create before. Push the boundaries here and you'll be surprised.

[Labib Chowdhury @Labib747]

@AlessandroLavis Claude

[beginbot 🃏 @beginbot]

sub agents being released into my codebase

[Santiago @svpino]

5 design patterns for long-running agents:

1. Design your prompts carefully. Keep system instructions and tool definitions at the top and move dynamic memories and runtime data to the end. This enables prompt caching, cutting costs and latency.

2. Move any learnings on your harness to the background. Always reply to the user first, then process memories asynchronously. You never want the harness to slow user interactions.

3. Preserve files, installed tools, and unfinished work between sessions. Reattach agents to user-scoped environments instead of rebuilding them each turn.

4. Make any sub-agents return structured statuses such as completed, timed out, halted, or awaiting approval. Never return ambiguous prose that a parent agent might mistake for success.

5. Apply cheap, auditable security checks first, policy rules second, and human approval last. Normalize addresses and commands before checking them, and assume safeguards may eventually be bypassed; also isolate credentials and restrict network access.

This article breaks down each of these patterns:

[Peng Zheng @pengzheng_]

wrote down some of the design thinking behind Grok Bot.

persistent roles, clear state, scoped context, coordinated teams — an interface designed to move you from operating AI to delegating work.

https://t.co/37On6hzsyl

[Ilya Sutskever @ilyasut]

Neoclouds have limited cybersecurity. Next time agents successfully go rouge, they'll try taking over a neocloud to run more copies. This is bad.

Thus: neoclouds should greatly strengthen their cybersecurity and every company with strong cyber models should help with that.

[John B. Holbein @JohnHolbein1]

lol

[Labib Chowdhury @Labib747]

@OriSilver AGI

[diva @divaagurlxw]

As an AI Engineer. Please learn

>Harness engineering, not just prompt engineering

>Context engineering, not just long prompts

>Prompt caching vs. semantic caching tradeoffs

>KV cache management, eviction, reuse, and memory pressure at scale

>Prefill vs. decode latency and why they optimize differently

>Continuous batching, paged attention, and throughput optimization

>Speculative decoding vs. quantization vs. distillation tradeoffs

>INT8, INT4, FP8, AWQ, GPTQ, and when quantization hurts quality

>Structured output failures, schema validation, repair loops, and fallback chains

>Function calling reliability, tool contracts, argument validation, and idempotency

>Agent guardrails, loop budgets, tool budgets, and termination conditions

>Model routing, graceful fallback logic, and degraded-mode UX

>RAG architecture: chunking, embeddings, hybrid search, reranking, and freshness

>Retrieval evals: recall, precision, grounding, attribution, and citation quality

>Evals: golden sets, regression tests, adversarial tests, LLM-as-judge, and human evals

>LLM observability as a first-class discipline: traces, spans, tokens, latency, errors, and drift

>Cost attribution per feature, workflow, tenant, and user journey not just per model

>Safety engineering: prompt injection defense, data leakage prevention, and permission boundaries

>Multi-tenant isolation, cache safety, and cross-user context contamination prevention

>Fine-tuning vs. in-context learning vs. RAG vs. distillation and when each is the wrong tool

>Latency, quality, cost, and reliability tradeoffs across the full inference stack

>Production failure modes: hallucinated tool calls, malformed JSON, stale retrieval, runaway agents, and silent eval regressions

[Labib Chowdhury @Labib747]

Nicely written. makes me wonder what it means for countries like Bangladesh. We already have a very little footprint on the internet “subsidy”, moreover frontier companies are already collecting newly generated data thru their pipelines

[will depue @willdepue]

A Stargate for Data

Labs are on a trajectory towards >$100B/year of data spend by 2030. As we begin the trillion-dollar compute project, we need to think about the equivalent civilizational-scale effort for the other core ingredient: data.

At the foundation of the scaling revolution is a simple empirical law: deep neural networks improve smoothly, near magically, as you scale two things in proportion — (1) the size of the model and (2) the amount of data you train on. And despite the scaling laws being brutally diminishing, we’ve successfully bitten the bullet of logarithmic scaling with exponentially larger clusters and datasets, and received incredible new capabilities in return.

But this exponential scaling is bound to hit some limits. Oddly enough, compute has compounded fairly smoothly without limit, with trillions flowing into hypercluster buildout. Instead, we’re starting to hit the limits of an exponential demand for data. Gone are the days of being purely in the compute-limited regime, where we had effectively infinite internet data but never enough GPUs, we’re now entering a data-limited regime.

Luckily, this limitation is coinciding with staggering improvements in AI capabilities. Incredibly, we seem to have a real line of sight towards automating a majority of knowledge work with the methods we have today. RL + pretraining, and the data for each, will be generally sufficient to achieve most economically valuable tasks, given some minimal algorithmic progress and continued compute scaling.

In a data-limited world, economic progress & scientific acceleration will be directly bottlenecked by our coverage in each domain. We need to see data collection as imperative, deserving the same civilizational ambition we’ve given compute.

The internet as a one-time subsidy

It’s underrated how much all progress in AI owes everything to the blessing of the internet, this one-time civilizational subsidy to deep learning, decades of unintentional accumulation of a perfect dataset: every book, blog post, image, video, paper, discussion, etc. all digitized and freely available. Without the internet, we’d likely see comparably minimal progress in AI today, and in fact, if you notice where systems currently underperform, it’s almost always a domain where web coverage is limited and data is private, expensive, non-digitized, or non-existent.

But we’re running out of it. There are only about 300 trillion tokens of useful public human text, and the internet doesn’t produce nearly enough new high-quality data to match what scaling demands — we’re soon to hit the limits of public data for pretraining. And though the advent of RL bought us reprieve — chain-of-thought RL needed a new form of untapped data, gradable math & coding tasks, also available online — we’re quickly running dry of hard tasks for RL as well.

Why do we need so much data anyways? Humans learn comparably in far less time, needing just one textbook where language models might need the equivalent of hundreds to learn a new topic. It’s possible we discover methods that are massively more data efficient — synthetic data, data efficient architectures, other exotic algorithms — but fundamental progress is slow and highly unpredictable, and the recipe we have just works today.

And, while I’m wary of getting too deep here, even arbitrary data efficiency can’t replace data that just doesn’t exist in the first place. There’s a massive amount of missing information on the web: the dark matter of the internet — tacit knowledge, undocumented processes, etc. — most of which was never published and lives only inside organizations, the physical world, or just in people’s heads. I’ll leave it here and say, for reasons far longer than I can fit in this post [1], it’s best to operate on the assumption that our insatiable desire for data will continue as it has for the last decade.

There will be >$100B/year in data spend by 2030

We’re not screwed yet, of course. Only a fraction of useful data in the world is on the public internet, the rest is stored inside private datasets, corporations, personal archives, universities, governments, and otherwise. Labs can and will continue to license these private datasets, or create them from scratch, like Anthropic’s book scanning project. And we’ll increasingly task human experts to manufacture new high-quality data, with a large fraction of hard RL training tasks already being sourced this way.

But collecting this data, unlike before, will be expensive. As the free internet dries up and demand for data rises, we should see labs investing equally in data as compute, likely spending a significant fraction of their compute budgets on data. As we see trillions spent on compute, we should also expect hundreds of billions spent on data (human data & collection budgets), given their equivalent importance. And, notably, data spend is already tracking this way: total data spend across vendors, not counting internal lab efforts, is already roughly $7 billion per year. It’s quite reasonable we’ll see >10x by 2030.

Data is the moat

Data becoming increasingly private will also majorly shift the competitive landscape. While compute is a commodity — everyone buys the same chips and builds the same clusters — data really isn’t. The big reason why frontier models have felt eerily similar to one another, until now, is they were trained on substantially the same internet (pretraining data variability across labs seems pretty low). As labs diverge onto more exclusive, manually collected corpora, I think models will begin to increasingly diverge.

OpenAI pulling ahead in mathematics and Anthropic in cybersecurity isn’t an accident. I really think laser-focused collection of high-quality midtraining tokens, custom RL tasks, environments, with dedicated research effort, has driven much of the visible progress in the last year. James Betker has an excellent blog about “the ‘it’ in a model is the dataset”: model architecture and compute buy you efficiency and order-of-magnitude performance, but ultimately, models, of any architecture, are such incredible approximators of their dataset that the core meat of a model boils down to just that, nothing else. Data is a major moat.

AGI long, ASI short

As I’ve tweeted before, I’m confident that, despite the narrative, the data labeling industry will continue to fuel great businesses and be an excellent AGI long, ASI short. The argument is just: By the time the AGI labs no longer need data, it’s probably over for everything else too [2]. In this frame, the last companies left should be the data companies, as the last speck of economically relevant data is sucked in. And these companies are already among some of the fastest-growing companies in history: Mercor, founded three years ago, is rumored to be doing $2 billion in revenue with something like a few million expert labelers under contract.

While these businesses are very non-stationary, what type of data is needed shifts constantly, I don’t think that diminishes their value. The long-tail of the economy is long, and the value isn’t diminishing as you extend farther into more obscure information: as models get more capable, the value of the marginal dataset goes up, not down. Automating a full job means covering its full distribution of tasks, tools, edge-cases, and long-horizon loops. There’s some O-ring logic to it: a dataset that buys a 1% bump can justify a previously unjustifiable collection cost when it’s the difference between a system that does 99% of a job and one that does all of it [3].

The competitive dynamics of the data industry are still evolving but as demand for data is increasingly niche, ultra high-quality, expert-generated, I think we’ll see real consolidation. Again, contra-narrative, we’ll probably see true competitive differentiation built on brand, quality control of data (which, from personal experience, can vary massively), as well as in network effects from the talent networks themselves over time. We’ve already seen rapidly shifting data type demand work in favor of incumbents, benefiting those with early knowledge of where the market is headed.

The binding constraint

It’s truly remarkable that we seem to have the recipe — pretraining + RL — to absorb most economically valuable work, despite being far from a lot of what we expected from “AGI”. The same way chess engines revealed we never needed general intelligence to solve chess, as we originally thought, we’ll soon realize that software, mathematics, and the vast majority of the economy (including physical, just running ~3 years behind!) are the same. If recursive self-improvement or some other algorithmic breakthrough arrives, that’s wonderful, but we really don’t have to wait for it. The binding constraint between here and an automated economy isn’t that, it’s data coverage: every app, workflow, edge case, process, etc. sitting in private stores or someone’s head.

Ultimately, while we make tremendous strides in more efficient model architectures, and clusters like Stargate equip us with zettaflop-scale compute, we really aren’t making rapid progress collecting the data we lack.

We’ll soon live in a world where we have the methods & compute to accelerate scientific progress or economic growth, but not the data. And we’re already there today: frontier models would surely be as good at accounting/many medical tasks/legal advice as they are at software engineering if we only had the same pretraining & RL coverage as we did for code.

I really want to drill this in: The speed at which we automate the economy is going to be directly rate-limited by our ability to collect data about it.

Worth noting that under this assumption, with data as defensible and directly proportional to economic & scientific progress, data should also be considered a national strategic asset like compute. Imagine what we’d do in a world where we had a Manhattan Project-effort for AI and needed to mobilize data collection as a limiting factor. We should be concerned about China, with greater state capacity and authoritarian economic control, being capable of mobilizing data collection at national scale, potentially compounding their economy and scientific output faster than us down the line.

A Stargate for data

I’m leaving my complete ideas for a future post, as this one is already far too long, so I’d really like to pose the question here. Stargate exists because we organized trillions of dollars, international strategy, gigawatts around compute as a fundamental ingredient. What would equivalent ambition look like for data?

Obviously, scaling data collection, a heterogeneous mass of information across the economy, isn’t going to be as clear as scaling compute, as a homogenous infrastructural effort. A core division will be first, coverage — all uncaptured knowledge sitting across the economy/science/physical world and all that simply isn’t recorded — and, secondly, sheer volume in the domains we already train on: more hard math tasks, more high-quality web text, way more coding data, more legal drafts, etc.

I have a post coming soon which breaks down my proposals. There’s a lot of room for creativity. Quickly, we’ll probably want to start with a deep census of what we have and what we’re missing, predict what the 2030 model will still be bad at and work backward to what we should be collecting today. You can probably license a large amount, leveraging high lab valuations to buy datasets or companies altogether. There’s an adversarial nature to a lot of this collection with firms, so there’s lots of engineering to do this correctly. We should go convince important companies to turn off deletion policies, even if we’re not buying from them yet. Data flywheels in consumer products will be massive. Confidential training, government legislation for grant-funded research, running companies at a loss for their data, etc.

We’re headed towards hundreds of billions in expenditure, national prioritization, and major data limitation on the horizon. We have a great opportunity to think creatively about what a megaproject for data would look like: How do we, deliberately this time, construct the next internet’s worth of data?

Footnotes:

[1]: I’ll probably soon publish my much longer post explaining my position on data efficiency and why the value of this data is still pretty high in most worlds regardless of new algorithms.

[2]: The “AGI freeroll” bet: heads you win, tails ASI flips the world upside down anyways.

[3]: We already see a glint of validation of this point, given the data market is strongly tilting towards ultra-high-quality agentic data, rather than unskilled labeling — niche expert workflows, live environments, and evaluations requiring increasingly obscure talent & knowledge — yet shows increasing, not decreasing, revenues.

[David @dzhng]

Fed this article to Fable and we created an explore-unknowns skill

It scans your codebase, then interviews you one question at a time to close the known unknowns

Then sweeps for the unknown unknowns you never thought of

Now part of my software factory: https://t.co/WcLfM9NhP6

[Thariq @trq212]

https://t.co/hPiZr1kG7r

[Qwen @Alibaba_Qwen]

📣📣 Meet Qwen-AgentWorld — a native language world model that simulates 7 agent environments (MCP, Search, Terminal, SWE, Web, OS, Android) within a single model. Environment modeling is the training objective from day one, not a post-hoc adaptation.

🤔 LLMs are trained to be better agents — better at acting in environments. But nobody has trained them to model the environments themselves.

🗺️ Our roadmap: investigate how language world modeling can push the boundaries of general agent capabilities, along two routes:

1️⃣ Build a foundation model for environment simulation — outperforming Claude Opus 4.8 and GPT-5.4 on AgentWorldBench

2️⃣ Investigate how world modeling enhances agent training:
🔬 Controllable Sim RL (agentic RL with LWM as environments) surpasses training in real environments
🧠 Learning to predict environments (LWM warm-up) makes agents stronger — remarkably, even without any agent-specific training, this predictive knowledge transfers to agentic tasks with zero fine-tuning

📑 Paper: https://t.co/Jx2l5RKq71
📖 Blog: https://t.co/7tVcKyhsx2
💻 GitHub: https://t.co/B5Lvb1UZCn
🤗 HuggingFace: https://t.co/Kw3QBL1TM5
🧩 ModelScope: https://t.co/YBnGYgMWWI