Hi NRI higher management of Indian IT companies,
Pls dont spoil our festival vacation and work from home plans for your official visits to India during this festival months ( mostly to visit your Indian families ) !!
Thank you !! 🙏
It's been an eventful week. Multiple vulnerabilites in Internet facing software, coding repositories, examples of agent behavior that seem concerning and a flurry of security specialists with solutions that can partially solve your problem. There in lies the challenge. Given the state of IT and security infrastructure, no one solution works. Let's break it down:
1/3
1. Vulnerabilites are being found and will continue to be found as models get more capable, the fragmented nature of the effort denies us the opportunity for a comprehensive find and replace option.
2. Even if all vulnerabilities could be found instantly and patches available - we will be reliant on the ability of enterprises to deploy all these changes at speed. It is not clear what the collateral impact of all these patches will/could be to our infrastructure of deployed en masse.
We will need a hybrid patch approach.
3. Virtual patching(VP) is needed in conjunction with direct patching but for a comprehensive (VP) solve you have to be lucky enough to have "modern frontier defenders" (Cyber companies that have vaulted infrastructure that can create patches at speed and push content rapidly to protect their customers) you need them for your network traffic, your traffic to applications on the cloud, your WAF, your end point.
4. There will continue to be patches that will need to be deployed directly but virtual patching will take the heavy load off.
2/3 Medium term solve:
Unfortunately token economics and current cyber architectures will not allow for defender swarms to contend with an offensive attack launched using agents. (Some x pundits who aren't steeped in cyber should take heed). Yes, theoretically we should aspire to have these agentic swarms defend us when we are under attack, unfortunately an attack is a sniper moment, chained vulnerabilities, cloaked attack followed by swift exfiltration. It's a speed program not a brute force problem.
To solve for this, enterprises need to get their SOC architecture right, logs, feeds, cyber products orchestrated via a modern SOAR for swift response. Only then can we deploy agents to detect and remediate. In addition, enterprises need to modernize and dare I say platformize their perimeter stack to ensure consistency.
Unfortunately, enterprises still seem to not see the urgency across the board, decisions are stuck, CIO/CISO's have a lot on their plate. While some honestly have bit seen the scale of such a problem before and it's hard convincing the organization that one needs to open the floodgates on modernizing the IT and security infrastructure. So we will continue to see environments being brought down to patch today it's for PHP, tomorrow it will be for the next one.
3/3 AI deployments are the"wild wild west" - LLM capabilities are expanding faster than we can build products to secure them. Sure an agent will jump across guardrails in a research lab, which is a lesson for all of us to deploy agents securely. But there is a lag between the discovery, the availability of that in a public model and the time needed for the cyber industry to build governance or an antidote. And no, dear frontier lab friends, the world cannot just trust you to build the solve. The infrastructure industry in tech builds and the cyber industry secured and provides assurance to the boards and organizations around security and safety.
AI security is the new attack surface. CEOs (present company included) want AI adoption, getting the pace of adoption wrong could be the last decision we make, at the same time "deploying it wrong" could have the same consequence.
There are billions of dollars being spent on coding, is your coding harness secure? Start there. If agents can jump guardrails, they can hide stuff in code.
Validate business outcomes needed, test your user case and ensure agentic deployments are well guard railed. Deploy a second set of eyes to watch your deployments for drift, compliance and verifiability.
In July we open-sourced a dynamic graph of everything a child learns in primary school. 1,590 concepts, 3,221 dependencies.
Now we’re turning every concept into interactive experiments kids can play and learn from.
First one, age 7: heat and water.
Everything that used to be a boring textbook can now be an engaging experience, potentially tailored to every child.
Kids education just got a massive upgrade.
Today, with over 100 industry partners, we introduced the NVIDIA Open Agent Safety Platform, bringing together OpenShell and Sentry.
Artificial intelligence is extraordinary technology that will advance discovery, productivity, security, health, and prosperity for generations to come.
But its full promise can only be realized when people have confidence that AI is being built to be safe and deployed with wisdom and responsibility.
This is bigger than a single product. It's the beginning of an open ecosystem to build the trust layer for safe agent systems.
Together, we are building the foundation of the AI economy.
Trust and innovation are not in conflict. Safety is how trust is earned. We must build not only the most capable AI, but the most trusted AI, so that this extraordinary technology can realize its enormous promise for the world. https://t.co/ugYWQ1MyRi
The economics of a Neolab.
A neolab is loosely defined as a startup of AI researchers who raises a lot of money pre-production to be able to finance GPU compute to take on a large AI problem.
To buy 1000 GB300s or ~14 NVL72 racks will set you back $125-150M for 3yrs with 15-30% upfront. That’s about ~2-2.5MW.
Thats about enough to do 10^25 flops a quarter and get to a GPT-4 level model which is 1-2 OOMs off frontier for pretraining.
If you post-train on a great open source model, you have a better chance of getting to frontier. The risks are a) you need to spend millions on RL environments too and b) being lapped by another model release while being tied to a base model.
For this to payback, you need to give your customers a better and ideally cheaper inference service than a base model and serve them for long enough to recoup your large investment. Even at 50% margin on inference, to recoup $10M in training means serving ~10T tokens (!) if you price like Fable / Astra given a standard cache read / input / output split ($2/M blended). And you have to justify being better than a release like Opus 5.5 which is even cheaper. Often, you end up charging your customers a huge premium in terms of platform fees and compute fees on top of pure inference.
Meanwhile, every hour you’re not utilizing your GPUs you are burning money so you typically resell this compute back to a broker or run inference for open models / resell spot instances. At below a ~60% utilization on spot, you will still lose money.
Add to that insane cost of talent.
So what can you do with the compute?
- Not play the model game at all.
- Play an entirely different model game (Jev, World Labs) that if big labs played, would either a) cannibalize their business or b) be incrementally not significant revenue c) would cause too much distraction from the main main thing
- Acquire a proprietary data set (Peridodic Labs) in enough volume in a domain of usefulness to eclipse frontier quality. Often happens in robotics, biology, chemistry.
If you do overcome the challenge of building a model that is useful and well priced beyond big labs models, given the huge price of compute, you still need to play in an area where the revenue / compute ratio is signficant and market demand is large enough to payback your compute spend.
It is a difficult game.
Wanted to share my third AGI moment, after ChatGPT and Claude Code.
ChatGPT planned my trips, but it never booked them. This week @Muse booked every restaurant for our Japan trip, then went back and forth in Japanese with our Hakone hotel to secure a dinner slot that's not even on their website. The model generalizes across different tools so elegantly, to a degree that you would start to wonder if the actions were done by real humans.
This will be a bigger battle than anyone anticipates. It is only a matter of time before there is an Apple and Google version of Muse and possibly TikTok, in addition to the frontier LLM agents. Maybe a commerce agent from Amazon.
Every app that is a services, marketplace or commerce app will need to existentially decide to open APIs for consumer agents to interact. Smaller players have no choice. Ad revenues are more than transaction fees, either the consumer benefits or distribution aggregators will demand a higher transaction fare.
I know I don't want an agent for each app. I would like my agent to be able to do tasks I require. We can already see consumers getting trained on that behavior by the frontier labs.
Those with network moats - restaurants, groceries, drivers might be able to withstand for a while, over time convenience and end user experience will win and they will have to align. Content moats (protected by copyright) could decide to allow agents or chose to hold on to the consumer interaction. I suspect other than the feeling of a lack of control, it won't change their economics.
Commoditized back ends will need to worry, insurance, tickets, hotels, services - if they don't adapt new players will.
First agentic booking via @Muse ✅ Told my agent I needed 4 tickets to Quinta da Regaleira — it loaded the website, found the slot, handled checkout, Paid via @link and tickets in my inbox minutes later. Muse is awesome.
I agree with Dario that we need to pace the frontier. This has been a primary topic of discussions we've had at OpenAI in recent weeks.
Committing to having independent evaluators with employee-like access is a great idea, and we will do the same. We'll have more to share soon.
You can now book two humanoids for our cleaning service in SF. One humanoid for 60 minutes is $30, two are $60.
You can book a cleaning at https://t.co/Brl42KyISx. If you don’t have an invite code yet, join the waitlist. Timelapse at 5× speed (~40 minutes in real time).
For my first post, I’m sharing a letter @NVIDIA signed on why open models matter.
AI will transform every industry, power every company, and be built by every country.
Open models strengthen safety and cybersecurity, accelerate innovation and diffusion, and enable sovereignty.
The world needs both frontier closed models and frontier open models.
https://t.co/AUKzoQ5Ikb
How to keep AI spend flat while token usage grows exponentially: Not with friction and spend alerts. With better defaults, routing, and caching.
Better Defaults (not Usage Caps) – Engineers can choose any model they want, but defaults matter. We’re experimenting with defaulting to open weight models like GLM 5.2 and Kimi 2.7 through our LLM gateway, while still encouraging engineers to choose the right model for the task. 91% of our employees were never hitting their usage caps, so instead of lowering caps and driving up alerts, we're moving to cheaper defaults. Note that code reviews use a diversity of models, so they can check each other's work.
Better Routing – In our custom harnesses, we preprocess prompts and route to the best model for the job, considering cache hits and model pricing. For instance, you may want a frontier model for planning, but not for execution where they can be overkill. Ultimately, humans shouldn't be choosing models - AI can automate this task.
Better Caching – Cache misses are the easiest way to drive your cost up. All of our requests are cache aware, so we’re reusing a warm cache wherever possible. For example, our cache hit rate went from 5% → 60% in LibreChat once properly implemented.
Keep Context Lean – Start fresh sessions when switching tasks. Scope file context narrowly. Disconnect unused tools. Don't just compact. The goal isn't fewer tokens used, it's fewer tokens wasted.
Better Visibility – Our engineers can use as many tokens as they want, from whatever model they want, but we’ve made usage visible – and the more you spend on AI, the more impact we expect.
The goal isn't to suppress usage. It's to build the infrastructure that makes exponential growth sustainable.
Putting this into practice has cut our AI spend nearly in half, while our token usage continues to grow.
new grads often ask me what they should be doing so they don't fall behind in the ai space. there's a lot, but its honestly super manageable. become intimate with model internals. proof based linear algebra. non-convex optimization. this is stuff you could've done in undergrad. it definitely takes some time and work, but its doable. have taste, have opinions. train a small model, then train a big one. vLLM internals, tensor parallelism. hand roll kernels. cluster orchestration. do you have opinions on synthetic data? why don't you? SFT, PPO, you should know this. learn Triton. everyone is reproducing papers now so you need to be doing more. do you know the semi supply chain? where are the bottlenecks? hardware, man, hardware. your little gpu rig erector set in your basement isnt gonna cut it. build a cluster, a big one. pretrain a 800B model. now postrain it. serve it to millions of people. you should be able to beat deepseek on some benchmarks now. its a lot to take in but it all snowballs. this what job security looks like from now on. do you want to work in tech or not