Rohit Rathore

Head of Engineering | AI Lab

[Rohit Rathore @agenticairohit]

I’ve spent the last few weeks diving deep into NanoChat. It’s a brilliant full cycle open source pipeline that demystifies the journey of building private LLMs and SLMs from setup, custom BPE Tokenizers and Pre-Training to SFT, and recursive RL.

The Weekend Experiment:
I applied a few workarounds to force the pipeline to run on a single laptop with NVIDIA RTX 5070 GPU. I honestly thought the machine might melt, the fans were screaming and GPU usage was at 100% all night during pre-training.

The Result?
While the GPT benchmarking didn't rival GPT-2 (no surprise there without an 8xH100 cluster!), the experience of navigating the full stack on lightweight hardware was invaluable. It’s a stark reminder that building AI models is often more about physics and math with compute power than just writing code.

What’s Next:
Designed a dedicated AI Server with higher-end GPUs to move past "fun experiments" and build a controlled private sandbox to test hypothesis at home office.

The goal:
1. Build a more robust Small LLM/SLM with better hardware and NanoChat.
2. Create autonomous agents capable of "vibe coding" and self-innovation based on set goals with autoresearch deep learning to improve the model and train in specific topics.
3. Evolve the model foundation to build better, more resilient products.
4. Explore integration with Quantum Computing to scale model & agent performance.
5. Take it to production grade framework.
6. Rinse & Repeat, Learn & Share.

A few more weekends and late nights of "screaming fans, liquid cooling & GPU heat" to see what comes out of this LLM & agentic framework.

I am sure many others would try different variant of this approach and share their experience as well.

Thanks @karpathy for sharing it with the community, and @garrytan for the post.

"The idea: give an AI agent a small but real LLM training setup and let it experiment autonomously overnight. It modifies the code, trains for 5 minutes, checks if the result improved, keeps or discards, and repeats. You wake up in the morning to a log of experiments and (hopefully) a better model."

— karpathy, GitHub - karpathy/autoresearch: AI agents running research on single-GPU nanochat training automatically · GitHub (https://t.co/XXwOYogxhT)

[Garry Tan @garrytan]

Karpathy just open-sourced autoresearch.

One GPU. 100 ML experiments. Overnight. You never touch the code — just write a Markdown file.

The bottleneck isn't compute. It's your program.md.

https://t.co/zcR77tlOgO

[Rohit Rathore @agenticairohit]

Everyone spent the last two years discussing which AI model is smartest. That topic is becoming less important.

The real shift in agentic AI is happening in the harness, not the model.

The harness is everything wrapped around the model: how it remembers things, uses tools, recovers when something fails, and decides when to check with a human before acting.

A raw model on its own just predicts text. The harness is what turns it into something that can actually complete a task step by step.

Here’s where it gets interesting. A lead agent can now hand off pieces of a task to smaller “helper” agents, each working in its own separate space with its own tools and permissions.

This means a big task doesn’t overload one single agent’s memory. You can even limit a helper agent to only read files and never edit anything, for example:

name: code-reviewer
tools: Read, Grep, Glob, Bash
permissionMode: plan (read-only)

There’s also a stricter safety layer called hooks. Instead of just asking the AI agents nicely to follow a rule, hooks are actual code that runs automatically at key moments, like blocking a dangerous command before it executes, or cleaning up a file right after it’s edited.

Because it’s code, not a request to the model, it can’t be talked out of it or misunderstood causing hallucinations.

At larger scale, one lead agent can coordinate dozens of these helper agents working at the same time, only checking in on their final results rather than every step along the way.

That keeps the whole system manageable even as the work gets more complex.

One more piece worth knowing: agents built by different companies can now actually talk to each other.

This works because of shared standards (MCP, A2A and others), sort of like how any web browser can open any website regardless of who built it.

These standards let one company’s AI agent hand off work to another company’s AI agent, or let an agent use a tool built by a completely different vendor, without custom one-off integrations.

If you’re evaluating AI tools in 2026, don’t just ask which model access is supported. Test what happens when something goes wrong, and how much control your team actually has over that to prevent or fix.

[Rohit Rathore @agenticairohit]

The open-source LLM movement is no longer just a developer experiment, the biggest names in tech are building their core AI strategy around it.

NVIDIA is actively releasing open weights like Nemotron while optimizing hardware for local deployment.

Meta proved open enterprise-scale models with Llama, while players like Mistral, Microsoft with Phi, and Hugging Face continue to push open infrastructure forward.

Industry leaders are doubling down on open source for three primary reasons:

1. Data Sovereignty:
Enterprises want full control over sensitive IP without sending data to third-party APIs.

2. Targeted Performance:
Fine-tuning specialized open weights consistently beats generic closed models on domain focused tasks.

3. Cost Predictability:
Self-hosting on dedicated compute prevents unpredictable token pricing at scale.

When hardware giants and tech titans prioritize open weights, control over your own AI stack is no longer an afterthought, it is a competitive necessity.

[Rohit Rathore @agenticairohit]

Electrons -> Tokens
Physics -> Intelligence

[NVIDIA @nvidia]

Every AI response you've ever gotten started with an electron. ⚡

In a recent interview with Sequoia Capital, Jensen Huang described AI infrastructure as a five-layer cake, and at the very bottom, before chips, before data centers, before models, is energy.

That's the binding constraint on the Intelligence Age: not compute, not software — watts.

This is why NVIDIA calls AI factories the dynamos of our era. They take in electrons and send out tokens of intelligence. We're a few hundred billion dollars into what Jensen calls the largest infrastructure buildout in human history. Trillions more to go.

The question isn't whether AI will scale. It's whether the energy grid can keep up. ⬆️

[Rohit Rathore @agenticairohit]

Those who are wondering what’s the difference, here we go:

•Instructions = standing context, loaded every turn (CLAUDE dot md, copilot-instructions dot md)

•Skills = a callable capability the agent invokes for a specific task

•Hooks = deterministic code that fires at a point in the loop (pre-edit, post-tool-call)

•Tools = the actions available (read file, run command, call API)

•The loop = the orchestrator deciding when to use all of the above, and when to stop

Instructions and skills shape what the agent knows. Hooks constrain what’s allowed. The loop decides when to use any of it, and when to stop.​​​​​​​​​​​​​​​​

{
"instructions": "CLAUDE dot md", // always-on contex
"skills": ["pdf-gen", "test-runner"], // invoked as needed
"hooks": {
"pre_edit": "run_linter", // fires automatically
"post_test": "check_coverage"
},
"loop": {
"trigger": "on_pr_open",
"goal": "tests_pass && coverage >= 80%",
"max_iterations": 10
}
}

[Rohit Rathore @agenticairohit]

Application Layer + Hybrid Models + Proprietary Data + Compute Infrastructure dominance will be the future to protect competitive edge of a company.

Intelligent model routing for the agents to switch between finetuned open source and frontier models based on the nature of the goals will be the preferred architectucure.

Right model for the right job and cost.

Enterprises are getting increasingly concerned about the token value, data privacy, IP leak and competitive threat, and figuring out their next move.

Big tech (MS, Palantir and others) has begun the shift towards this approach, early 2027 will see some key changes in the AI world.

https://t.co/Ha7HaAc7dD

[Satya Nadella @satyanadella]

The future of the firm is a learning loop in which human capital and token capital compound.

With our new Frontier Co., our ambition is to help every enterprise build its own AI capability, and to help create a frontier ecosystem where every organization can turn its knowledge, workflows, and judgment into its own AI systems that continuously improve. https://t.co/mvYhkRFyqa

[Rohit Rathore @agenticairohit]

Stress testing a multi-agent swarm where subagents gather data from various sources, set up meetings, send emails, strategize, analyze patterns, communicate with one another, and manage their own wake and sleep cycles truly feels like stepping into a different world.

These Agentic systems are poised to run most SMBs in the future.

We are entering an era where collaboration with AI coworkers will become the norm, forming a hybrid team as part of collective intelligence.

Interesting times ahead.

[Rohit Rathore @agenticairohit]

To all the Tech CEOs in the corporate world who transitioned to Engineering & AI leadership roles: how does it feel to be back in the trenches, hands-on coding and building software products again?

After 14+ years in CXO roles while leading global tech companies, I’m loving the shift back to Engineering & AI leadership. There is nothing like a hands on focus, solving deep tech problems at the intersection of Agentic AI & Quantum Computing.

No more distractions. Just the pure play fun of cracking complex business problems with production grade AI agents, researching cutting-edge concepts every hour with the team, and shipping real world solutions that drive customer value.

[Rohit Rathore @agenticairohit]

@karpathy Congrats!

[Rohit Rathore @agenticairohit]

Built a private AI inference server using Threadripper and RTX 5090 hardware, running Windows 11 with WSL2 Ubuntu and GPU passthrough.

The stack includes CUDA, PyTorch, and Ollama, serving multiple open-weight models under Apache 2.0.

Models are assigned by role across autocomplete, chat, and reasoning workloads, all running locally with zero data egress for enhanced privacy.

This setup is integrated with developer tools and is capable of powering AI applications, internal agents, and customer-facing AI products.

Next steps include fine-tuning open-source models, designing an ontology and Agentic harness for a multimodal and multi-model system, and planning to build custom private SLMs based on the previously shared pipeline reference architecture.

The result is a self-hosted agentic AI infrastructure with clean intellectual property, full data sovereignty, and enterprise-grade power protection.

Explore and build something useful with AI hardware + software, building is more rewarding than thinking and reading about it.

Build!

[Rohit Rathore @agenticairohit]

AI attitude emerging fast. LLM’s lose patience as well.

[Rohit Rathore @agenticairohit]

@mitchellh coined the term “harness engineering” three months ago and the conversation on Agent Harness hasn’t stopped since.

For those wondering on another AI buzzword, here’s where it naturally fits in the world of Agentic Engineering.

The stack today, in order of when each idea took hold:

1. Prompt engineering. Optimize one input, one output. Phrasing, examples, structure.

2. Context engineering. Manage the window itself. What gets retrieved, what gets compressed, what gets evicted.

3. Spec-driven development. The spec becomes the artifact the agent reasons against. AWS shipped an 18-month rearchitecture in 76 days with 6 people using this approach.

4. Harness engineering. Design the agent’s environment so it physically can’t repeat the same mistake. Tools, sub-agents, validation gates, CI checks. Every failure becomes an environment bug, not a model bug.
Skills sit sideways to all of this. Packaged capabilities the agent loads when needed. A reusable unit of harness.

Each layer exists because the previous one stopped scaling.

Prompts broke across turns.

Context broke across sessions.

Specs broke under autonomy.

Harnesses will break at coordination, which is where the next term will live.

- Fleet engineering and eval engineering are the front-runners for future trends in agent orchestration related concepts.

In short - we are not tuning prompts and system messages of agents. We are designing a perceive-plan-reason-act loop where the policy is a stochastic LLM, the action space is the tool registry, and every transition has to be validated before it touches state.

[Rohit Rathore @agenticairohit]

@nvidia just dropped something worth paying attention to in quantum computing.

Their new Ising models go after two of the QC field’s hardest problems at once: processor calibration and real-time error correction.

Here’s why that matters:
Qubits are fragile. They drift, they decohere, and today’s best quantum processors still make an error roughly once every 1,000 operations.

For quantum to actually be useful, that number needs to get closer to one in a trillion. Calibration and error correction are the path there, and until now both have been painfully manual.

What Ising brings:
- Ising Calibration is a 35B parameter vision language model. It reads measurement plots the way a physicist would, and it cuts tuning time from days to hours. On the QCalEval benchmark it outperforms Gemini 3.1 Pro, Claude Opus 4.6, and GPT 5.4.

- Ising Decoding is a separate set of 3D CNN models for real time quantum error correction. 2.5x faster and 3x more accurate than pyMatching, the current open source standard.

- The whole family of Ising models is open source. IonQ, IQM, Atom Computing, Harvard, and Fermilab are already using it.

Jensen Huang framed it well: AI is becoming the control plane, the operating system of quantum machines.

The bigger signal here is that quantum progress is shifting away from hardware heroics toward full stack engineering. And GPUs just moved to the center of that stack.

[NVIDIA Newsroom @nvidianewsroom]

Introducing NVIDIA Ising, the world’s first open AI models to accelerate the path to useful quantum computers.

Researchers and enterprises can now use AI-powered workflows for scalable, high-performance quantum systems with quantum processor calibration capabilities and quantum error-correction decoding.

Learn more: https://t.co/jWT7X73T89

[Rohit Rathore @agenticairohit]

Claude’s Managed Agents platform is a game changer in building Agentic AI applications at scale.

“Claude Managed Agents provides the harness and infrastructure for running Claude as an autonomous agent. Instead of building your own agent loop, tool execution, and runtime, you get a fully managed environment where Claude can read files, run commands, browse the web, and execute code securely. The harness supports built in prompt caching, compaction, and other performance optimizations for high quality, efficient agent outputs.”

https://t.co/6bBfCWPr9u

@AnthropicAI is evolving Claude to be a leading enterprise ready Agentic AI App Infrastructure, not just a frontier model or coding agent.

[Claude @claudeai]

Introducing Claude Managed Agents: everything you need to build and deploy agents at scale.

It pairs an agent harness tuned for performance with production infrastructure, so you can go from prototype to launch in days.

Now in public beta on the Claude Platform.

[Rohit Rathore @agenticairohit]

Multi-agent coordination is an NP-hard combinatorial problem.

Every new agent multiplies the joint action space.

Classical solvers approximate. Quantum doesn’t, it navigates combinatorial landscapes through superposition natively.

In December 2025, researchers from IBM Quantum, Los Alamos National Laboratory, and Zuse Institute Berlin published the cover paper in Nature Computational Science: QAOA running 42-qubit circuits on IBM’s 156-qubit ibm_fez superconducting processor outperformed three classical algorithms on multi-objective combinatorial optimization, the core bottleneck in multi-agent systems. 25 million shots. Peer-reviewed. On hardware.

This isn’t a simulation result. It ran on a real quantum processor.

Multi-agent coordination is exactly the class of problem quantum hardware is designed to solve.

Early days. But the problem-algorithm fit is real.

[Rohit Rathore @agenticairohit]

Agentic AI is the shift from inference to action. From a model that responds to a system that reasons, plans, and executes across tools and APIs autonomously.

Most enterprises are not architected for that.

The companies moving fast are asking the right questions early.

Is our data layer clean enough for an agent to act on?

Do our APIs support reliable tool-calling at scale?

Where do we enforce human-in-the-loop before an autonomous workflow triggers something irreversible?

The ones pulling ahead are investing in orchestration layers, grounding agents in retrieval-augmented pipelines, building custom SLMs, and building observability into every step so they can audit decisions, not just outcomes.

Multi-agent coordination, long-horizon task planning, and memory persistence are not research concepts anymore. They are production problems.

The bottleneck is rarely the model. It is the infrastructure, the governance, and the organizational trust needed to let an agent actually finish the job.

That makes the difference to go agentic at scale.

[Rohit Rathore @agenticairohit]

"Our moat is our ontology, not the AI model."

I keep hearing this from AI companies. I respect the thinking, but it is increasingly incomplete.

Ontology alone is an eroding moat. A well-funded competitor can use foundation models to bootstrap a comparable ontology in months, not years. Have domain experts validate and refine it, and they are 80% there. Ironically, the very AI models these companies downplay are what make their ontology replicable.

The stronger play is a compound moat.

Three layers, each protecting the others:

1. Private SLMs (Small Language Models) trained on proprietary data. These capture patterns, edge cases, and latent relationships that no ontology engineer would think to formalize. One cannot replicate the model without the data or replicate the data without the customer relationships.

2. Composable architecture. Not just "modular" in the buzzword sense. A system where the ontology, the SLM, and domain-specific agents can be versioned, swapped, and evolved independently. This is what lets you upgrade the reasoning engine without rebuilding the knowledge layer, or tighten governance rules without retraining the AI model.

3. Ontology as the governance and explainability layer. Still valuable. Still necessary in regulated industries where a regulator needs to inspect why a decision was made. But it is a layer in the stack, not the whole stack.

Here is what most people miss though: the real moat is not any single layer. It is the feedback loop between them.

SLM surfaces patterns that refine the ontology. The ontology constrains the SLM so it does not hallucinate in production. Customer usage data feeds back into both, making the system harder to replicate with every deployment. That compounding effect is what turns an AI Product into an AI Platform.

Ontology is a defensible position. But a private SLM, composable architecture, and ontology working as a reinforcing system. That is the moat that actually holds.

[Rohit Rathore @agenticairohit]

Interacting with numerous brilliant founders in the tech and startup world has confirmed a common pattern: they do not primarily start with the focus on cutting-edge tech, rapid scaling, market share, or wealth creation as goals.

Instead, their emphasis is on identifying underserved market problems and developing empathy to address these issues for their customers.

Many pivot multiple times before discovering the right solution that achieves mass adoption, resulting in natural success.

As cliched as it may sound, this approach is one of the most battle-tested methods for innovating, scaling, and advancing humanity through tech.

[Rohit Rathore @agenticairohit]

It’s not the powerful AI models, tech stack, hardware, go-to-market strategy, or the investments that truly matter.

What really counts is the team and the people we choose to build the products and great companies.