Rohit Choudhary

Founder CEO, Acceldata. Ex-Hortonworks, Ex-Inmobi. Can build, love building. Building.

[Dev Khare @dkhare]

At @acceldataio’s Autonomous conference today with Rohit and Ashwin here in San Francisco.

T-Mobile, Verizon, Barclays, Merck, Pfizer, Qualcomm in the room.

All of them hitting the same wall: agentic AI doesn’t work in Fortune 500 enterprises if the underlying data — spread across on-prem, cloud, and hybrid — isn’t compute-ready at petabyte scale.

You still need a control/compute plane that runs compute and data workloads across your entire legacy and modern data stacks as well as within any agentic harness like Claude or Codex.

That’s exactly what Acceldata’s AI-native data runtime is.

Proud to be an investor in this category leader in enterprise data & AI platforms.
https://t.co/tal8Ef4sXN

[Dwarkesh Patel @dwarkesh_sp]

Did a very different format with @reinerpope – a blackboard lecture where he walks through how frontier LLMs are trained and served.

It's shocking how much you can deduce about what the labs are doing from a handful of equations, public API prices, and some chalk.

It’s a bit technical, but I encourage you to hang in there - it’s really worth it.

There are less than a handful of people who understand the full stack of AI, from chip design to model architecture, as well as Reiner. It was a real delight to learn from him.

Recommend watching this one on YouTube so you can see the chalkboard.

0:00:00 – How batch size affects token cost and speed
0:31:59 – How MoE models are laid out across GPU racks
0:47:02 – How pipeline parallelism spreads model layers across racks
1:03:27 – Why Ilya said, “As we now know, pipelining is not wise.”
1:18:49 – Because of RL, models may be 100x over-trained beyond Chinchilla-optimal
1:32:52 – Deducing long context memory costs from API pricing
2:03:52 – Convergent evolution between neural nets and cryptography

[Andrej Karpathy @karpathy]

RT to help Simon raise awareness of prompt injection attacks in LLMs.

Feels a bit like the wild west of early computing, with computer viruses (now = malicious prompts hiding in web data/tools), and not well developed defenses (antivirus, or a lot more developed kernel/user space security paradigm where e.g. an agent is given very specific action types instead of the ability to run arbitrary bash scripts).

Conflicted because I want to be an early adopter of LLM agents in my personal computing but the wild west of possibility is holding me back.

[Rohit Choudhary @rconline]

Maybe that’s all that’s really needed!!

[Prof. Lee Cronin @leecronin]

It is trivial to explain why a LLM can never ever be conscious or intelligent. Utterly trivial. It goes like this - LLMs have zero causal power. Zero agency. Zero internal monologue. Zero abstracting ability. Zero understanding of the world. They are tools for conscious beings.

[Aaron Levie @levie]

AI Agents dramatically expand software TAMs because you’ll have endless use cases for AI that you never had previously. There’s no universe where every day you could give out PhD level work to someone and wake up with it all done. AI Agents will do that for anything.

[Aaron Levie @levie]

Every enterprise is going to become AI-first whether they know it or not yet. And the ones that move early will develop a far faster learning curve on how these tools shape work than those that don’t. This will become a strategic advantage for those that get started now.

[Andrej Karpathy @karpathy]

The reality of building web apps in 2025 is that it's a bit like assembling IKEA furniture. There's no "full-stack" product with batteries included, you have to piece together and configure many individual services:

- frontend / backend (e.g. React, Next.js, APIs)
- hosting (cdn, https, domains, autoscaling)
- database
- authentication (custom, social logins)
- blob storage (file uploads, urls, cdn-backed)
- email
- payments
- background jobs
- analytics
- monitoring
- dev tools (CI/CD, staging)
- secrets
- ...

I'm relatively new to modern web dev and find the above a bit overwhelming, e.g. I'm embarrassed to share it took me ~3 hours the other day to create and configure a supabase with a vercel app and resolve a few errors. The second you stray just slightly from the "getting started" tutorial in the docs you're suddenly in the wilderness. It's not even code, it's... configurations, plumbing, orchestration, workflows, best practices. A lot of glory will go to whoever figures out how to make it accessible and "just work" out of the box, for both humans and, increasingly and especially, AIs.

[Aaron Levie @levie]

It never stops being amazing that you can fit the world’s intelligence on a usb thumb drive that costs less than $90

[Andrej Karpathy @karpathy]

Good post! It will take some time to settle on definitions. Personally I use "vibe coding" when I feel like this dog. My iOS app last night being a good example. But I find that in practice I rarely go full out vibe coding, and more often I still look at the code, I add complexity slowly and I try to learn over time how the pieces work, to ask clarifying questions etc.

[Balaji @balajis]

What happens if high quality AI models become free, ubiquitous, and inexpensive to run on even low-spec hardware?

(1) First, you can rebuild every productivity app AI-first. That starts with Microsoft Word, Google Sheets, and Apple Keynote. But it extends to wholly new kinds of productivity apps.

(2) Second, every “smart” device becomes truly smart. Your fridge can double as your nutritionist. Your alarm clock is your sleep therapist. And so on. Just like your car is already your driver.

(3) Third, moats move to the app layer. As others have remarked, the GPT wrappers may end up more defensible than the GPT model itself.

(4) Fourth, physicality becomes relatively more valuable. The hardware, the secure real estate, the in-person community — these are all things digital AI can’t deliver.

(5) Fifth, high human IQ actually becomes increasingly valuable. Because AI is really amplified intelligence rather than truly agentic intelligence, since it requires the creative prompt to get started.

(6) Sixth, prompt engineering is here to stay, because prompting is programming — just in a higher-level language.

(7) Seventh, the most common form of AI doomerism is proven false, because we are getting decentralized ubiquitous AI rather than centralized monotheistic AI. More like a garden of smart things than a vengeful Old Testament God that’ll turn you into paperclips.

(8) Eighth, the combination of cuts to US “industrialized” academic research at the same time AI models accelerate discovery will mean a return to individual gentleman scientists and the advance of desci (decentralized science).

(9) Ninth, the complement to probabilistic AI is deterministic crypto. For captchas, for identity, for money, for all these things — crypto is the digital scarcity that AI can’t fake.

(10) Tenth, the main cost of software development may reduce to reducing the costs of the physical environment. That is: to providing society-as-a-service, to simply giving engineers time to type and experiment in peace. This was already so, but may become even more so.

Several of these points have been made by others, but I think that collectively they help define the second mover era.

[Andrej Karpathy @karpathy]

yep exactly, great work spelling it out step by step.
sometimes I talk about it as "breadth is free, depth is expensive" in the imagined full compute graph of the neural net. afaik this was the major insight / inspiration behind the Transformer in the first place. The first time it properly hit me is when I read the Neural GPU paper a long time ago
https://t.co/TmrwEGdn8r

also btw in "from bits to intelligence" why keep including python? delete python and I think you can make it ~10X less, just along the lines of llmc.

[DHH @dhh]

Our S3 exit is going full steam ahead for a final departure this summer (when our 4-year contract expires!). Look at that beautiful Pure NVMe gear! 😍

[Andrej Karpathy @karpathy]

When working with LLMs I am used to starting "New Conversation" for each request.

But there is also the polar opposite approach of keeping one giant conversation going forever. The standard approach can still choose to use a Memory tool to write things down in between conversations (e.g. ChatGPT does so), so the "One Thread" approach can be seen as the extreme special case of using memory always and for everything.

The other day I've come across someone saying that their conversation with Grok (which was free to them at the time) has now grown way too long for them to switch to ChatGPT. i.e. it functions like a moat hah.

LLMs are rapidly growing in the allowed maximum context length *in principle*, and it's clear that this might allow the LLM to have a lot more context and knowledge of you, but there are some caveats. Few of the major ones as an example:

- Speed. A giant context window will cost more compute and will be slower.
- Ability. Just because you can feed in all those tokens doesn't mean that they can also be manipulated effectively by the LLM's attention and its in-context-learning mechanism for problem solving (the simplest demonstration is the "needle in the haystack" eval).
- Signal to noise. Too many tokens fighting for attention may *decrease* performance due to being too "distracting", diffusing attention too broadly and decreasing a signal to noise ratio in the features.
- Data; i.e. train - test data mismatch. Most of the training data in the finetuning conversation is likely ~short. Indeed, a large fraction of it in academic datasets is often single-turn (one single question -> answer). One giant conversation forces the LLM into a new data distribution it hasn't seen that much of during training. This is in large part because...
- Data labeling. Keep in mind that LLMs still primarily and quite fundamentally rely on human supervision. A human labeler (or an engineer) can understand a short conversation and write optimal responses or rank them, or inspect whether an LLM judge is getting things right. But things grind to a halt with giant conversations. Who is supposed to write or inspect an alleged "optimal response" for a conversation of a few hundred thousand tokens?

Certainly, it's not clear if an LLM should have a "New Conversation" button at all in the long run. It feels a bit like an internal implementation detail that is surfaced to the user for developer convenience and for the time being. And that the right solution is a very well-implemented memory feature, along the lines of active, agentic context management. Something I haven't really seen at all so far.

Anyway curious to poll if people have tried One Thread and what the word is.

[Eade @eade_bengard]

Here's a 16-hour timelapse of an embryo building a central nervous system so 23 years from now it can sell b2b SAAS

[Andrej Karpathy @karpathy]

It's 2025 and most content is still written for humans instead of LLMs. 99.9% of attention is about to be LLM attention, not human attention.

E.g. 99% of libraries still have docs that basically render to some pretty .html static pages assuming a human will click through them. In 2025 the docs should be a single your_project.md text file that is intended to go into the context window of an LLM.

Repeat for everything.

[Ankur Jain @Ankur_Jain_VC]

Awed by Miracle of Mind, a free meditation app by Sadhguru which is setting records. Slickly designed with a very cool AI based Q&A. Built by non-profit volunteers, it hit 1M+ downloads within 15 hours of launch, faster than ChatGPT, TikTok and Instagram! https://t.co/dLp3HHgkiy

[Together AI @togethercompute]

The AI revolution depends on infrastructure. With over $75B invested in AI hardware and software, enterprises are racing to build the infrastructure stack that will power the future.

Next week at @MontySummit, Together AI’s Founding SVP of Product @jamiedg will join industry leaders @PierreFERRAGU (@NewStreetR), Brian S. Raymond (@UnstructuredIO), @kbsdigital (@AMD), and @rconline (@acceldataio) to discuss the state of AI infrastructure, the challenges ahead, and what it takes to build intelligent systems that scale.

[Andrej Karpathy @karpathy]

Agency > Intelligence

I had this intuitively wrong for decades, I think due to a pervasive cultural veneration of intelligence, various entertainment/media, obsession with IQ etc. Agency is significantly more powerful and significantly more scarce. Are you hiring for agency? Are we educating for agency? Are you acting as if you had 10X agency?

Grok explanation is ~close:

“Agency, as a personality trait, refers to an individual's capacity to take initiative, make decisions, and exert control over their actions and environment. It’s about being proactive rather than reactive—someone with high agency doesn’t just let life happen to them; they shape it. Think of it as a blend of self-efficacy, determination, and a sense of ownership over one’s path.

People with strong agency tend to set goals and pursue them with confidence, even in the face of obstacles. They’re the type to say, “I’ll figure it out,” and then actually do it. On the flip side, someone low in agency might feel more like a passenger in their own life, waiting for external forces—like luck, other people, or circumstances—to dictate what happens next.

It’s not quite the same as assertiveness or ambition, though it can overlap. Agency is quieter, more internal—it’s the belief that you *can* act, paired with the will to follow through. Psychologists often tie it to concepts like locus of control: high-agency folks lean toward an internal locus, feeling they steer their fate, while low-agency folks might lean external, seeing life as something that happens *to* them.”

[Narendra Modi @narendramodi]

सभी देशवासियों की ओर से स्वामी रामकृष्ण परमहंस जी को उनकी जयंती पर शत-शत नमन।

[Andrew Ng @AndrewYNg]

A “10x engineer” — a widely accepted concept in tech — purportedly has 10 times the impact of the average engineer. But we don’t seem to talk about 10x marketers, 10x recruiters, or 10x financial analysts. As more jobs become AI enabled, I think this will change, and there will be a lot more “10x professionals.”

There aren’t already more 10x professionals because, in many roles, the gap between the best and the average worker has a ceiling. No matter how athletic a supermarket checkout clerk is, they’re not likely to scan groceries so fast that customers get out of the store 10x faster. Similarly, even the best doctor is unlikely to make patients heal 10x faster than an average one (but to a sick patient, even a small difference is worth a lot). In many jobs, the laws of physics place a limit on what any human or AI can do (unless we completely reimagine that job).

But for many jobs that primarily involve applying knowledge or processing information, AI will be transformative. In a few roles, I’m starting to see tech-savvy individuals coordinate a suite of technology tools to do things differently and start to have, if not yet 10x impact, then easily 2x impact. I expect this gap to grow.

10x engineers don’t write code 10 times faster. Instead, they make technical architecture decisions that result in dramatically better downstream impact, they spot problems and prioritize tasks more effectively, and instead of rewriting 10,000 lines of code (or labeling 10,000 training examples) they might figure out how to write just 100 lines (or collect 100 examples) to get the job done.

I think 10x marketers, recruiters, and analysts will, similarly, do things differently. For example, perhaps traditional marketers repeatedly write social media posts. 10x marketers might use AI to help write, but the transformation will go deeper than that. If they are deeply sophisticated in how to apply AI — ideally able to write code themselves to test ideas, automate tasks, or analyze data — they might end up running a lot more experiments, get better insights about what customers want, and generate much more precise or personalized messages than a traditional marketer, and thereby end up making 10x impact.

Similarly, 10x recruiters won’t just use generative AI to help write emails to candidates or summarize interviews. (This level of use of prompting-based AI will soon become table stakes for many knowledge roles.) They might coordinate a suite of AI tools to efficiently identify and carry out research on a large set of candidates, enabling them to have dramatically greater impact than the average recruiter. And 10x analysts won’t just use generative AI to edit their reports. They might write code to orchestrate a suite of AI agents to do deep research into the products, markets, and companies, and thereby derive far more valuable conclusions than someone who does research the traditional way.

A 2023 Harvard/BCG study estimated that, provided with GPT-4, consultants could complete 12% more tasks, and completed tasks 25% more quickly. This was just the average, using 2023 technology. The maximum advantage to be gained by using AI in a sophisticated way will be much bigger, and will only grow as technology improves.

Here in Silicon Valley, I see more and more AI-native teams reinvent workflows and do things very differently. In software engineering, we've venerated the best engineers because they can have a really massive impact. This has motivated many generations of engineers to keep learning and working hard, because doing those things increases the odds of doing high-impact work. As AI becomes more helpful in many more job roles, I believe we will open up similar paths to a lot more people becoming a “10x professional.”

[Original text: https://t.co/svQYHp3XVW ]