Atul Dhingra

Machine Learning Engineer | Technical Leader | Generative AI, Computer Vision, and Edge AI

[Atul Dhingra @dhingratul]

The best reason for making Machines more like people is to make people less like machines. #Humans

[Vishal Patel @vishalm_patel]

🧠📸 Long-term memory is becoming a key capability for personalized AI agents, yet most existing memory systems remain fundamentally text-centric—often reducing images to generic captions.

In our new work, VISUALMEM, we ask: Can AI agents remember what images reveal about us, not just what we explicitly say? We introduce the first benchmark for personal visual memory, capturing both explicit visual evidence (recurring people, pets, and objects) and implicit personal facts inferred from visual and multimodal cues.

We also propose a hybrid visual–text memory architecture that stores structured visual memories rather than collapsing them into captions. VISUALMEM significantly outperforms existing memory systems on visually grounded personalization tasks while remaining competitive on standard text-memory benchmarks. @vng_sofw @thaoshibe @Yuheng120766 @JHUCompSci @HopkinsEngineer @JHUECE

Paper: https://t.co/75msHD9X4E
Project: https://t.co/IvWoYfOTsJ
Code: https://t.co/cQAbN4QfpR

#AI #LLM #MLLM #ComputerVision #MultimodalAI #LongTermMemory #Personalization #AIAgents

[Vishal Patel @vishalm_patel]

We are recruiting multiple Postdoctoral Fellows at the Johns Hopkins University Vision and Image Understanding (VIU) Lab. We are looking for candidates with strong expertise in multimodal AI, large language models (LLMs), diffusion/generative models, image & video synthesis, computer vision, and machine learning.

Postdoctoral researchers will collaborate with a highly interdisciplinary team on cutting-edge problems spanning multimodal reasoning, generative AI, visual understanding, synthetic data generation, and foundation models for real-world applications.

Interested candidates should send their CV, recent publications, and a brief description of their research interests to ([email protected]).

#AI #ComputerVision #MultimodalAI #LLM #DiffusionModels #GenerativeAI #Postdoc #JohnsHopkins

[Jason Corso @_JasonCorso_]

"Disagree and Commit" is the hardest shift for technical founders who are used to being "right." If you're still second-guessing a team decision 2 weeks later, you're the friction.

My take: Conviction > Certainty.

Read the full essay on navigating constructive confrontation: https://t.co/BSMRIlo1Kn

[Andrej Karpathy @karpathy]

This is the the quote I've been citing a lot recently.

[Feross @feross]

🚨 CRITICAL: Active supply chain attack on axios -- one of npm's most depended-on packages.

The latest [email protected] now pulls in [email protected], a package that did not exist before today. This is a live compromise.

This is textbook supply chain installer malware. axios has 100M+ weekly downloads. Every npm install pulling the latest version is potentially compromised right now.

Socket AI analysis confirms this is malware. plain-crypto-js is an obfuscated dropper/loader that:

• Deobfuscates embedded payloads and operational strings at runtime
• Dynamically loads fs, os, and execSync to evade static analysis
• Executes decoded shell commands
• Stages and copies payload files into OS temp and Windows ProgramData directories
• Deletes and renames artifacts post-execution to destroy forensic evidence

If you use axios, pin your version immediately and audit your lockfiles. Do not upgrade.

[Andrej Karpathy @karpathy]

Software horror: litellm PyPI supply chain attack.

Simple `pip install litellm` was enough to exfiltrate SSH keys, AWS/GCP/Azure creds, Kubernetes configs, git credentials, env vars (all your API keys), shell history, crypto wallets, SSL private keys, CI/CD secrets, database passwords.

LiteLLM itself has 97 million downloads per month which is already terrible, but much worse, the contagion spreads to any project that depends on litellm. For example, if you did `pip install dspy` (which depended on litellm>=1.64.0), you'd also be pwnd. Same for any other large project that depended on litellm.

Afaict the poisoned version was up for only less than ~1 hour. The attack had a bug which led to its discovery - Callum McMahon was using an MCP plugin inside Cursor that pulled in litellm as a transitive dependency. When litellm 1.82.8 installed, their machine ran out of RAM and crashed. So if the attacker didn't vibe code this attack it could have been undetected for many days or weeks.

Supply chain attacks like this are basically the scariest thing imaginable in modern software. Every time you install any depedency you could be pulling in a poisoned package anywhere deep inside its entire depedency tree. This is especially risky with large projects that might have lots and lots of dependencies. The credentials that do get stolen in each attack can then be used to take over more accounts and compromise more packages.

Classical software engineering would have you believe that dependencies are good (we're building pyramids from bricks), but imo this has to be re-evaluated, and it's why I've been so growingly averse to them, preferring to use LLMs to "yoink" functionality when it's simple enough and possible.

[Devi Parikh @deviparikh]

Hire well, share context well, and then default to trust, autonomy, delegation. Step in to manage only if evidence suggests it is needed. And then go as deep as needed.

The other way around — defaulting to micro managing and then backing off over time — risks capping someone's potential to contribute for far too long. It eliminates opportunities for pleasant surprises. And doesn't scale.

[Mitchell Troyanovsky @mitch_troy]

Making devs finally write docs by calling them “skills” is a great arb

[Andrej Karpathy @karpathy]

nanoGPT - the first LLM to train and inference in space 🥹. It begins.

[Andrej Karpathy @karpathy]

Happy weekend to those who celebrate

[vLLM @vllm_project]

🚀 No More Train–Inference Mismatch!
We demonstrate bitwise consistent on-policy RL with TorchTitan (training) + vLLM (inference) — the first open-source run where training and inference numerics match exactly.

It only takes 3 steps:
1️⃣ Make vLLM batch-invariant (same seq → same output regardless of batching)
2️⃣ Ensure forward passes in training use identical kernels as inference
3️⃣ Add custom backward passes in PyTorch

✅ Verified on Qwen3 1.7B + GSM8K:
• batch_inv_ON (bitwise exact) → KL=0.0, faster convergence, higher reward
• batch_inv_OFF → reduced reward, instability

We audited every op, imported vLLM’s fused kernels (SiLU MLPs, RMSNorm+residual), and wrote matching backward passes. Run is fully on-policy, deterministic, and reproducible.

Next:
• Unified model code
• torch.compile support
• Perf tuning (current bitwise RL ≈2.4× slower)
• Broader model + op coverage

🔗 https://t.co/e0YChOkYm6
#vLLM #TorchTitan #RL #LLM #AIResearch

[François Chollet @fchollet]

ML research is an engineering discipline, not a philosophy seminar. You build, you test, you learn. Untested ideas are just speculation.

[Zichen Liu @zzlccc]

BF16 -> FP16 is such a simple (one configuration change in Oat) yet fundamental fix for inference-training mismatch.

With FP16, the most basic importance sampling PG outperforms all algorithmic fixes in BF16. Let's rethink RL stability from the precision perspective.🔎

[François Chollet @fchollet]

To really understand a concept, you have to "invent" it yourself in some capacity. Understanding doesn't come from passive content consumption. It is always self-built. It is an active, high-agency, self-directed process of creating and debugging your own mental models.

[CJ Zafir @cjzafir]

vibe coding:

[Charlotte Chess Center @CLTchesscenter]

The Naroditsky family shares the sad news of Daniel’s unexpected passing. Daniel was a talented chess player, educator, and beloved member of the chess community. We ask for privacy as the family grieves.

[Rohan Paul @rohanpaul_ai]

Columbia CS Prof explains why LLMs can’t generate new scientific ideas.

Bcz LLMs learn a structured “map”, Bayesian manifold, of known data and work well within it, but fail outside it.
But true discovery means creating new maps, which LLMs cannot do.

[Rohan Paul @rohanpaul_ai]

Fei-Fei Li ( @drfeifei ) beautifully explains Robotics.

She defines robotics not by form, like humanoids or cars, but by function: they are any "embodied machines" that must perceive, understand, and act within a physical, 3D space.

This core requirement is "spatial intelligence," the unifying principle of all robotics, allowing them to perform tasks and even collaborate with humans.

Throughout all of human history, we have been confined to a single, shared reality: the "physical Earth 3D world."

This singularity has been our only playground.

However, new technologies that combine 3D generation and reconstruction are shattering this limitation.

We can now create "infinite universes"—a multiverse of digital worlds for countless purposes, from training robots to enabling creativity, travel, and storytelling.

This leap from one physical world to an infinite multiverse unlocks boundless possibilities for human imagination and interaction.

Video from @a16z

[Anish Giri @anishgiri]

Two of the best blitz players in the world are playing a match.

You are allowed to retweet. Don't be scared.

[Vishal Patel @vishalm_patel]

Honored to be speaking alongside other respected experts at the Biometrics Institute US Biometrics Seminar. We’ll be diving into US biometrics developments and the crucial topic of AI’s impact on vulnerabilities. @HopkinsDSAI @JHUECE @JHUCompSci @HopkinsEngineer @BiometricsInst