Prashanth Rao

AI engineer @lancedb | prev @kuzudb. Blogging @

[Prashanth Rao @tech_optimist]

@1eo @ragnorco Ahh nvm, I just remembered that this is at the LanceDB layer, and you're operating at the Lance layer. Oh well!

[Prashanth Rao @tech_optimist]

👀 My eyes spy a killer 🔥 new feature coming out soon in @lancedb: "first-class support for table branching across the Rust, Python and TypeScript SDKs".

The team is COOKING!!

🥳 🚀

https://t.co/myj62XERbs

[Prashanth Rao @tech_optimist]

@1eo and @ragnorco, you folks would be the perfect alpha-testers! Hopefully this proves useful!

[mandu (🥟) @manduw0rld]

unsurprisingly, the best food i ate in London was Chinese

[Prashanth Rao @tech_optimist]

At @lancedb, these kinds of problems around ease of experimentation, exploration, reusability and reproducibility, along with the immense scale of data that AI researchers are handling on a daily basis, are fundamental to how we think about where AI data infra is headed.

Read the blog for more details, and watch out for more interesting posts, including benchmarks and training experiments, coming soon! 👇🏽

6/6

https://t.co/tqV57Bc9ty

[Prashanth Rao @tech_optimist]

Dataset curation looks like a search problem at first, but for training pipelines the hard part is everything around it: filtering, deduping, sampling, inspection, materializing the subset, and being able to reproduce the decision later.

I wrote about how we’re thinking about this at @lancedb for the next generation of AI data infrastructure:

https://t.co/tqV57Bc9ty

1/6

[Prashanth Rao @tech_optimist]

From a user perspective, the end state is not simply "I found a useful subset."

It’s a durable training + search artifact: a Lance table, accessible in @lancedb with the selected rows, raw data, embeddings, metadata, indexes, provenance, and version history.

Downstream feature engineering, search, analytics and training workflows can pin to the same curated dataset instead of reconstructing it from an ad hoc notebook state that knows nothing about data provenance.

5/6

[Prashanth Rao @tech_optimist]

@chandnirao_here Time to move to Toronto 😄🍁

[Prashanth Rao @tech_optimist]

@FAHADSH97656634 Thank you! Glad to hear that resonated. Hopefully more on how DataFusion powers other parts of Lance, too!

[Prashanth Rao @tech_optimist]

Lots of new followers, so allow me to say hi! I'm an AI engineer, with a passion for technical communication and writing good content that helps people get productive with amazing open source tools and frameworks. 👋

I've been writing a lot with AI lately, and it's so much better and more fun than before. Look for more stuff from me over the coming weeks!

[Prashanth Rao @tech_optimist]

So cool! @nvidia's Cosmos 3 team (building next gen Physical AI infrastructure) is building on top of Lance!

"Unified data layer: SILA organizes data curation as a unified columnar Lance dataset (Pace et al., 2025), where each row represents a data sample and each typed column represents a curation signal such as a caption, tag, quality score, or annotation."

Link to the full report: https://t.co/CpcQZZXgHf

[LanceDB @lancedb]

Cosmos 3 by @nvidia released today — a frontier omnimodal world model for Physical AI.

For the data infrastructure behind it, they built on Lance.

SILA, NVIDIA's internal curation platform, processes tens of billions of multimodal training candidates as a single Lance dataset. Curation signals, embeddings, and vector indexes all in one table. No separate vector DB.

One table from raw data to training-ready.

[Prashanth Rao @tech_optimist]

@raveeshbhalla @LinghuaJ @georgehe0 @cocoindex_io No, just got back to Toronto. But I'll make it a point to let folks know when I'm next in SF, life always brings me back there eventually 😇

[Prashanth Rao @tech_optimist]

It's been amazing to see the vision that @LinghuaJ and @georgehe0 have coming to fruition. They're building out in the open, engaging with users, and are approaching building a thriving OSS community in all the right ways.

I can't wait to see where @cocoindex_io goes in the coming months! Will definitely be writing more about it, but if you haven't already, do check it out!

[Linghua Jin 🥥 🌴 @LinghuaJ]

Catching up with the amazing @tech_optimist this week on ai & data infra and rust 🦀 for ai. Excited for what's coming on @cocoindex_io and @lancedb and many amazing projects in the space.

when friends in town, we crack coconut together !

[Prashanth Rao @tech_optimist]

@sdhilip @cocoindex_io @LinghuaJ This is really cool! Glad it's showing such value, it really solving SUCH a relevant problem for AI/data engineers. I was hooked the moment I saw it myself.

[CocoIndex @cocoindex_io]

Congrats @sdhilip on shipping his 10th projects in production and thanks for sharing what you build with @cocoindex_io continuously with the community!

CocoIndex helps agents with fresh data views 𝐟𝐫𝐨𝐦 𝐲𝐨𝐮𝐫 𝐬𝐨𝐮𝐫𝐜𝐞. It is an 𝐢𝐧𝐜𝐫𝐞𝐦𝐞𝐧𝐭𝐚𝐥 𝐞𝐧𝐠𝐢𝐧𝐞 that continuously takes codebase, meeting notes and any dynamic contents and transform them into insights for ai.

We are grateful to have such a great community, many of our users have been with us for long time since our early launches and continuously giving us feedbacks . Thank you all and let's go!

[Olsen @olsenbdnr]

Canada’s greatest export is their people, esp cracked engineers and founders