Varun Madan

I work for @Onehousehq. Building the next-gen data platform. Opinions are my own.

[Daft @daftengine]

Scaling multimodal workloads ≠ scaling rows of numbers.

🖼️ PDFs, images, video = 💥 OOMs + endless tuning.

That’s why we rebuilt Daft’s distributed engine, Flotilla.

Here’s what broke, what we built, and what we learned 👇

[Varun Madan @madvar]

This integration is extremely valuable; I look forward to it being used by everyone working with multimodal data and models. ❤️

[Quentin Lhoest 🤗 @lhoestq]

Wow the team at @daftengine cooked!

You can now read/write to 🤗Hugging Face with Daft!

> DataFrame engine in 🦀 runs distributed and supports multimodal datasets to train/eval models

Best part: it's optimized for Xet, the dedupe-based HF storage that makes uploads crazy fast!

[Onehouse @Onehousehq]

🚨 Announcing Open Engines™, a quick + reliable way to deploy @trinodb, @raydistributed, and @ApacheFlink making it easy to choose the right engine for analytics, streaming, or ML/DS.

Read the details👉
https://t.co/Gvg7J95NYR

[Shamil @shamiltech]

Am ashamed that my taxes are funding this genocide.

[Kyle Weller @KyleJWeller]

Even Iceberg is faster with Delta Lake... 🤣

#ApacheIceberg #DeltaLake

[Apache Hudi @apachehudi]

📢 NEW RELEASE
Hudi 1.0.0-beta1 is out 📢! This release brings database-like qualities to the data lake!

🔐 Non-blocking concurrency control for high streaming writes
🎾 Filegroup reader & Page Skips
🗂️Functional indexes
🌲 LSM-tree-based timeline

[Onehouse @Onehousehq]

This @AWSOpen blog w/ Onehouse makes it easy to leverage open source on #AWSCloud! Onehouse accelerates build of a Universal Data Lakehouse with MSK, RDS, S3, EMR, Athena, Glue, Redshift. Powered by #apachehudi #apacheiceberg, #deltalake and @OnetableOSS

https://t.co/K5HI71mh4P

[Varun Madan @madvar]

Great blog by @andywalner on how a managed data lakehouse gets you to your goals much faster than a DIY approach.

[Onehouse @Onehousehq]

Are you building a data lakehouse? Check out this guide to help you consider tradeoffs between a DIY approach vs. a managed solution. Learn how you can get your lakehouse off the ground faster 🚀

https://t.co/HpflpCsgKE

#datalakehouse #apachehudi #apacheiceberg #deltalake

[Varun Madan @madvar]

USA! USA! USA!

[Varun Madan @madvar]

Live demo the day after launch is the way to do it! Super excited and proud of the Stream Designer launch! #Current22

[Varun Madan @madvar]

@DataSciBae Confluent!

[Ben Stopford @benstopford]

There's that saying that if you "wrote the book on something" you know your stuff. It's not true. You know your stuff when you wrote the thing that someone else wrote the book on.
Below is how Kafka actually works by the guy who (more than anyone) actually wrote Kafka!

[Varun Madan @madvar]

I am extremely fortunate to be working with such inspirational women across Confluent and working together to #BreakTheBias and built an equitable workplace! Happy International Women's Day!

#BreaktheBias #WIN-erg #IWD #WHM #Confluent

[Varun Madan @madvar]

Wordle 211 3/6

🟨⬛⬛⬛🟨
🟨🟩🟨⬛🟨
🟩🟩🟩🟩🟩

Third row again.

[Varun Madan @madvar]

Wordle 210 3/6

🟨⬛⬛🟩⬛
🟩🟩⬛🟩⬛
🟩🟩🟩🟩🟩

[Varun Madan @madvar]

Wordle 206 3/6

⬛⬛🟨🟨⬛
⬛🟨🟨⬛🟨
🟩🟩🟩🟩🟩

[Thoughts of Dog @dog_feelings]

SOMEONE LEFT A FULL PLATE OF COOKIES. AND A GLASS OF MILK. RIGHT BY THE FIREPLACE FOR ME. I AM NOT KIDDING. WHAT A NIGHT

[Matthias J. Sax 🦦 @MatthiasJSax]

Great to see #StreamingAudio featuring our SIGMOD paper!

[Confluent @confluentinc]

Six years ago we built an entirely new data infrastructure to set #datainmotion. Our journey is built on the passion, energy, and dedication of our team, community, customers, and partners. We’re here because of you, THANK YOU! https://t.co/La89FhXAVg #ConfluentIPO

[Varun Madan @madvar]

Let's set data in motion! https://t.co/mAhWYjBWX8 #CFLT #Confluent #DataInMotion