rtdl ⚡️

The Real-Time Data Lake ⚡️ rtdl is the easiest way to build a real-time data lake on AWS, GCP, & Azure. #OpenSource

[rtdl ⚡️ @rtdlio]

Been a bit busy recently, but we're back.
We just released rtdl v0.2.0.

What we shipped:
* Added @DeltaLakeOSS support.
* Switched from Postgres to file-based configuration storage.
https://t.co/VwCd4hTBRu

[rtdl ⚡️ @rtdlio]

New blog post! We released rtdl v0.1.2 yesterday, adding support for @hadoop HDFS, @awscloud Glue, and @SnowflakeDB External Tables. Also includes a quick bonus tutorial on how to set up rtdl with Snowflake External Tables.

https://t.co/tZfkhR6QpY

[rtdl ⚡️ @rtdlio]

We just released rtdl v0.1.2.

What we shipped:
* Added HDFS support.
* Added @awscloud Glue support.
* Added @SnowflakeDB External Tables support.
https://t.co/1b3XkBkQeP

[Opemipo Disu 🦅 @coderoflagos]

👋🏽Recently, @gavinjtech called me to hop on this project, and I’m super happy that I’m a part of it!

Just if you don’t know, rtdl is the easiest way to store and build and maintain a real data time lake on AWS, Kafka or Segment! You should check - https://t.co/hZmtAOWiWe 🚀🎉

[rtdl ⚡️ @rtdlio]

It should be as easy to load data into any data lake as it is a warehouse.

[rtdl ⚡️ @rtdlio]

This is a good discussion about Data Lakehouses on Reddit.

https://t.co/FP28E83GZq

[rtdl ⚡️ @rtdlio]

First batch of stickers just came in

[rtdl ⚡️ @rtdlio]

We merged our first major PR from the community today 🎉

@misterbykl built a @statefun_io function to detect and mask PII. Thank you for your contribution to rtdl #OpenSource

https://t.co/qSDs0RWvQB

[rtdl ⚡️ @rtdlio]

New docs out tonight https://t.co/6Ne4PkI9hP

[rtdl ⚡️ @rtdlio]

@Decodableco @esammer @netflixdata This is the exact reason we built https://t.co/EyryO5WO2I. It shouldn't take a full data team to have real-time data.

[Pekka Enberg @penberg]

Latency is increasingly important across variety of use cases. For example, in e-commerce, going beyond single digit millisecond latencies is believed to hurt revenue. But how is this need for speed driving infrastructure changes today?

Here's one answer. 🧵 1/

[Opemipo Disu 🦅 @coderoflagos]

Happy to be work on @rtdlio with @gavinjtech and @Dipanja22544292. Here’s what the project is all about :)

rtdl makes it easy to build and maintain a real-time data lake. You send rtdl a real-time data stream – often from a tool like Kafka or Segment.

https://t.co/hZmtAOWiWe

[rtdl ⚡️ @rtdlio]

New blog post! Learn how to build a real-time data lake on @awscloud with rtdl and @segment.
Guide covers:
* Creating and configuring EC2
* Deploying rtdl
* Configuring S3
* Creating a Webhooks destination in Segment
* Creating a stream in rtdl

https://t.co/ADBYeB41GV

[rtdl ⚡️ @rtdlio]

We just released rtdl v0.1.1.

What we shipped:
* Replaced Kafka & Zookeeper with @redpandadata
* Added support for HDFS
* Fixed a few issues
* Added several logo and banner graphics

https://t.co/5QsSrwcQpM

[rtdl ⚡️ @rtdlio]

We agree. Solving one difficult problem really well is the way.

[Aaron Phethean @aphethean]

I think we all know the feeling. The tool analogy is very good for software, the mistake we seem to repeat is trying to invent and reinvent a multi tool.

[Dremio @dremio]

We’re thrilled to announce the general availability of our fully-managed lakehouse platform, Dremio Cloud, as well as the public preview of Dremio Arctic, an intelligent metastore for Apache Iceberg that automates data management!
https://t.co/t7f4XeKqjT

[Robert Yi 🐳 @imrobertyi]

Decision science is going to go mainstream.

The 2010s belonged to data science. Then we saw the rise of analytics eng. But aeng -> less time on prep + better data -> more questions -> decision science. Will purple people cleave into magenta and fuschia people?

[Gavin Johnson 👉 @thtmnisamnstr.bsky.social @gavinjtech]

@kunalstwt We're building @rtdlio. It’s a real time data lake built with @statefun_io, @redpandadata, and @dremio written in @golang. It’s #opensource on GitHub. https://t.co/FdW3fpG76T

[rtdl ⚡️ @rtdlio]

That's a nice streaming service you've built @redpandadata. We're working on replacing Kafka and Zookeeper with it in our stack. It's been really easy so far.