Apache Hudi

Official twitter handle of Apache Hudi, an open data lakehouse platform.

[Apache Hudi @apachehudi]

Hudi 1.0 is the most powerful release to date for data lakehouses. Read the blog for details:

Secondary Indexing, Expression Indexes, Partial Updates, Non-blocking Concurrency Control, New LSM timeline, +more: https://t.co/QHi7rnNsn9

#datalakehouse #opentableformat

[Apache Hudi @apachehudi]

Full case study on AWS Big Data Blog: https://t.co/GdkDj7joPB

[Apache Hudi @apachehudi]

GE Aviation: 30+ source systems in production, several hundred Apache Hudi tables, 14+ months in production, 10,000+ tables in the dev pipeline.

Modernizing 150+ source systems across the aviation ecosystem ↓

[Apache Hudi @apachehudi]

Aviation is deeply regulated. Hudi ran in production here at enterprise scale.

Links ↓

#ApacheHudi #AWS

[Apache Hudi @apachehudi]

Compaction docs: https://t.co/lwftszd3Lt

Async services overview: https://t.co/THG2II0EFx

[Apache Hudi @apachehudi]

Compaction is the price you pay for write speed on MoR tables. Hudi is the only open table format that does it without blocking writes.

[Apache Hudi @apachehudi]

Tunable: inline / after N commits / time-based / async. Selection by log size, file-group activity, or partition.

A fully compacted MoR table reads identically to CoW. The question is just when to reconcile.

Links ↓

#ApacheHudi #MergeOnRead

[Apache Hudi @apachehudi]

AI overview docs: https://t.co/6KYOp4evlF

Vector search: https://t.co/fwH3VvuoTr

BLOB type: https://t.co/g67rl6saZw

Lance file format: https://t.co/AfTT58MEhN

[Apache Hudi @apachehudi]

AI workloads on the lakehouse have a different shape than analytics.

Embeddings (768–3072 floats/row) replace counts and sums. Raw assets become first-class. Queries shift to "nearest neighbors filtered by tenant." Feature tables get thousands of columns wide.

[Apache Hudi @apachehudi]

AI-native isn't a separate stack. It's the same lakehouse with the primitives AI workloads actually need.

No need to copy data into another specialized system.

Links ↓

#ApacheHudi #AINativeLakehouse

[Apache Hudi @apachehudi]

Full AWS reference architecture: https://t.co/xHznv3wrzy

[Apache Hudi @apachehudi]

Configuration Manager — table settings in DynamoDB; one Glue job processes multiple tables (tested with 18).

[Apache Hudi @apachehudi]

DynamoDB global tables for cross-region failover. Audit tracking for the regulatory crowd.

Links ↓

#ApacheHudi #AWS

[Apache Hudi @apachehudi]

File Manager — detects CDC files via EventBridge, queues through SQS, tracks metadata in DynamoDB.

File Processor — Step Functions orchestrating parallel Glue jobs that produce Hudi datasets.

[Apache Hudi @apachehudi]

AWS pattern for ingesting hundreds of operational databases into one Hudi lakehouse — without N pipelines for N tables.

A three-component framework ↓