Data + AI Summit session: Tyler Croy (delta-rs maintainer) will introduce Virtual Delta Tables and the associated open source code designed for multimodal inference.👇
🗓️ June 15-18
📍 San Francisco
🔗 Session details: https://t.co/WY89dsCvEK
#DeltaLake#MultimodalAI #DataAndAISummit #Lakehouse
#DataAISummit Session Spotlight ➡️ Learn how to build agentic workflows with OSS Spark Declarative Pipelines, with patterns for deterministic, testable, production-ready data workflows.
🗓️ June 15–18
📍 San Francisco
🔗 Session details: https://t.co/fWzAL6uUcQ
#ApacheSpark #DataAISummit
Advancing #ApacheIceberg on Databricks: Iceberg v3 GA, Open Sharing, and Unified Governance
Big drop from the UC team comes in the form of multiple Apache Iceberg capabilities, all landing at once, and the throughline is real: the catalog is the center of gravity for the open lakehouse.
What’s GA:
1.Managed Iceberg (with Predictive Optimization + Liquid Clustering handling the maintenance)
2.Iceberg v3 across managed/foreign/UniForm tables (deletion vectors, row tracking, VARIANT).
3.Foreign Iceberg + credential vending, and external sharing to any Iceberg REST client over Delta Sharing.
What’s New:
1. Foreign Iceberg sharing is in Public Preview.
2. Iceberg-compatible materialized views are in Gated Preview — CREATE MATERIALIZED VIEW ... USING ICEBERG, which is the ergonomics you’d actually want.
3. Cross-engine ABAC is in Beta, enforced server-side through the Iceberg REST scan planning API, so any 1.11+ client gets row filters and column masks essentially for free.
4. Federation now reaches Google Cloud Lakehouse and Palantir too.
https://t.co/bHPSp1JKwm
Headed to Data + AI Summit (June 15–18)? 🗓️ Don't miss Your Guide to Open Table Formats — Delta, Iceberg, Best Practices, and What's Next!
Benjamin Mathew & Scott Sandre will cover Delta 5.0, Iceberg v4, Unified Delta Kernel (GA), and best practices across formats. Add it to your agenda. 👇
https://t.co/toMqbklq4Q
#DeltaLake #ApacheIceberg #DataAISummit
Delta Lake and Apache Iceberg™ have converged on similar ideas—columnar metadata, manifest trees, deletion vectors—but two separate metadata structures still duplicate work.
At DAIS (June 15–18, San Francisco): the next evolution of Delta Lake metadata. 👇
🔹 Delta commits → Iceberg v4’s adaptive metadata tree
🔹 Tree-structured manifests + Iceberg interoperability
🔹 Transactional guarantees Delta users depend on — preserved
🔗 https://t.co/rUwZZtYwl6
#DeltaLake #ApacheIceberg #DataandAISummit
Headed to Data + AI Summit? Don't miss New Foundations of Delta Lake with Kernel + Spark's DSv2.
Delta on Spark DSv1 set the standard for the past 8 years. Rahul Potharaju & Tathagata Das will highlight the move to DSv2 and new foundations for the next decade.
🗓️June 15-18
📍San Francisco
Add it to your agenda 👇
https://t.co/qg9iXIAdPO
#DeltaLake #DataAISummit
If you’re in the Bay Area tomorrow and looking for an excuse to hang out with the @ApacheIceberg community. Come join us at the Bay Area Apache Iceberg meetup (https://t.co/920tD5c1tk)
Engineers from @awscloud, @puppyquery , @databricks, @Snowflake, @Cisco, hyperparam, supermetal, @LakeSailHQ, and @ZetaGlobal will keep you up to date on the latest and greatest.
This is a fantastic team. Check it out if you want to help build agents and models that reliably answer the most challenging analytical questions in biotech, finance, etc for our customers.
The Delta and Unity Catalog extensions in the latest DuckDB release come with a fresh set of features and have shed their experimental labels. In today's blog post, Ben Fleis walks you through the key improvements:
✍️ You can now write Delta tables with DuckDB. Multiple inserts within a transaction produce a single atomic version in the Delta table.
🤝 The Unity Catalog unlocks multi-writer access. DuckDB and other clients such as Spark can now perform writes alongside each other with the catalog handling concurrency control.
⏪ You can use the coolest feature of data lake formats: time travel. This lets you query any Delta table at a specific historical version. Thanks to incremental snapshot loading, this is fast even across large Delta logs.
Read the full blog post for more – link in the thread 🧵
📣 Join us for the next Delta Lake community meetup on May 19!
Agenda:
🔹 Issues & PR backlog
🔹 Unified Kernel
🔹Community highlights
Register: https://t.co/Kwvv3d9lVE
#DeltaLake#OpenSource#Rust#OpenLakehouse
#DataAISummit early bird pricing ends tomorrow, April 30!
Don’t miss your chance to join the leaders shaping the future of data + AI. Lock in 50% savings today: https://t.co/9PhDU1IurZ