Just merged into Lance: a completely new FTS index layout
๐ 9.35x faster index builds (34s โ 3.6s)
๐ 8.87x smaller index size (955MB โ 107MB)
๐ 3.41x faster phrase queries (21ms โ 6ms)
Block bitpacking + delta encoding + new phrase query algo.
Opt in today with LANCE_FTS_FORMAT_VERSION=2
https://t.co/CQ8F4aacph
We can just build.
I build this as a toy project but now itโs available to all duckdb users over the world. So amazing!
Let me what we should build next ๐
๐ Today, we are releasing DuckDB version 1.5.1, a patch release with bug fixes, performance improvements and support for the Lance lakehouse format.
โก๏ธ https://t.co/tkZWYxs4EN
Did you spot Lance & LanceDB in Jensen Huang's keynote at @nvidia GTC today? ๐
Jensen highlighted one of the biggest shifts in AI infrastructure: unlocking unstructured data.
1/5 Large multimodal blobs donโt have to break dataset workflows.
Images and videos are often treated as external files, separate from metadata and indexes. Once datasets get large, that split makes exploration, curation, and training painful.
Lance changes that on the ๐ค @huggingface Hub. ๐งต๐
Capping off a cold, wintry, locked-in day โ๏ธ with this AWESOME PR on a graph benchmark, it totally made my day.
The #Lance format is something to hold close to the heart ๐ค, and there's LOTS more to do on the lance-graph front, stay tuned!
See: https://t.co/T3qk2VuuNA
What does it take to store the web as a database?
exa-d is our internal data framework that orchestrates declarative typed dependencies, sparse updates with precise granularity, efficient and parallel execution across scaling compute, and more.
https://t.co/w2QhhgEhrj
1/5 @lancedb ๐ซถ๐ป @duckdb
Weโre happy to announce a new Lance extension for DuckDB! You can simply install this extension in DuckDB and point at your Lance datasets from within a DuckDB CLI or a Python script, while getting ๐ณ๐๐น๐น ๐ฆ๐ค๐ ๐ฐ๐ฎ๐ฝ๐ฎ๐ฏ๐ถ๐น๐ถ๐๐ถ๐ฒ๐ ๐ผ๐ป ๐๐ผ๐ฝ ๐ผ๐ณ ๐๐ฎ๐ป๐ฐ๐ฒ without copying your data!
1/5 In this blog post, we discuss what we mean by โmultimodalโ and the different lenses through which it can be viewed. Rather than thinking of โmultimodalโ as just a content label (text, images or video/audio), it makes sense to think of it at a *system* level.
lance-duckdb v0.2.0 is coming!
This version will have:
- vector/fts/hybird search in @lancedb
- native support of @duckdb 's secret system
- filter & project pushdown to @lancedb
- native support of @huggingface fs
- explain diagnostics support