André Kohn

[Jordan Tigani @jrdntgn]

When we started talking about Hybrid Execution in MotherDuck, people asked "why would you want to run queries in your browser?" This launch shows what kinds of experiences it enables... rich, low latency exploration of data that lives in the cloud (and client!)

[DuckDB @duckdb]

New blog post by @carlo_piovesan: Extensions for DuckDB-Wasm

https://t.co/RuNDqhhYir

Thanks to recent developments, DuckDB-Wasm users can now load DuckDB extensions, allowing them to run extensions in the browser.

[André Kohn @ankohn]

@przemur @duckdb @ApacheArrow Nit „it is executed in a stack-based virtual machine rather than as a native library code“ - it’s not really a VM. Browsers are compiling it down to native code immediately (e.g. Liftoff for Chrome & Cranelift for Firefox)

[André Kohn @ankohn]

@antonycourtney @domoritz I like this one https://t.co/M8UUNXIp3Q by @jeffrey_heer. (Arrow.js uses #2)
Native arrays > pojos for almost everything with data. Turbofan will like the additional type info and tight simple loops. Your CPU will like the cache friendly access and much less GC work.

[André Kohn @ankohn]

@domoritz @antonycourtney That also matches my experience. Better leave the DOM to React + (arrow).js. The shell is somewhat special since we're writing vt100 to xterm.js. But even there, all the utf16 conversions are very painful. It's just that writing a terminal emulator in js hurts more.

[André Kohn @ankohn]

@antonycourtney @domoritz Far too many js data grids are row-major. I used react-virtualized in DashQL but you have to massage a bit to get column-major rendering. @antonycourtney rather use the grid of react-window. Same author, better library design, much easier to glue on top of arrow.js.

[Mim @mim_djo]

this is the nearest thing to magic, Data in Parquet plus a semantic model and Visual all rendered in your browser using WASM , #DuckDB and Malloy and you can run arbitrary Queries too, the model is using TPCH Dataset ( 8 tables)
https://t.co/Ww0ZMUSLAu

[Dmitriy Ryaboy 🇺🇸🇮🇱 @squarecog]

Remember when back near the turn of the century you saw google maps using ajax for the first time, after years of mapquest, and knew everything is going to be different? That's what duckdb-wasm feels like.

[André Kohn @ankohn]

Watch this video if you want to learn more about @duckdb-wasm and our new web shell demo!
https://t.co/aBrItqlw1s

[PVLDB @pvldb]

Vol:15 No:12 → DuckDB-Wasm: Fast Analytical Processing for the Web https://t.co/sT0LpcyvO3

[André Kohn @ankohn]

@kylebarron2 @domoritz @wasmerio @duckdb @ApacheArrow I got the impression, that cross-origin isolation effectively kills any attempts to integrate SharedArrayBuffers (and current wasm multithreading) in general data processing libraries for the browser. But I'd love to be corrected here.

[André Kohn @ankohn]

@kylebarron2 @domoritz @wasmerio @duckdb @ApacheArrow Yes, but `coop: same-origin` and `coep: require-corp` have quite severe effects that are hard to communicate. E.g. including arbitrary third-party data in your tools won't be possible anymore if they don't take care of cors.

[André Kohn @ankohn]

@kylebarron2 @domoritz @wasmerio @duckdb @ApacheArrow We decided against SharedArrayBuffers because you they are locked behind a strict cross-origin policy since Spectre and Meltdown.

[André Kohn @ankohn]

@domoritz @kylebarron2 @wasmerio @duckdb @ApacheArrow Still, this is very relevant for the node client of @duckdb and js-udfs in duckdb-wasm.
We would definitely use a `_import_from_c(buffer, pointer)` in Arrow JS.

[André Kohn @ankohn]

@domoritz @kylebarron2 @wasmerio @duckdb @ApacheArrow Great post @kylebarron2 . In the browser, we're running asynchronously in a web worker where transferrables justify the copy (, I assume). Also, constructing the CDataInterface from JS will be a bit more tedious when replacing the inbound ipc stream (your comment).

[André Kohn @ankohn]

@domoritz @kylebarron2 @IsaacBrodsky @medriscoll @MurrayData @criccomini @matsonj @motherduckdb @RillData @BoilingData @_hex_tech @mim_djo @duckdb @Sina_Kashuk You can build "dynamic" wasm linking today, it's just not pretty. Forward declare an external function in the module A, implement the function in a second module B and either pass a js dummy or the actual function in B when instantiating A.

[André Kohn @ankohn]

@hfmuehleisen @fractorious @duckdb @jlongster We could maybe work around open and close being async but async truncation, flushing, and getSize are seriously limiting the usefulness of the synchronous access handle for us. And the workaround to hide async calls through another web worker locks onto SharedArrayBuffers.

[André Kohn @ankohn]

@hfmuehleisen @fractorious @duckdb Thanks! I've looked at this before but we cannot easily use this today as truncation, getSize, flush and close are still asynchronous in the FileSystemSyncAccessHandle.
Comments by @jlongster in this issue match our situation perfectly.
https://t.co/QsxbnAdpjx

[Alex Monahan @__AlexMonahan__]

Did you know that you can query any parquet or csv file on all of GitHub just by going to the @DuckDB WASM shell website?

https://t.co/Tmd099Y4pI

[André Kohn @ankohn]

@__AlexMonahan__ @davidgasquez @IanCal @a_ghasemi @Wikipedia @duckdb @streamlit We currently depend on synchronous XHRs in our web filesystem. A quick search hinted that they are not supported in chrome extensions? That would be a blocker then until DuckDB gets async I/O.