Post #3623351
2026-05-12 23:58 UTC
@noamross@ecoevo.social @hrbrmstr@mastodon.social @jeroenooms@fosstodon.org @gws@fosstodon.org @kirill@fosstodon.org duckdb against s3 buckets is phenomenal. It’s the only tool/library I use for serious data retrieval of column oriented data structures. You want to prep your parquet files thoughtfully. decent row group size and sort order but it does the rest.
Replies (1)
-
@hrbrmstr@mastodon.social 2026-05-13 07:19
@dplattsf@sfba.social @noamross@ecoevo.social def second this. and with the new "quack" from motherduck, you can have local or WASM duckdb client talk to a duckdb "server" that's running on an AWS node (or any other S3-compatible object store cloud node) super close to the object store network for even more speed gains (at the expense of running yet-another node in the $ cloud).