Post #2312782
2026-05-08 00:08 UTC
@rdnielsen@floss.social Hmm... yeah maybe a SQL or @duckdb@mastodon.social way to ingest all these CSVs into a database despite the differing number of columns.
Replies (2)
-
@rdnielsen@floss.social 2026-05-08 00:20
@LeafyEricScott@hachyderm.io @duckdb@mastodon.social The execsql.py tool has configuration options to import only common columns, force all imports as text, and clean imported data and column headings in other ways. Those can make it easy to import everything into a single table. But even if you have to import the files into more than one table structure, a SQL query or two is then the easiest way to merge those tables into a single format.
-
@dodecadron@datasci.social 2026-05-08 08:40
@LeafyEricScott@hachyderm.io @rdnielsen@floss.social @duckdb@mastodon.social #duckdb seems to work on your example (instead of a list of files as I do here, passing in a glob should also work).