Post #2192381
2026-05-06 08:33 UTC
@lenotaire@mastodon.social @aeva@mastodon.gamedev.place jokes aside I know of a company who moved most of their data from mongodb into gzipped CSV files in folders. Basically using folders/symlinks as tables/indices, and the file system cache as memory store.
The cluster size dropped from 60 TiB to 3, and performance went up significantly, lol.
Replies (1)
-
@lenotaire@mastodon.social 2026-05-06 09:03
@rhoot@mastodon.gamedev.place @aeva@mastodon.gamedev.place actually this is not a bad idea at all if you look up for the data using only the primary key, but if you need to query by other fields then it effectively depends on the filesystem cache. However, I've also seen tables with too many indices seldom used that have a notorious negative impact on both performance and disk usage (working with PostgreSQL)