Post #2484935
2026-04-03 06:44 UTC
@picofarad@noauthority.social CCleaner, the old windoze crap cleaner has a dedupe file feature hidden in its menus. That should help you out. I saved MS's old FCIV.exe which can create MD5 & SHA1 checksums of whole dirs to a file or validate those dirs/files against said checksum file. Easy to use for deduping from. In Linux you can run a recursive find running a checksum cmd to a file & sort it to find dupes. Same images in diff formats will need eyeballing though.
Replies (1)
-
@picofarad@noauthority.social 2026-04-03 06:48
@klaatu@noauthority.social whatever dedupe it is needs to be unaware that windows is using it. I don't trust windows links to be copied correctly in all circumstances, so a big checksummed blob containing everything that can be mounted as a file system and not written to is ideal for a yearly backup, if i can get it under 4TB. For reference, helldivers2 was ~130GB but dwarFS shrunk it to ~43GB, and the publisher re-released it as ~24GB. So naive pico using DwarFS does ok...