Post #2392732

2026-05-10 18:20 UTC

I&#39;ve had to temporarily disable access to the S3 backend for my #Peertube instance I use for VoDs again because another bot scraper found it and started downloading everything over and over again. It&#39;s got about 3.1 TB of content on it. It had about 4 TB of transfer on it this morning, and then by the time I turned it off it was up to about 5.99 TB of transfer before I turned it off. Which is wild considering the videos only have a small handful of views on each one of them. If some asshole is going to going to abusively scrape all my content, could they at least download it *once* and then keep it around for their shitty slop-machine training, instead of running up my bandwidth bill continuously? Sheesh.

Replies (7)

@ExpiredPopsicle@vt.social 2026-05-10 18:21

Anyway, all the videos on my #Peertube instance are going to be broken until I get this sorted out.

Open ##2591093
@ExpiredPopsicle@vt.social 2026-05-11 06:15

Update: I've managed to move everything to a new bucket and obscured it behind a proxy I can use to ban and throttle their asses if they ever try it again, which I'm sure they will. VoDs should be viewable again.

Open ##2591094
@coderanger@cloudisland.nz 2026-05-10 18:28

@ExpiredPopsicle@vt.social Are they already getting past Anubis? 🫠

Open ##2591095
@rick@a.n0id.space 2026-05-10 19:04

@ExpiredPopsicle@vt.social oooh this sucks....

Open ##2591099
@sajakunova@mastodon.social 2026-05-10 19:17

@ExpiredPopsicle@vt.social That really sucks. Small independent #PeerTube instances already take enough work and money to maintain — having bots repeatedly scrape terabytes of content is just awful. Sorry you have to deal with that. I don’t really have a smart solution either, except maybe avoiding S3/object storage and keeping things as simple as possible where feasible.

Open ##2591101
@miawgogo@fedi.miawgogo.me 2026-05-10 20:52

@ExpiredPopsicle@vt.social urgh, i remember you talking about this happening a few months ago too :/ why are these scrapers so persistant :<

Open ##2591102
@d1@autistics.life 2026-05-11 16:24

@ExpiredPopsicle@vt.social yeah, that's nasty. Can anubis protect you?

Open ##2591103