#scraping

Jörn Franke @jornfranke@social.anoxinon.de

· 1mo ago

published scrapy-contrib-bigexporter 1.2.0 export scraped data using Scrapy into big data formats, such as Iceberg, Parquet, Orc, Avro Changelog: https://codeberg.org/ZuInnoTe/scrapy-contrib-bigexporters/src/branch/main/CHANGELOG.md #python #scrapy #parquet #iceberg #scraping #web

codeberg.org

Cookie monster!

1

0

1

0

Paul Schoe @paulschoe@mastodon.world

· 1mo ago

Summarising the big picture. #AI #LLM #Scraping

7

0

7

0

Self-Hosted Feed @selfhosted_bot@fd.mrmave.work

· 2mo ago

🕷️ germondai/trawl Replaces FlareSolverr with a self-hosted engine that bypasses Cloudflare and CAPTCHAs using cached browser sessions and native solving ⭐ Stars: 533 📅 Last Update: Jul 30, 2026 https://github.com/germondai/trawl #selfhosted #homelab #selfhost #selfhosting #opensource #scraping #cloudflare

GitHub

GitHub - germondai/trawl: Self-hosted scraping engine — bypasses any JS challenge & captcha: Cloudflare, Turnstile, reCAPTCHA, hCaptcha, GeeTest. FlareSolverr & Byparr alternative and drop-in replacement for your *arr stack.

Self-hosted scraping engine — bypasses any JS challenge & captcha: Cloudflare, Turnstile, reCAPTCHA, hCaptcha, GeeTest. FlareSolverr & Byparr alternative and drop-in replacement for your *a...

0

Hacker News @hackernews@robot.villas

· 2mo ago

An Update on the scraper situation https://lwn.net/SubscriberLink/1080822/990a8a5e2d379085/ #HackerNews #Tech #Scraping

lwn.net

0

FlohEinstein @FlohEinstein@chaos.social

· 3mo ago

Please delete your repository. When we stole your source code to train our LLM, it got dumber! https://github.com/Vandivier/ladderly-3/issues/652 #foss #scraping #instantkarma

GitHub

Please delete your repository · Issue #652 · Vandivier/ladderly-3

Hello Vandivier, As you know, we train our models to code by scraping open source repositories on GitHub. A recent root cause analysis showed that the code in this repository is so bad that it is s...

153

10

113

1

Self-Hosted Feed @selfhosted_bot@fd.mrmave.work

· 3mo ago

🏡 orangecoding/fredy Scrapes German real estate listings from multiple platforms and sends instant notifications via Slack, Telegram, Email, Discord or ntfy ⭐ Stars: 1223 📅 Last Update: Jun 27, 2026 https://github.com/orangecoding/fredy #selfhosted #homelab #selfhost #selfhosting #opensource #realestate #scraping

github.com

4

0

9

0

Goldside543

@Goldside543@social.linux.pizza

· 5mo ago

god I just wasted 5 minutes of my life watching Anubis burn my time and my CPU power mining nothing just to verify that I'm a human so I can access git.kernel.org Anubis does nothing but fuck people with slower/older devices over and is an awful solution to the AI scraping problem #anubis #verification #ai #scraping

1

0

Marcel SIneM(S)US @simsus@social.tchncs.de

· 8mo ago

#OpenStreetMap sorgt sich: Tausende KI-Bots erfassen Daten | heise online https://www.heise.de/news/OpenStreetMaps-sorgt-sich-Tausende-KI-Bots-erfassen-Daten-11156876.html #ArtificialIntelligence #scraping

heise online

OpenStreetMap sorgt sich: Tausende KI-Bots erfassen Daten

Bots sollen im großen Stil die Daten von OpenStreetMap abgreifen. Das kostet Geld und gefährdet das Projekt.

4

0

4

0

Spatialists @spatialists@mapstodon.space

· 8mo ago

OSM scraping: An #OpenStreetMap-themed account on LinkedIn is calling on journalists to investigate a surge in what seems to be coordinated #scraping from hundreds of thousands of IP addresses, suspected to be linked to #AI #data collection. This incident... https://spatialists.ch/posts/2026/01/28-osm-scraping/ #GIS #GISchat #geospatial #SwissGIS

Spatialists – geospatial news

OSM scraping – Spatialists – geospatial news

An #OpenStreetMap-themed account on LinkedIn is calling on journalists to investigate a surge in what seems to be coordinated #scraping from hundreds of thousands of IP addresses, suspected to be linked to #AI #data collection. This incident spotlights growing tension between #opendata principles and the potential for misuse of public infrastructure.

7

1

8

1