aiagent.club
中文
GitHub repo

D4Vinci/Scrapling

Python Tracked since 2024-10-13 Updated 2026-09-15 View source ↗

#19 of 803 by Stars among GitHub repos

81.1k Stars

🕷️ An adaptive Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ

About

Selection methods · Fetchers · Spiders · Proxy Rotation · CLI · MCP

Scrapling is an adaptive Web Scraping framework that handles everything from a single request to a full-scale crawl.

Its parser learns from website changes and automatically relocates your elements when pages update. Its fetchers bypass anti-bot systems like Cloudflare Turnstile out of the box. And its spider framework lets you scale up to concurrent, multi-session crawls with pause/resume and automatic proxy rotation - all in a few lines of Python. One library, zero compromises.

Blazing fast crawls with real-time stats and streaming. Built by Web Scrapers for Web Scrapers and regular…

Excerpted from github.com/D4Vinci/Scrapling

Latest metrics

Stars 81.1k 2026-09-15
Forks 8.2k 2026-09-15
Commits 1.6k 2026-09-15
Releases 53 2026-09-15
Watchers 297 2026-09-15
Open issues 2 2026-09-15
Open PRs 4 2026-09-15