crawl4ai

About

Reliable, large-scale web extraction, now built to be drastically more cost-effective than any of the existing solutions.

👉 Apply here for early access We’ll be onboarding in phases and working closely with early users. Limited slots.

Crawl4AI turns the web into clean, LLM ready Markdown for RAG, agents, and data pipelines. Fast, controllable, battle tested by a 50k+ star community.

✨ New in v0.9.2: Maintenance patch release. Fixes a MemoryAdaptiveDispatcher task/page leak when a streaming crawl is closed, Docker Playground "Advanced Config" and Monitor WebSocket auth, Playwright headless-shell packaging, and GPU (ENABLEGPU=true) Docker builds. Release notes →

✨ Recent v0.9.0: Major…

Excerpted from pypi.org/project/crawl4ai/