Crawl4AI
github.com/unclecode/crawl4ai- Category
- AI Tools
- Type
- APP
- Builder
- unclecode
- GitHub
- 75.7k stars
- Added
- Jun 16, 2026
About
An open-source, LLM-friendly web crawler and scraper that turns websites into clean, structured data for AI pipelines.
Why it made the leaderboard
An open-source crawler that turns websites into clean, LLM-ready Markdown and structured data, with anti-bot handling and crash recovery for real-world crawls. The self-hosted answer when you'd rather not meter your data pipeline through a scraping API.
Tech Stack
PythonDocker
Comments (0)
No comments yet
Indexed by a proprietary survey. Corrections welcome.