DataCrawl-AI/datacrawl
A simple and easy to use web crawler for Python
6 Releases
Latest: 1y ago
v0.5.0Latest
indrajithi·1y ago·July 11, 2024v0.4.0-betaPre-release
indrajithi·2y ago·June 22, 2024v0.3.0
indrajithi·2y ago·June 13, 2024📋 Changes
- Set concurrent workers, custom delay and optional verbose
v0.2.0
indrajithi·2y ago·June 12, 2024v0.1.4.1
indrajithi·2y ago·June 12, 2024v0.1.3
indrajithi·2y ago·June 12, 2024📋 Changes
- Crawl web pages and extract links
- Handle relative and absolute URLs
- Save crawl results to a JSON file