Generate XML sitemaps by crawling a website. Use when a user needs to create a sitemap.xml for SEO, audit site structure, discover all pages on a domain, or...
Crawl any website and produce a standards-compliant XML sitemap ready for search engine submission.
python3 scripts/sitemap_gen.py https://example.com
Output: sitemap.xml in the current directory.
# Basic — crawl and write sitemap.xml
python3 scripts/sitemap_gen.py https://example.com
# Custom output path
python3 scripts/sitemap_gen.py https://example.com -o /tmp/sitemap.xml
# Limit crawl scope
python3 scripts/sitemap_gen.py https://example.com --max-pages 500 --max-depth 3
# Polite crawling with delay
python3 scripts/sitemap_gen.py https://example.com --delay 1.0
# Set SEO hints
python3 scripts/sitemap_gen.py https://example.com --changefreq daily --priority 0.8
# Verbose progress
python3 scripts/sitemap_gen.py https://example.com -v
# Pipe to stdout
python3 scripts/sitemap_gen.py https://example.com -o -
| Flag | Default | Description |
|---|---|---|
--output, -o | sitemap.xml | Output file path (use - for stdout) |
--max-pages | 200 | Maximum pages to crawl |
--max-depth | 5 | Maximum link depth from start URL |
--delay | 0.2 | Seconds between requests |
--timeout | 10 | Request timeout in seconds |
--changefreq | weekly | Sitemap changefreq hint |
--priority | 0.5 | Sitemap priority hint (0.0–1.0) |
--verbose, -v | off | Print crawl progress to stderr |
pip install requests beautifulsoup4
ZIP package — ready to use