Scoping & Running Crawls
Scoping Your Crawls
- How Archive-It crawlers determine scope
- Scope Rules and how to use them
- Modify your collection or seed scope
- How to add Seed level scope rules to multiple seeds at once
- How to limit your crawl
- Expand the scope of your crawl
Running Crawls
Crawling Technology
Managing Crawls
Scoping Recommendations for Specific Sites
- Scoping guidance for specific types of sites
- Archiving sites protected by Cloudflare
- Archiving ArcGIS
- Archiving Blogspot sites
- Archiving Facebook
- Archiving Flickr streams