With known capture/replay constraints in mind, here are the recommendations for crawling ArcGIS with Archive-it:
- Expand the scope of your seed to include arcgis.com. ArcGIS often hosts data on multiple servers/hosts, thus making it hard to find and implement a simple fix.
- Crawl with Brozzler.
- Run a test crawl - because expanding scope can increase the amount of data captured, we recommend running test crawls before committing data directly to your account.
- Use Wayback QA on the archived pages.