You can launch test or one-time crawls of new seeds directly from the home page of the web application by using the “InstaCrawl” feature. InstaCrawl lets you immediately add and crawl new seeds in new or existing collections without going through existing collection creation and management workflows. As soon as the crawl begins, these new seeds or collections will appear in the web application like any others for later management.
On this page:
- Step 1: Select the InstaCrawl button
- Step 2: Select a collection and add seed(s)
- Step 3: Set crawl limits
- Step 4: Run your crawl
- Related Content
Step 1: Select the InstaCrawl button
To use the InstaCrawl feature, begin by clicking the button on the home screen of your account:
Step 2: Select a collection and add seed(s)
In the pop-up dialog, select an existing collection or name a new one to which to add your seeds. Enter the seed URLs in the box as you would add them to any normal collection:
By default, new seeds will otherwise be publicly visible, set to the One-Time frequency, and of the “standard” type. Seeds added to a frequency already recurring in an existing collection will crawl automatically with others at that frequency. New recurring crawl frequencies need to be scheduled in addition to this one-time crawl.
Step 3: Set crawl limits
When you have added your seeds to the list, click the “Set Limits” button to advance to crawl configurations. Like any manually launched crawl, select its test or one-time production type and apply any desired document, data, or time limits.
Step 4: Run your crawl
Clicking the Crawl button will start your crawl.
Once initiated, your crawl will be added to the “Current Crawls” lists in your account and your new or existing collection. You may monitor and review this crawl’s reports as you would any other Archive-It crawl.