How to crawl new seeds immediately with InstaCrawl

Overview

You can launch test or one-time crawls of new seeds directly from the home page of the web application by using the “InstaCrawl” feature. InstaCrawl lets you immediately add and crawl new seeds in new or existing collections without going through existing collection creation and management workflows. As soon as the crawl begins, these new seeds or collections will appear in the web application like any others for later management.

On this page:

• Instructions
• Step 1: Select the InstaCrawl button
• Step 2: Select a collection and add seed(s)
• Step 3: Set crawl limits
• Step 4: Run your crawl 
• Outcome
• Related Content

Instructions

Step 1: Select the InstaCrawl button

To use the InstaCrawl feature, begin by clicking the button on the home screen of your account:

Step 2: Select a collection and add seed(s)

In the pop-up dialog, select an existing collection or name a new one to which to add your seeds. Enter the seed URLs in the box as you would add them to any normal collection:

When you have added your seeds to the list, click the “Set Limits” button to advance to crawl configurations. Like any manually launched crawl, select its test or one-time production type and apply any desired document, data, or time limits. 

Step 4: Run your crawl

Clicking the Crawl button will start your crawl.

Outcome

Once initiated, your crawl will be added to the “Current Crawls” lists in your account and your new or existing collection. You may monitor and review this crawl’s reports as you would any other Archive-It crawl.

Related content

How to add and use the Archive This! bookmarklet

How to manually start test and one-time crawls

How to run, monitor, and save a test crawl