New seed-scope rule: Exclude audio and video

We’ve added a new seed-scope rule that makes it easier to manage crawls when you don’t need to collect audio or video.

When this rule is added to a seed: 

• Audio and video file types (e.g., mp3, mp4, wav, etc.) will not be collected from any page discovered from that seed
  
  
• Yt-dlp will not run on any pages discovered from the seed
  
  
• You can reduce crawl size by avoiding data-heavy media files
  
  

How to use it

You’ll find the new option to Exclude Audio and Video alongside your other rules when adding individual seed-scope rules or when adding rules to multiple seeds at the same time. For more information on all of Archive-It's scope rules, see our Scope Rules and how to use them article. 

We hope this helps you streamline your crawls and reduce data storage for media you don’t need in your web archives.