Archive-It will not store the same content twice from the same web page. The service uses URL agnostic data-deduplication (commonly referred to as 'de-dupe') in order to ensure that any URL archived–be it a homepage URL, an image file, CSS stylesheet, etc.–will not be archived again if the content has not changed. Even in the initial crawl of a seed URL, resources may exist throughout the site in a duplicate form and you may notice de-duplication in effect while analyzing the number of documents and data archived.
What is the difference between all and new documents/data?
Comments
0 comments
Please sign in to leave a comment.