I am trying to archive all content written by Jens Foell to the site realscientists.org.
I used the seed URI http://realscientists.org/author/jens-foell which contains links to all of the author's contributions to this site.
I am using the Standard seed type and have not modified the scoping rules from the default.
Archive-It archives the seed URI and the next page URI of http://realscientists.org/author/jens-foell/page/2/, but according to the crawl reports, it considers all of the article URIs to be out of scope.
Does anyone have suggestions as to why Archive-It considers URIs like these to be out of scope:
This is not the only site I am archiving where large parts of the crawl are considered to be out of scope, so I would really like to understand what is going on.
Thanks in advance,
Please sign in to leave a comment.