Archiving Instagram feeds

To Successfully Crawl Instagram Seeds:

1. Be specific. Always include a specific user, followed by a / at the end. For example https://www.instagram.com/internetarchive/
2. Add a collection level scoping rule to ignore robots.txt for the hosts www.instagram.com and fbcdn.net -OR- Ignore robots.txt at the seed level.

Your archived Instagram pages should play back accurately in Wayback with the following exceptions: 

1. We are at present only able to replay first two scrolls of the dynamically loading content for Instagram pages.
2. You may not be able to enlarge images or see their comments and likes without first detecting and patch crawling them through Wayback QA.