Important Update : Crawling Instagram

Comments

1 comment

  • Official comment
    Avatar
    Mary Haberle

    Please note that Instagram recently changed their robots.txt file. Now, to capture Instagram, you'll need to either ignore robots.txt for the both the hosts fbcdn.net and www.instagram.com at the collection level - OR - ignore robots.txt at the seed level for your Instagram seeds.

    If you were already ignoring robots.txt at the seed level, you will not need to make any changes in order to continue capturing Instagram.

    Our help center documentation on Instagram has been updated to reflect this change.

Please sign in to leave a comment.