I am trying to crawl some Twitter hashtags related to our university but I'm not able to get it to work regardless of what I try.
More often than not the crawl produces a wayback instance with "no results found" but if one searches the hashtag on twitter there are multiple results.
I have experimented with both regular crawls and Brozzler. Brozzler seems to have better results in that if there is a twitter profile with the same name as the hashtag, it will grab that as a result but still does not pull individual tweets.
I'm logged in to twitter through the dummy account we use for web archiving and I've got everything scoped the way all the advice says to.
What technique do you use to successfully crawl hashtags in a way that produces a reliable result in the wayback instance?
Please sign in to leave a comment.