Rotary International maintains a couple of different sites that are available in multiple languages. Ideally we want to capture all language versions. And this should be easy, because each translated site is a subdirectory under the main domain:
So setting up the seed URL as rotary.org/ captures all languages.
But my most recent crawl of endpolio.org, which is set up similarly, only seems to be capturing the default English version of the site.
Maybe Brozzler can't access the other sites via the drop down menu - but I'm not really sure.
Anyone else encountered a similar problem? I'd like to avoid having to create separate seed URLs for each language, if I can.
Please sign in to leave a comment.