New Wix seeds that contain wixsite.com or wix.com in their URLs will have the following default scoping rules automatically applied at the seed level when they are added to a collection. To learn more, including how you can add default scoping rules to existing seeds, please visit Sites with automated scoping rules.

It’s possible to crawl, archive, and replay most websites built on the platform Wix. These sites sometimes do not require any special steps. However, some Wix templates can result in incomplete capture or replay.  There are two options to help mitigate these issues: using Brozzler or using the standard crawler, with the below recommended scoping rules.  We recommend using one or the other (not both), but either of these options will increase the likelihood that the website will capture and replay fully, however some Wix sites experience further replay issues.  Please see our prioritization of support tickets for more information.

Scoping rules for Wix

Capturing Necessary Content: If not using Brozzler, to ensure that all necessary scripts, stylesheets, and embedded media are archived, we recommend modifying the scope of crawls of Wix-based sites to include the following rules:

1. Ignore Robots.txt files on the following hosts:
   • frog.wix.com
   • static.parastorage.com
2. Expand the scope of your crawl to include URLs that contain the following strings of text:
   • frog.wix.com
   • static.parastorage.com
   • static.wixstatic.com
   • sslstatic.wix.com

Avoiding Crawler Traps: Some templates used for Wix-based sites generate a significant number of long invalid URLs that look like the one below:

http://www.example.com/uU5dR1gpXCHX45K8aOMct11OrLtyrYJeUnw_RxaUsg.eyJpbnN0YW5jZUlkIjoiMTNkZDc

Add the following Regular Expression to keep the crawler from capturing unnecessary URLs generated by some wix sites ^.*/[^/]{300,}$

Adding One Page Seeds: Replay of sites built on the Wix platform may be improved by adding and crawling sub-pages(e.g. http://www.website.com/subpage) added as private One Page seeds. To do this, add sub-pages as private One Page seeds, add the recommended Wix scoping rules to each seed, and include them in a crawl with the main Wix seed. Keep in mind when you are reviewing your crawls, you should also navigate through the base domain's Wayback link to QA the sub-pages.