Comparing live sites and captures

Follow

Evie Morris

July 08, 2025 09:02

Hi all,

We are having a few issues with a site we have to crawl monthly. Overall a large bulk of the content changes very rarely, so we are looking for small differences/updates and ensuring we have captured them correctly for compliance. We find that QA by eye - side by side windows, playing professional spot-the-difference - isn't as effective as we would like, particularly after the 3rd day looking at the website. 

Wondering if anyone has any "tricks" or methods to identify discrepancies, either between crawls and a live site or between a new crawl and the previous? I'm aware of the 'compare' function in Archive-It, but that doesn't direct you to the location of the differences (that I can work out).

Any and all suggestions appreciated :)

1

• 

  Evie Morris September 16, 2025 11:57

  We have developed a python pipeline which uses difflib to identify differences in the html files produced by the live site and capture. If anyone is interested, do get in touch :)

  1

  Comment actions Permalink

Please sign in to leave a comment.