Scalar crawling

Follow

Dan Nooonan

May 27, 2020 13:19

In late 2017, there was a posting "Has anyone been able to successfully crawl and capture a Scalar site? If so, what was your trick? Can you share?"

There were no responses then. Does anyone have insight now? Thank you!

0

• 

  Stephen Klein May 27, 2020 13:30

  Haven't tried in a while, but I have not been successful in the past and there is a partnership between Archive-It and Scalar. I have been manually crawling with WebRecorder.io to create WARCs.

  0

  Comment actions Permalink
• 

  Julianna Barrera-Gomez June 08, 2020 13:41

  I just tried a Scalar site we created with both Standard and Brozzler and I got nothing and 67 KB of random metadata from the page source, respectively. 

  Is there some official statement from Archive-It about this partnership, or any details on an expected timeline?

  Stephen Klein, if you don't mind: Are you making your WARCs accessible to users? How do you share information with the public that you've captured these? Thanks!

  0

  Comment actions Permalink
• 

  Stephen Klein June 08, 2020 16:32

  Hi Julianna,

   

  FYI:

  https://scalar.me/anvc/features/archive-partners/

   

  We lost our subscription due to funding issues, however as of 18 months ago I never had a successful crawl. I would crawl with Archive-It to get architecture, etc, but WebRecorder capture for hi-fidelity

  0

  Comment actions Permalink

Please sign in to leave a comment.