Posts

  • Autopilot: Testable Automated Behaviors for ArchiveWeb.page and Browsertrix

    Autopilot in ArchiveWeb.page Web archiving can be complex and often tedious work, especially when trying to archive dynamic, infitely complex content such as social media. A key goal of Webrecorder tools is to make web archiving simpler, and we’ve taken an important step with latest update to our tools. Over the last week, the Webrecorder team has been quietly testing our new automated, in-page behavior system, sometimes also known as Autopilot! The system is available...

  • Introducing Browsertrix Crawler

    I wanted to more publicly announce Webrecorder’s new automated browser-based crawling system: Browsertrix Crawler. The premise of the crawler is simple: to run a single command that produces a high-fidelity crawl (based on the specified params and config options). The Browsertrix Crawler is a self-contained, single Docker image that can run a full browser-based crawl, using Puppeteer. The Docker image contains pywb, a recent version of Chrome, Puppeteer and a customizable JavaScript ‘driver’. The crawler...

  • Announcing New ArchiveWeb.page App, Deprecating Older Tools

    Over the years, the Webrecorder project has developed a lot of tools to make web archiving easier and accessible for all. To continue pushing the boundaries of high-fidelity web archiving and make tools that are easy to use and easy to maintain, it is sometimes necessary to discontinue older tools and focus on new ones. If you are currently using the following tools, we recommend transitioning to the newer tools mentioned below. If you’re using...

  • Announcing WACZ Format 1.0

    The Webrecorder team has just finished a new release for WACZ and we’re delighted to share it with you! WACZ stands for Web Archive Collection Zipped, and is a new file format designed to make creating and hosting web archives quicker and easier. The format has been in development for a few months, and we’re excited to announce the release of WACZ Format 1.0. The spec for the format can be found on github. ReplayWeb.page...

  • Introducing ArchiveWeb.page - Local High-Fidelity Web Archiving directly in your browser

    Introducing ArchiveWeb.page Chrome Extension I am excited to announce the launch of ArchiveWeb.page, a brand-new high-fidelity web archiving system available as a Chrome extension from the Chrome Web Store The extension has been tested in latest versions of Chrome, as well as with the Edge and Brave browsers. In classic Webrecorder style, the extension allows users to ‘record’ highly interactive websites, including social media, video, customized content, and even local intranet content. When the original...

  • Flash Ain't Dead Yet! Even more ways to run Flash using OldWeb.today

    Faster Emulation, More Browsers A new version of OldWeb.today was released two weeks ago, switching to in-browser Javascript and WebAssembly emulation. Today, one of the emulators used in OldWeb.today, v86, received a major upgrade with WebAssembly. OldWeb.today has now also been updated to support this new version, allow for Windows and Linux based browsers to run even faster. This should also offer a noticeable upgrade to Flash emulation in these browsers. OldWeb.today now supports five...

  • Announcing the New OldWeb.today

    Just over five years ago, at the beginning of December 2015, I released the initial version of OldWeb.today, which demonstrated running emulated browsers connected to web archives. This system used Docker to run emulated versions of browsers in the cloud, and required significant resources to maintain and could only support a fixed number of users at a time. (The old version of OWT is still available as classic.oldweb.today) I imagined that this would be a...

  • OpenWayback to pywb Transition Guide and pywb update

    Earlier this year, members of the IIPC (International Internet Preservation Consortium), after an internal survey, recommended the adoption of Webrecorder pywb as the primary replay system for their members’ web archives. Webrecorder and IIPC established a multi-part collaboration to help with this transition and advance the development of pywb. To meet these goals, I’m excited to announce the launch of an official guide for migrating from OpenWayback to Webrecorder pywb, available at: https://pywb.readthedocs.io/en/latest/manual/owb-transition.html This guide...

  • Web Object Encapsulation Complexity (Part I)

    What does it take to archive a web page/project? As the web transitioned from static documents to interactive web applications, the challenge of archiving and preserving the web have only increased. But some web pages/projects/publications - lets refer to them as ‘web objects’ - are easier to archive than others. Some require no effort at all, while others require significant effort and still can not be correctly archived. Sure, the number of pages there are,...

  • Lorena Ramírez-López joins Webrecorder as Community Manager

    I’m excited to announce that Lorena Ramírez-López (DaleLore.net) has joined Webrecorder team as a part-time community manager! Lorena is a trained moving image specialist for film, video and digital collections. Her main interests focus on the preservation and conservation of time-based media art as well as research and access to Net Art and web archives. A native New Yorker from Queens, Lorena believes in access and sharing resources, which is why she participates and collaborates...

  • Web Archives as Digital Publications / Digital Publications as Web Archives

    Web archiving is often done after the fact – a digital publication is designed, built, published and only then, archived for preservation. But what if the archiving process became part of the publication pipeline, complementary to online publishing, or an alternative distribution medium free from hosting requirements and available for offline use? Parametric Press, a digital magazine focused on interactive, data-driven content has been experimenting with this approach from the beginning. With the release of...

  • Emma Dickson joins Webrecorder as Generalist Developer

    Emma Dickson Joins Webrecorder as a Generalist Developer I’m excited to announce that the Webrecorder team has expanded, and that Emma Dickson has joined Webrecorder as a part-time Generalist Developer! Emma is fascinated by outdated technology and the process of translation and obsolescence in technical languages. They love creating archives and archival tools. In addition to their work as a conservation technician on time-based media projects, Emma also produces net art and new media sculptures....

  • Next Generation Web Archiving: Loading Complex Web Archives On-Demand in the Browser

    I’m excited to present an exciting new milestone for Webrecorder, the release of six high-fidelity web archives of complex digital publications, accessible directly in any modern browser. These projects represent the entire catalog of Stanford University Press’s Mellon-funded digital publications, and are the culmination of a multi-year collaboration between Webrecorder and Stanford University Press (SUP). You can read more about this collaboration, and additional details on each of the publications on the corresponding blog post...

  • Supporting IIPC community in transitioning to Webrecorder pywb

    I’m excited to announce an exciting new collaboration between Webrecorder and International Internet Preservation Consortium (IIPC), a group of national, university and regional libraries and archives involved in web archiving all over the world. The IIPC will recommend the adoption of Webrecorder pywb, the core Python web archiving toolset developed by Webrecorder as the ‘go to’ web archive replay system. To support IIPC members in switching to pywb, I will be developing a migration guide,...

  • A New Phase for Webrecorder Project, Conifer and ReplayWeb.page

    Today, I’m excited to announce a new phase for the Webrecorder Project, and several major releases/updates. First, welcome to https://webrecorder.net/ - the new official site of the Webrecorder Project. Feel free to look around, and pardon the dust. This site will contain all news and updates from Webrecorder, and the tools page is being updated to maintain a current index of all Webrecorder software. Long Live Webrecorder, Long Live Conifer In 2014, I created Webrecorder...

Subscribe