A New Phase for Webrecorder Project, Conifer and ReplayWeb.page

By Ilya Kreymer

Today, I’m excited to announce a new phase for the Webrecorder Project, and several major releases/updates.

First, welcome to https://webrecorder.net/ - the new official site of the Webrecorder Project. Feel free to look around, and pardon the dust.

This site will contain all news and updates from Webrecorder, and the tools page is being updated to maintain a current index of all Webrecorder software.

Long Live Webrecorder, Long Live Conifer

In 2014, I created Webrecorder with the goal of building high quality tooling to support “web archives for all”, allowing anyone to create and share exactly what they experience in their browser, capturing interactive web sites as accurately as possible. Webrecorder started with a hosted service at webrecorder.io, but has since grown into an ecosystem of open source tools and free products to support web archiving capture and replay.

Thanks to generous support from the Mellon Foundation, Webrecorder was able to join Rhizome and together this hosted service was developed with a robust set of features, including remote browser and Autopilot automation, providing high-fidelity web archiving with a trusted cultural and digital arts institution. The hosted service known as webrecorder.io has now been renamed to Conifer and Rhizome will continue to run this unique service and build new features around this service. See Rhizome’s blog post for more details about Conifer and a brief post on the Conifer blog.

With the launch of Conifer, Rhizome will focus on running a brand new web archiving service, while Webrecorder will focus on software development.

As of 2020, Webrecorder is once again an independent project. Organizationally, I have formed ‘Webrecorder Software LLC’ to represent my work but the goals of the project remain the same as ever.

The Webrecorder Project will continue to work with Rhizome, and with many other partners to develop and maintain the best free and open source web archiving tools, and to further push the web archiving field forward with accessible, free and easy to use tools for all.

With that, I’d like to introduce the newest addition to the Webrecorder tool suite.

Introducing ReplayWeb.page

TODO TODO

In an uncertain world, web archiving is becoming critical more than ever. A key, if not defining reason, for creating web archives, is to be able to access (“replay”) the archived web sites at a later time. The Webrecorder project has always focused on the accuracy and fidelity of capture and replay — recording and reproducing interactive websites as accurately as possible compared to the original.

Six years ago, when Webrecorder.io was started, high-fidelity web archive replay was possible only through a hosted service/centralized server. But with advances in browser and web technology, along with growth in decentralized web and storage, this is certainly no longer the case! Today’s browsers can natively run all sorts of applications, including a fully-featured web archive replay system.

Previously, I’ve introduced wabac.js, as a fully Javascript, browser-based experiment for rendering web archive replay. Today, I want to announce ReplayWeb.page, a new, fully featured browser-based replay system (or ‘wayback machine’) that further develops this idea.

The entire system is implemented as a static web page running from GitHub (https://github.com/webrecorder/replayweb.page), and is bundled as just two Javascript files: one for UI and one for the backend/service worker. ReplayWeb.page can load web archives located anywhere on the web, or from your local machine. No data is uploaded anywhere, and the browser stores the web archive (or loads it directly from the file system). ReplayWeb.page provides a brand new interface and a new replay engine, but should remain fully compatible with the existing Webrecorder (now Conifer) system, including supporting familiar curatorial features such as lists.

Webrecorder’s original goal of ‘web archives for all’ can only be realized when users have the tools to create and view web archives, on their own devices, or having a choice as to where to store their data. ReplayWeb.page takes a step further in this direction, by allowing a wide array of options for web archive storage. Have web archives on an institutional repository, or S3, or any cloud storage? No problem! ReplayWeb.page can load web archives directly from there!

Have web archives on Google Drive that should only be shared with select collaborators? The ReplayWeb.page Google Drive integration should allow that! Interested in storing data on decentralized web? ReplayWeb.page is designed to be able to support IPFS and Dat/Hypercore protocols in the future as well.

For larger local archives, or for archives requiring Flash, there is also the ReplayWeb.page App, which provides the same interface, but can support Flash even if your current browser can not. The ReplayWeb.page App fully replaces Webrecorder Player for offline use.

See the full User Docs for more details or just try out https://replayweb.page/ for yourself!

Not sure where to start? Here’s some example of web archive links to get started with replayweb.page:

More Updates / Summary of Changes

That was a lot of info! Here is a few more recent releases and overall summary of changes.

Webrecorder or Conifer?

TODO

Perhaps you are confused about what tools will be provided by whom. Don’t worry! Here is a more clear delineation:

  • If you are interested in running web archiving tools on your own, running a desktop app, pywb, ReplayWeb.page, etc… Webrecorder will continue maintaining these tools! If you would like to integrate web archiving into other software or service, Webrecorder is here to help!

  • If you are looking for a well-established arts non-profit that has been committed to digital preservation to provide a dedicated web archiving service, then Rhizome’s Conifer is for you!

Of course, you can continue to use both and we will continue to work together in expanding the web archiving ecosystem.

Thank Yous

I wanted to thank Ashley Blewer for help in making this site and the new Webrecorder logo.

I would like to thank the folks at Rhizome for their continued support for Webrecorder from 2016-2020. The Webrecorder team that was at Rhizome over the years: Mark, Pat, Anna, John and especially Dragan, and Zachary Kaplan for supporting Webrecorder all the way as Executive Director of Rhizome. I look forward to further collaboration as we continue to work on Webrecorder and Conifer.

Stay in touch

Follow @webrecorder_io or @IlyaKreymer or check back on this blog for latest updates on Webrecorder.

You can also reach me via e-mail.

Stay tuned for a lot more updates in the coming weeks!

Ilya