docs and README: add wacz format doc, tweak links, tweak README

This commit is contained in:
Ilya Kreymer
2020-06-11 15:08:24 -07:00
parent 53bb291406
commit ae872cd219
6 changed files with 42 additions and 12 deletions
+1 -1
View File
@@ -8,7 +8,7 @@ GEM
eventmachine (>= 0.12.9)
http_parser.rb (~> 0.6.0)
eventmachine (1.2.7)
ffi (1.13.0)
ffi (1.13.1)
forwardable-extended (2.6.0)
http_parser.rb (0.6.0)
i18n (0.9.5)
+10 -6
View File
@@ -5,14 +5,18 @@
## Serverless Web Archive Replay
ReplayWeb.page provides a full web archive replay system running directly in the browser,
available at: https://replayweb.page/
available at: [https://replayweb.page/](https://replayweb.page)
For full user docs, see: https://replayweb.page/docs
For full user docs, see: [https://replayweb.page/docs](https://replayweb.page/docs)
The ReplayWeb.page App can be downloaded from: https://replayweb.page/releases
The ReplayWeb.page App can be downloaded from the [Releases](https://replayweb.page/releases) page.
### Embedding Guide
See the [Embedding Guide](https://replayweb.page/docs/embedding) for more info on embedding web archives in other sites.
## Architecture / What's in this repo
## What's in this repo
ReplayWeb.page is a static web site / offline web app + Electron app.
@@ -59,12 +63,12 @@ For service workers to work, they must be served from either localhost or an HTT
See the [user docs](https://replayweb.page/docs/) for additional info about using ReplayWeb.page
## LICENSE
ReplayWeb.page is made available under the AGPLv3 License.
[Embedding ReplayWeb.page](https://replayweb.page/docs/embedding) from published releases is encouraged.
If you would like to use it under a different license or have a question, please reach out as that may be a possibility.
+2 -2
View File
@@ -24,12 +24,12 @@ The archive view presents several tabs:
- **Story** - This Story view presents lists of curated pages, as developed by the creator of the web archive.
This option is only shown if there is a curated story. As curated lists are not a standard part of WARC, only WARCs exported from Webrecorder.io/Conifer can have this option.
The new [Web Archive Collection (WACZ)](web-archive-collection-format) can also include curated lists.
The new [Web Archive Collection (WACZ)](wacz-format) can also include curated lists.
- **Pages** - The Pages view presents all pages in the web archive. As pages are not a standard part of WARC format,
generally only WARCs from Webrecorder.io/Conifer will have pages.
The new [Web Archive Collection (WACZ)](web-archive-collection-format) can also store pages.
The new [Web Archive Collection (WACZ)](wacz-format) can also store pages.
- **Page Resources** - This view allows searching the archive by URLs, as well as by common MIME type.
+1 -1
View File
@@ -12,7 +12,7 @@ parent: Reference
ReplayWeb.Page supports the archive formats listed below.
Format is currently determined based on the file extension.
The `.wacz` refers to the newly proposed [Web Archive Collection Zip Format](web-archive-collection).
The `.wacz` refers to the newly proposed [Web Archive Collection Zip Format](wacz-format).
| Format | Extensions | Status |
+2 -2
View File
@@ -44,7 +44,7 @@ To load a remote archive, simply enter the URL of the archive and click `Load`.
{: .fs-3 .pad .bg-grey-lt-100}
See [Supported Locations](locations) for details on where archives can be loaded from.
The archive will be downloaded, either fully or [only as needed (if possible)](streaming-archives.md) and presented on the archive page.
The archive will be downloaded, either fully or on-demand (if possible) and presented on the archive page.
The system supports WARC files, as well as several other formats
@@ -74,7 +74,7 @@ Due to the nature of the WARC format, the entire file must be read on first use
For WARC files **>25MB**, only the index is initially stored in the browser, and the actual content is loaded 'on-demand',
when the content is first accessed. This leads to faster loading and saves memory when dealing with large archives.
[Web Archive Collection (WACZ)](web-archive-collection-format) are always loaded on-demand, as no indexing is required.
[Web Archive Collection (WACZ)](wacz-format) are always loaded on-demand, as no indexing is required.
The initial archive view should load almost instantly as a result.
If an archive could not be loaded, an error will be displayed instead of the progress.
+26
View File
@@ -0,0 +1,26 @@
---
layout: default
title: 'Web Archive Collection Zipped (WACZ) Format'
nav_order: 1
permalink: /docs/wacz-format
parent: Reference
---
## Web Archive Collection Format Specification
ReplayWeb.page supports a new format for bundling raw web archive data (usually WARC files), indices,
page lists and other metadata into a single ZIP file.
The full spec for this format is available at: [https://github.com/webrecorder/web-archive-collection-format/blob/master/README.md](https://github.com/webrecorder/web-archive-collection-format/blob/master/README.md)
Files bundled into this format can use the .wacz (web archive collection zipped) file extension.
ReplayWeb.page will recognize this extension (as well as regular .zip) and will also load it from Google Drive when the
[Google Drive Integration](https://gsuite.google.com/u/2/marketplace/app/replaywebpage/160798412227) is installed.
The key benefit of this format is that large web archive collections can be loaded very quickly, to show the page list
and other key metadata, by downloading only parts of the WACZ file, as [outlined here](https://github.com/webrecorder/web-archive-collection-format/blob/master/README.md#appendix-a-use-case-random-access-to-web-archives-in-zip)
The actual raw content is loaded on-demand when the user requests each page.
With a WARC file, the entire contents must be loaded or indexed to determine the contents of the web archive collection.