Docs Improvements (#39)

* Restructure - wrote and added photos for online and offline use

* add new images

* add more info on embedding options, add CORS configuration example

Co-authored-by: DaleLore <DaleLore@protonmail.com>
This commit is contained in:
Ilya Kreymer
2021-03-17 20:34:19 -07:00
committed by GitHub
co-authored by DaleLore
parent 6035cbae50
commit f35776e165
25 changed files with 286 additions and 86 deletions
+2
View File
@@ -1,3 +1,5 @@
source 'https://rubygems.org'
gem "just-the-docs" gem "just-the-docs"
gem "jekyll-remote-theme" gem "jekyll-remote-theme"
+1
View File
@@ -1,4 +1,5 @@
GEM GEM
remote: https://rubygems.org/
specs: specs:
addressable (2.7.0) addressable (2.7.0)
public_suffix (>= 2.0.2, < 5.0) public_suffix (>= 2.0.2, < 5.0)
+4 -1
View File
@@ -4,7 +4,7 @@ remote_theme: pmarsceill/just-the-docs
title: ReplayWeb.Page title: ReplayWeb.Page
# Enable or disable the site search # Enable or disable the site search
search_enabled: false search_enabled: true
# Set the search token separator for hyphenated-word search: # Set the search token separator for hyphenated-word search:
search_tokenizer_separator: /[\s/]+/ search_tokenizer_separator: /[\s/]+/
@@ -23,6 +23,9 @@ plugins:
- jekyll-remote-theme - jekyll-remote-theme
version:
- 1.3.11
#baseUrl: '/' #baseUrl: '/'
exclude: exclude:
Binary file not shown.

After

Width:  |  Height:  |  Size: 592 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 6.5 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 825 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 8.0 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 71 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 75 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 9.9 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 79 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 300 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 452 KiB

+34
View File
@@ -0,0 +1,34 @@
---
layout: page
title: Contact
nav_order: 6
description: Contact Webrecorder
permalink: /docs/contact
---
# CONTACT US
We are always trying to improve and we'd appreciate your feedback on how we can make the documentation easy and concise to support your needs! Please contact us in any of the following ways. Looking forward to hearing from you.
### Discuss: Post on our forum
We have a forum for all our users!
Webrecorder users post their questions, insights, and issues on our <a href="https://forum.webrecorder.net/" target="_blank"><b>forum</b></a>. Join the webrecorder community to participate as well!
If you are running into an issue with the extension, check the forum to see if it has been discussed there already.
### GitHub: Open an Issue
The docs and the code for replayweb.page are both on GitHub.
- If you found mistake or a bug, please open an issue at [webrecorder/replayweb.page](https://github.com/webrecorder/replayweb.page){:target=_blank}
### Email: Send us a message
We prefer to discuss issues or bugs on the forum or on github so that others running into the same issues can see the previous discussion.
For other questions,, you can also reach us by e-mail.
Send us a message at <info@webrecorder.net> and someone from the Webrecorder team will get back to you.
+70 -10
View File
@@ -23,7 +23,7 @@ the following snippet to your HTML page:
my-web-archive-embed.html my-web-archive-embed.html
```html ```html
<script src="https://unpkg.com/replaywebpage@1.0.0/ui.js"></script> <script src="https://cdn.jsdelivr.net/npm/replaywebpage/@{{ site.version }}/ui.js"></script>
<replay-web-page source="s3://webrecorder-builds/warcs/netpreserve-twitter.warc" <replay-web-page source="s3://webrecorder-builds/warcs/netpreserve-twitter.warc"
url="https://twitter.com/netpreserve"></replay-web-page> url="https://twitter.com/netpreserve"></replay-web-page>
``` ```
@@ -39,7 +39,7 @@ one-line script
./replay/sw.js ./replay/sw.js
```javascript ```javascript
importScripts("https://unpkg.com/replaywebpage@1.0.0/sw.js"); importScripts("https://cdn.jsdelivr.net/npm/replaywebpage/@{{ site.version }}/sw.js");
``` ```
Thus, if the HTML snippet was added to `https://my-site.example.com/path/my-web-archive-embed.html` Thus, if the HTML snippet was added to `https://my-site.example.com/path/my-web-archive-embed.html`
@@ -47,7 +47,7 @@ then the sw.js should be added such that it is at: `https://my-site.example.com/
That's it! Loading `https://my-site.example.com/path/my-web-archive-embed.html` should now load the web archive. That's it! Loading `https://my-site.example.com/path/my-web-archive-embed.html` should now load the web archive.
(Be sure to add sizes to the `<replay-web-page>` tag as needed to size the embed). (Be sure to add width and height styles to the `<replay-web-page>` tag as needed to size the embed).
You can replace `s3://webrecorder-builds/warcs/netpreserve-twitter.warc` with any web archive hosted on your site, You can replace `s3://webrecorder-builds/warcs/netpreserve-twitter.warc` with any web archive hosted on your site,
eg. `https://my-site.example.com/warcs/my-warc-file.warc`. eg. `https://my-site.example.com/warcs/my-warc-file.warc`.
@@ -56,19 +56,35 @@ eg. `https://my-site.example.com/warcs/my-warc-file.warc`.
If the file is loaded from a different origin, your site must have CORS access to download the web archive. If the file is loaded from a different origin, your site must have CORS access to download the web archive.
### Embedding Options
The `<replay-web-page>` tag is a web component and supports a number of additional attributes:
| Attribute | Description |
|:-------------|:-----------------|
| `source` | Source URL for the archive file. It should be a URL in one of the [supported formats](/docs/formats) loaded from one of the [support locations](/docs/locations) and is required. |
| `url` | The starting URL to load from the archive. If omitted, will start with the page list or URL search view. |
| `ts` | The timestamp of the starting URL to load. If omitted, the latest available version is used. |
| `deepLink` | (`true` / `false`) if set to true, allow 'deep linking' to exact pages in the embed, besides the starting URL. |
| `embed` | (`default` / `full` / `replayonly` ) - if set to `full`, will show the replayweb.page nav bar and logo. if set to `default`, omitted, will show the location bar. if set to `replayonly`, will only show the `url` and not location bar or any nav bar buttons, which may be useful for embedding a single page only. |
| `replayBase` | Location of the `sw.js` file, defaults to `./replay/` as mentioned above, but can be overridden. |
| `coll` | Internal id for this collection, usually generated automatically.
| `config` | Extra per collection config options (such as custom fuzzy matching rules, TODO add more info!) |
### Versioning ### Versioning
Note that the above example uses the paths as: Note that the above example uses the paths as, which is a CDN for Javascript packages:
- `https://unpkg.com/replaywebpage@1.0.0/ui.js` - `https://cdn.jsdelivr.net/npm/replaywebpage@{{ site.version }}/ui.js`
- `https://unpkg.com/replaywebpage@1.0.0/sw.js` - `https://cdn.jsdelivr.net/npm/replaywebpage@{{ site.version }}/sw.js`
Another alternative would be: Another alternative using a different would be:
- `https://cdn.jsdelivr.net/npm/replaywebpage@1.0.0/ui.js` - `https://unpkg.com/replaywebpage@{{ site.version }}/ui.js`
- `https://cdn.jsdelivr.net/npm/replaywebpage@1.0.0/sw.js` - `https://unpkg.com/replaywebpage@{{ site.version }}/sw.js`
These URLs point to a specific version of ReplayWeb.page software released on NPM, eg. `1.0.0`, meaning that your replay should stay stable, even if ReplayWeb.page is updated. These URLs point to a specific version of ReplayWeb.page software released on NPM, eg. `{{ site.version }}`, meaning that your replay should stay stable, even if ReplayWeb.page is updated.
You can choose another of ReplayWeb.page (or even try different versions) to ensure that you have the best available replay. You can choose another of ReplayWeb.page (or even try different versions) to ensure that you have the best available replay.
@@ -77,6 +93,50 @@ This addresses the potential issue of older sites breaking when web archive repl
For production use, it is advised against linking to the latest version, eg. `https://replayweb.page/ui.js` For production use, it is advised against linking to the latest version, eg. `https://replayweb.page/ui.js`
and `https://replayweb.page/sw.js` as these will be updated frequently and make break your embed. and `https://replayweb.page/sw.js` as these will be updated frequently and make break your embed.
## Common Issues
Below are some possible issues that you may encounter when embedding and possible workarounds.
### Embed is too small / doesn't fill page.
If the `<replay-web-page>` tag is the only element on a page, and you want it to use the full window width and height, adding the following
style should fix the issue:
```html
<style>
html, body {
width: 100%;
height: 100%;
}
</style>
```
### CORS restrictions
Browsers restrict access to files hosted on a different domain than the websites that is trying to load them.
If you are loading web archives hosted on a different domain (for example, S3 or other cloud storage) and your site is hosted on a different domain,
the site hosting the file needs to 'allow' the website to load the file using special CORS (Cross-Origin Resource Sharing) headers.
If you are hosting from S3 or S3-compatible service, here is a potential CORS bucket configuration that should work.
Replace `https://myarchive.example.com/` with the server (origin) of the URL where the `<replay-web-page>` embed is hosted.
```xml
<CORSConfiguration
xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
<CORSRule>
<AllowedMethod>GET</AllowedMethod>
<AllowedMethod>HEAD</AllowedMethod>
<AllowedOrigin>https://myarchive.example.com</AllowedOrigin>
<AllowedHeader>*</AllowedHeader>
<ExposeHeader>Content-Range</ExposeHeader>
<ExposeHeader>Content-Encoding</ExposeHeader>
<ExposeHeader>Content-Length</ExposeHeader>
</CORSRule>
```
See [S3 Docs](https://docs.aws.amazon.com/AmazonS3/latest/userguide/ManageCorsUsing.html) for more info on how to set this policy.
Other cloud providers may have a similar settings for configuring CORS.
+2 -1
View File
@@ -1,7 +1,8 @@
--- ---
layout: default layout: default
title: Examples title: Examples
nav_order: 1 nav_order: 7
parent: Reference
permalink: /docs/examples permalink: /docs/examples
--- ---
+4 -6
View File
@@ -1,8 +1,8 @@
--- ---
layout: default layout: default
title: Exploring Web Archives title: Exploring Web Archives
nav_order: 2 nav_order: 3
parent: About parent: Loading Web Archives
permalink: /docs/exploring permalink: /docs/exploring
--- ---
@@ -57,10 +57,8 @@ The contains option will not find any URLs that have not yet been loaded.
The Page view search includes page titles, urls and page full text search, if available. The Page view search includes page titles, urls and page full text search, if available.
ReplayWeb.page will currently generate full ext search data from WARC pages automatically. ReplayWeb.page will load pre-generated full-text search data pages when loading WACZ files, and will also attempt to generate full-text index from HTML pages when loading WARC files.
ReplayWeb.page will soon load existing extracted full-text data as well.
<hr> <hr>
Next: [Sharing Links to Archived Pages](sharing.md) Next: [Sharing Links to Archived Pages](/docs/sharing)
+32
View File
@@ -0,0 +1,32 @@
---
layout: page
title: Welcome
nav_order: 1
description: About ReplayWeb.page
permalink: /docs/index.html
---
# USER GUIDE: REPLAYWEB.PAGE
## What is this document?
Welcome to this instructional guide on how to use Webrecorder's replayweb.page for "<i>replaying</i>" your web archives in WARC, CDX, and WACZ formats (you can check out our [supported formats](/docs/formats) for more information).
## What is Replayweb.page?
ReplayWeb.page is an open-source is a browser-based viewer that loads web archive files provided by the user and renders them for replay in the browser.
It functions as a <a alt="A serverless (client-side) web archive which means that the action takes place on the user's (the client's AKA you) computer rather than server-side which means that the action takes place on a web server." title="This means that the action takes place on the user's (the client's AKA your) computer rather than server-side which means that the action takes place on a web server."><span style="cursor: help;">serverless (client-side)</span></a> web archive replay tool.
Archives can be loaded as static files from anywhere a browser can connect to, or from users local machine.
### ReplayWeb.page provides the following features:
* High-Fidelity replay of web archives in several formats and from various locations directly in the browser.
* Several way to explore web archives: story view, page search and URL search.
* On-demand, incremental loading of large archives.
* Several options for fully-functional offline usage.
* Available as a standalone desktop app with Flash support.
* Support for versioned embedding of web archives.
### Check it out on our <a href="https://github.com/webrecorder/replayweb.page" target="_blank">GitHub repo</a>
+26 -32
View File
@@ -1,31 +1,28 @@
--- ---
layout: default layout: default
title: Loading Web Archives title: Online Use
nav_order: 1 nav_order: 1
parent: Usage parent: Loading Web Archives
permalink: /docs/loading permalink: /docs/loading_online
--- ---
# Online Use
# Browsing Web Archives via the ReplayWeb.page
{: .no_toc } {: .no_toc }
## Table of contents Using [https://replayweb.page](https://replayweb.page), you can select a web archive file from a local file or enter their unique remote URL directly to load into the site.
<img alt="screenshot of replayweb.page (online)" src="/docs/assets/replaywebpage-online.png">
<hr>
## Browsing Web Archives via the ReplayWeb.page
{: .no_toc .text-delta } {: .no_toc .text-delta }
1. TOC
1.
{:toc} {:toc}
## Loading Web Archives
Using [https://replayweb.page](https://replayweb.page), you can select a web archive file from a local
file or remote URL to load into the site.
Archives loaded by ReplayWeb.page are identified by their unique URL, which can be entered directly or selected from a local file chooser.
### Loading Local Web Archives ### Loading Local Web Archives
Using the `Choose File...` button, you can pick any local archive file to load for replay. From [https://replayweb.page](https://replayweb.page), use the `Choose File...` button. You can pick any local archive file from your computer to load for replay.
The click `Load` to load the file. The click `Load` to load the file.
The archive is loaded in the browser directly and *not* uploaded anywhere -- it never leaves your computer! The archive is loaded in the browser directly and *not* uploaded anywhere -- it never leaves your computer!
@@ -36,7 +33,9 @@ this URL is not shareable.
{: .fs-3 .pad .bg-grey-lt-100} {: .fs-3 .pad .bg-grey-lt-100}
This behavior is similar to how [Webrecorder Player](https://github.com/webrecorder/webrecorder-player) worked, but happens entirely in the browser. <br>The [ReplayWeb.page App](https://github.com/webrecorder/replayweb.page/releases) is also an option for [opening local files when offline](offline#replaywebpage-as-standalone-desktop-electron-app) This behavior is similar to how [Webrecorder Player](https://github.com/webrecorder/webrecorder-player) worked, but happens entirely in the browser. <br>The [ReplayWeb.page App](https://github.com/webrecorder/replayweb.page/releases) is also an option for [opening local files when offline](offline#replaywebpage-as-standalone-desktop-electron-app)
<img alt="screenshot of replayweb.page (online)" src="/docs/assets/replaywebpage-online-local.png">
<hr>
### Loading Remote Web Archives ### Loading Remote Web Archives
To load a remote archive, simply enter the URL of the archive and click `Load`. To load a remote archive, simply enter the URL of the archive and click `Load`.
@@ -51,39 +50,34 @@ The system supports WARC files, as well as several other formats
{: .fs-3 .pad .bg-grey-lt-100} {: .fs-3 .pad .bg-grey-lt-100}
See: [Supported Formats](formats) for more details on supported formats. See: [Supported Formats](formats) for more details on supported formats.
<hr>
### Loading Remote Archives from Google Drive ### Loading Remote Archives from Google Drive
ReplayWeb.page also supports loading archives directly from Google Drive. ReplayWeb.page also supports loading archives directly from Google Drive.
ReplayWeb.page can be used to open .warc, .warc.gz files directly from Google Drive using the following steps. You would first need to integrate replayReplayWeb.page, but it's a fairly easy integration. And once integrated, you can use it to open .warc, and .warc.gz files directly from Google Drive using the following steps.
1. Install [ReplayWeb.page Google Drive Integration](https://gsuite.google.com/u/2/marketplace/app/replaywebpage/160798412227) 1. Navigate to any WARC (.warc, .warc.gz) files or WACZ (Web archive collection) files that you have on Google Drive.
2. Navigate to any WARC (.warc, .warc.gz) files or WACZ (Web archive collection) files that you have on google drive. 2. When you select the file, you should see an 'Open With ReplayWeb.page' option at the top.
3. You should see an 'Open With ReplayWeb.page' option at the top. 3. Clicking on this should redirect to ReplayWeb.page and begin loading. Google Drive loaded archives will have a source URL starting with `googledrive://`.
4. Clicking on this should redirect to ReplayWeb.page and begin loading. Google Drive loaded archives will have a source URL starting with `googledrive://`.
### Loading Large Files: Full vs On-Demand Loading #### Install [ReplayWeb.page Google Drive Integration](https://gsuite.google.com/u/2/marketplace/app/replaywebpage/160798412227)
* This is a Google Drive Add-On
* Click install and Google will walk you through the steps of installation
* Once installed, Google will confirm the install and where to find replayweb.page in your drive.
Due to the nature of the WARC format, the entire file must be read on first use to generate an index. <img alt="screenshot of replayweb.page google drive add-on" src="/docs/assets/replaywebpage-googledrive-install.gif">
For WARC files **>25MB**, only the index is initially stored in the browser, and the actual content is loaded 'on-demand',
when the content is first accessed. This leads to faster loading and saves memory when dealing with large archives.
[Web Archive Collection (WACZ)](wacz-format) are always loaded on-demand, as no indexing is required.
The initial archive view should load almost instantly as a result.
If an archive could not be loaded, an error will be displayed instead of the progress.
Please report any issues encountered via [GitHub Issues](https://github.com/webrecorder/replayweb.page/issues)
<hr> <hr>
Once an archive has successfully loaded, you will be redirected to the archive view from where you can explore the archive. Once an archive has successfully loaded, you will be redirected to the archive view from where you can explore the archive.
Next: [Exploring the Archive](explore) Next: [Exploring the Archive](/docs/exploring)
+1 -1
View File
@@ -25,7 +25,7 @@ The following is a list of currently supported URL schemes and details below:
| `file://` | <span class="d-inline-block p-2 mr-1 v-align-middle bg-green-000"> Supported | | `file://` | <span class="d-inline-block p-2 mr-1 v-align-middle bg-green-000"> Supported |
| `s3://` | <span class="d-inline-block p-2 mr-1 v-align-middle bg-green-000"> Supported | | `s3://` | <span class="d-inline-block p-2 mr-1 v-align-middle bg-green-000"> Supported |
| `googledrive://` | <span class="d-inline-block p-2 mr-1 v-align-middle bg-green-000"> Supported | | `googledrive://` | <span class="d-inline-block p-2 mr-1 v-align-middle bg-green-000"> Supported |
| `ipfs://` | <span class="d-inline-block p-2 mr-1 v-align-middle bg-purple-000"> Planned | | `ipfs://` | <span class="d-inline-block p-2 mr-1 v-align-middle bg-green-000"> Supported |
| `hyper://` | <span class="d-inline-block p-2 mr-1 v-align-middle text-grey-lt-000 bg-purple-000"> Planned | | `hyper://` | <span class="d-inline-block p-2 mr-1 v-align-middle text-grey-lt-000 bg-purple-000"> Planned |
+60 -10
View File
@@ -1,33 +1,67 @@
--- ---
layout: default layout: default
title: Offline Use title: Offline Use
nav_order: 4 nav_order: 2
parent: Usage parent: Loading Web Archives
permalink: /docs/offline permalink: /docs/loading_offline
--- ---
# Offline Use # Offline Use
{: .no_toc }
## ReplayWeb.page is available for offline use in several ways.
{: .no_toc .text-delta }
1.
{:toc}
ReplayWeb.page is available for offline use in several ways.
## ReplayWeb.page Offline Cache ## ReplayWeb.page Offline Cache
After ReplayWeb.page is loaded, it can automatically be used offline without requiring an internet connection. After ReplayWeb.page is loaded, it can automatically be used offline without requiring an Internet connection.
If a connection is not available, all web archive already loaded will continue to be available. That's to say, once you have loaded ReplayWeb.page online (with an Internet connection), if an Internet connection is not available later on, all web archives already loaded will continue to be available because it's cached.
To do this, https://replayweb.page/ in your Chrome browser, and the site will start loading local archives even without an Internet connection.
<details>
<summary>
What's cache?
</summary>
<p>
Cache is a <i>temporary</i> storage space in your browser that keeps certain files. Cached data is used to quickly load an application or website (like replayweb.page!) every time you log on.
</p>
</details>
[Loading local archives](loading#loading-local-web-archives) should fully work without an internet connection.
## ReplayWeb.page as Progress Web App ## ReplayWeb.page as Progress Web App
ReplayWeb.page can also be installed as [Progressive Web App](https://developer.mozilla.org/en-US/docs/Web/Progressive_web_apps) and be available as a standalone app on desktop and mobile. ReplayWeb.page can also be installed as [Progressive Web App](https://developer.mozilla.org/en-US/docs/Web/Progressive_web_apps) and be available as a standalone app on desktop and mobile.
This is still experimental, but should provide the same behavior as regular https://replayweb.page/ These type of apps use the same, current version of Chrome you normally use while searching or browsing the web. This is still experimental, but should provide the same behavior as regular https://replayweb.page/
To do this, on the ReplayWeb.page site, you might have noticed a "new window icon" in your search bar. Clicking on this icon will open the progressive web app on your computer so you can use it offline.
<details>
<summary>
What's a Progressive Web App?
</summary>
<p>
Progressive Web Apps can be considered as a cross-platform app: part web and part native. PWAs are built on the web and has web features that make is faster and easier to discover and share. AND it also has native features that makes it better integrated with your operating system and allows you to install on your computer so that it works offline!
</p>
</details>
<img alt="screenshot of replayweb.page (offline)" src="/docs/assets/replaywebpage-pwa.png">
## ReplayWeb.page as Standalone Desktop (Electron) App ## ReplayWeb.page as Standalone Desktop (Electron) App
ReplayWeb.page is also available as a standalone Electron App. In this mode, ReplayWeb.page ReplayWeb.page is also available as a standalone Electron App. In this mode, ReplayWeb.page packages its own version of Chrome.
ships its own version of Chrome.
To do this, go to our [GitHub repo for replayweb.page](https://github.com/webrecorder/replayweb.page) and scroll down to "what's in this repo" for [app releases](https://github.com/webrecorder/replayweb.page/releases). You can download as a .dmg (for Apple) or .exe (for Windows).
Although the Web Page/Progressive Web App versions have almost the same features, the standalone App is useful for two primary reasons: Although the Web Page/Progressive Web App versions have almost the same features, the standalone App is useful for two primary reasons:
@@ -39,3 +73,19 @@ Once loaded, ReplayWeb.page App can continue to load web archives from disk.
The desktop app fully replaces [Webrecorder Player](https://github.com/webrecorder/webrecorder-player) The desktop app fully replaces [Webrecorder Player](https://github.com/webrecorder/webrecorder-player)
When using [https://replayweb.page](https://replayweb.page), you may be asked to re-open a local file because a web site can not have persistent access to your file system. When using [https://replayweb.page](https://replayweb.page), you may be asked to re-open a local file because a web site can not have persistent access to your file system.
<details>
<summary>
What's a standalone desktop (Electron) app?
</summary>
<p>
Electron is the framework being used for a native application. A standalone desktop app just means that you can download the application to your desktop and open it up whenever without Internet!
</p>
</details>
<img alt="screenshot of replayweb.page (offline)" src="/docs/assets/replaywebpage-releases.png">
<hr>
Once an archive has successfully loaded, you will be redirected to the archive view from where you can explore the archive.
Next: [Exploring the Archive](/docs/exploring)
-1
View File
@@ -2,7 +2,6 @@
layout: default layout: default
title: Sharing Web Archives title: Sharing Web Archives
nav_order: 3 nav_order: 3
parent: Usage
permalink: /docs/sharing permalink: /docs/sharing
--- ---
+35
View File
@@ -0,0 +1,35 @@
---
layout: page
title: Troubleshooting
nav_order: 4
description: Troubleshooting ReplayWeb.page
permalink: /docs/troubleshooting
---
This section will be dedicated to common errors and issues that we’ve encountered during web archiving.
### WARC file is too big / too slow / fails to load.
#### Loading Large Files: Full vs On-Demand Loading
Due to the nature of the WARC format, the entire file must be read on first use to generate an index.
For WARC files **>25MB**, only the index is initially stored in the browser, and the actual content is loaded 'on-demand',
when the content is first accessed. This leads to faster loading and saves memory when dealing with large archives, but the entire
file must still be read.
[Web Archive Collection (WACZ)](wacz-format) are always loaded on-demand, as no indexing is required.
The initial archive view should load almost instantly as a result.
We recommend converting larger WARC files to WACZ files.
This is currently possible by using the `wacz` python library, see [py-wacz documentation](https://github.com/webrecorder/wacz-format/tree/main/py-wacz)
for more info.
While we've recently made improvements to large WARC loading, for very large files, this may still take some time and occasionally the browser may not load the full file.
If an archive could not be loaded, an error will be displayed instead of the progress.
Please report any issues encountered via [GitHub Issues](https://github.com/webrecorder/replayweb.page/issues)
## Have another error or issue?
Since we just launched, we’ll need your help with what we should troubleshoot! Please contact us and ask us questions and show us examples.
+13 -22
View File
@@ -1,35 +1,26 @@
--- ---
layout: default layout: default
title: Usage title: Loading Web Archives
nav_order: 2 nav_order: 2
description: About ReplayWeb.page description: How to use ReplayWeb.page
has_children: true has_children: true
permalink: /docs/index.html permalink: /docs/usage
--- ---
# Using ReplayWeb.Page # How to use ReplayWeb.Page
## Serverless / Client-Side Web Archive Replay So you generated WARC files after capturing your website. Now what? <br><br>
At this point in your web archiving workflow, you'll most likely want to <i><a alt="Replay in this case would be to reproduce, playback or see your web archives again." title="Reproduce, playback, or see again."><span style="cursor: help;">replay</span></a></i> these WARC files.
{: .fs-6 .fw-300 } This is where replayweb.page can be helpful for you! You can view your web archives either from various locations directly in the browser or offline on your computer.
ReplayWeb.page is an [open-source](https://github.com/webrecorder/replayweb.page) browser-based system to replay archived web pages as accurately as possible, with interactive elements preserved.
{: .fs-6 .fw-300 } ## <a href="/docs/loading_online">Online Use</a>
Archives can be loaded as static files from anywhere a browser can connect to, or from users local machine.
{: .fs-6 .fw-300 } Currently the easiest way to view your WARC and/or WACZ files is online through <a href="https://replayweb.page/" target="_blank">replayweb.page</a>. You can load web archives locally from your computer or from other remote web archives. Check out [Online Use](/docs/loading_online) for more information how to load your web archives.
ReplayWeb.page provides the following features:
- High-Fidelity replay of web archives in [several formats](formats) and from [various locations](locations) directly in the browser. <img alt="GIF of uploading a WACZ file into replayweb.page" src="/docs/assets/replaywebpage-online.gif">
- Several way to [explore web archives](explore): story view, page search and URL search.
- On-demand, incremental [loading of large archives](loading#loading-large-files-full-vs-on-demand-loading/).
- Several options for fully-functional [offline usage](offline).
- Available as a standalone [desktop app](https://github.com/webrecorder/replayweb.page/releases) with Flash support.
- Support for [versioned embedding of web archives](embedding).
## <a href="/docs/loading_offline">Offline Use</a>
You can run replayweb.page on your computer by running it locally. And then you can load web archives the same way you would online. Check out [Offline Use](/docs/loading_offline) for more information how to load your web archives - offline.
<img alt="GIF of uploading a WACZ file into replayweb.page running off of localhost (offline)" src="/docs/assets/replaywebpage-offline.gif">
+1 -1
View File
@@ -1,6 +1,6 @@
--- ---
layout: default layout: default
title: 'Web Archive Collection Zipped (WACZ) Format' title: WACZ Format
nav_order: 1 nav_order: 1
permalink: /docs/wacz-format permalink: /docs/wacz-format
parent: Reference parent: Reference