See [CORS restrictions](#cors-restrictions) below for more info. - -### Self Hosting - -Sometimes it can be desirable to *self-host* the user interface, service worker, and WACZ files. This is useful in situations where you want to prevent tracking by CDNs, or to make it easier to host the content in one place without needing to work out the details of Cross Origin Resource Sharing (CORS). Self-hosting is easy if you follow the same steps as above, but instead of loading the JavaScript and WACZ file from external locations you will need to [download](https://www.jsdelivr.com/package/npm/replaywebpage) the `ui.js` and `sw.js` JavaScript files and put them on the same server as the HTML that you are publishing. - -For example if you are publishing a page at `https://my-site.example.com/path/my-web-archive-embed.html` you could adjust the <script> element to load the ui from a location on your website: - -```html - - -
-
-Here is an example embed added directly to this page, with
-
-
-
-
-
-For a working example of hosting multiple web archives take a look at our *example-webarchive* [repository](https://github.com/webrecorder/example-webarchive/) and [static website](https://webrecorder.github.io/example-webarchive) which hosted on Github pages. It may give you ideas for how to integrate the ReplayWeb.page component into your site You may also be interested in [Web Replay Gen](https://github.com/webrecorder/web-replay-gen) which is an 11ty based static site builder for a web archive of WACZ files.
-
-### Embedding Options
-
-The `
-Here is an example embed added directly to this page, with replay-with-info embed mode:
-
-
-
-
-- -## Embed Examples - -Here are some additional examples of [embedding ReplayWeb.page](./embedding), hosted separately on [https://webrecorder.net](https://webrecorder.net){:target="_blank"} - -- [Embed Example 1](https://webrecorder.net/embed-demo-1.html){:target="_blank"} - -- [Embed Example 2](https://webrecorder.net/embed-demo-2.html){:target="_blank"} - - -Next: [Using ReplayWeb.page](./) diff --git a/docs/exploring.md b/docs/exploring.md deleted file mode 100644 index 84ccebc..0000000 --- a/docs/exploring.md +++ /dev/null @@ -1,64 +0,0 @@ ---- -layout: default -title: Exploring Web Archives -nav_order: 3 -parent: Loading Web Archives -permalink: /docs/exploring ---- - -## Exploring Archives (Browse, Search and Replay) - - -The ReplayWeb.page homepage lists an index of all loaded archives. You can also search by archive title or source -and filter by date loaded or title. - -Once loaded, the archive remains cached in the browser for quick access. This view will be unique to your browser. -To remove the archive from your browser, click on the 'X'. (Of course, this does not delete the archive from its original location, only your local copy of it) - -### Archive Views - -Although primarily designed for replay, ReplayWeb.page also offers several ways to interact with web archives. - -The archive view presents several tabs: - -- **Story** - This Story view presents lists of curated pages, as developed by the creator of the web archive. -This option is only shown if there is a curated story. As curated lists are not a standard part of WARC, only WARCs exported from Webrecorder.io/Conifer can have this option. - -The new [Web Archive Collection (WACZ)](wacz-format) can also include curated lists. - -- **Pages** - The Pages view presents all pages in the web archive. As pages are not a standard part of WARC format, -generally only WARCs from Webrecorder.io/Conifer will have pages. - -The new [Web Archive Collection (WACZ)](wacz-format) can also store pages. - - -- **Page Resources** - This view allows searching the archive by URLs, as well as by common MIME type. - -For many archives with no page or curatorial metadata, this is a way to explore the archive data in more detail. - -This view is available for all archives that only store raw data. - -- **Replay** - The view presents a replay of the archived web content in a mini-browser directly your browser. The view allows entering a URL directly. Clicking on links on any of the other views will switch to the **Replay** view. - -### Search - -Both the Page Resources view provide ways to search the archive directly. - - -#### Searching Page Resources -Page Resources allows searching by URL only, with additional sorting options. -Searches can be done by exact url, by url prefix, or by any string contained in the URL. - -The URL Prefix option is best for searching large archives that require on-demand loading. -The contains option will not find any URLs that have not yet been loaded. - - -#### Searching Pages - -The Page view search includes page titles, urls and page full text search, if available. - -ReplayWeb.page will load pre-generated full-text search data pages when loading WACZ files, and will also attempt to generate full-text index from HTML pages when loading WARC files. - - -
-Next: [Sharing Links to Archived Pages](/docs/sharing) diff --git a/docs/formats.md b/docs/formats.md deleted file mode 100644 index d4cf4ac..0000000 --- a/docs/formats.md +++ /dev/null @@ -1,28 +0,0 @@ ---- -layout: default -title: 'Supported Formats' -nav_order: 3 -description: 'Supported Formats' -permalink: /docs/formats -parent: Reference ---- - -## Supported Formats - -ReplayWeb.Page supports the archive formats listed below. -Format is currently determined based on the file extension. - -The `.wacz` refers to the newly proposed [Web Archive Collection Zip Format](wacz-format). - - -| Format | Extensions | Status | -|:--------|:--------------------|:--------------| -| WARC | `.warc`, `.warc.gz` | Supported | -| HAR | `.har` | In Progress | -| WBN | `.wbn` | Experimental | -| ARC | `.arc` | Not Supported | -| CDX | `.cdx`, `.cdxj` | Supported | -| WACZ | `.wacz` | In Progress | - - - diff --git a/docs/index.md b/docs/index.md deleted file mode 100644 index 05e238e..0000000 --- a/docs/index.md +++ /dev/null @@ -1,32 +0,0 @@ ---- -layout: page -title: Welcome -nav_order: 1 -description: About ReplayWeb.page -permalink: /docs/index.html - ---- - -# USER GUIDE: REPLAYWEB.PAGE - -## What is this document? -Welcome to this instructional guide on how to use Webrecorder's replayweb.page for "replaying" your web archives in WARC, CDX, and WACZ formats (you can check out our [supported formats](/docs/formats) for more information). - -## What is Replayweb.page? -ReplayWeb.page is an open-source, browser-based viewer that loads web archive files provided by the user and renders them for replay in the browser. - -It functions as a serverless (client-side) web archive replay tool. - -Archives can be loaded as static files from anywhere a browser can connect to, or from a user's local machine. - - -### ReplayWeb.page provides the following features: - -* High-Fidelity replay of web archives in several formats and from various locations directly in the browser. -* Several ways to explore web archives: story view, page search and URL search. -* On-demand, incremental loading of large archives. -* Several options for fully-functional offline usage. -* Available as a standalone desktop app with Flash support. -* Support for versioned embedding of web archives. - -### Check it out on our GitHub repo diff --git a/docs/loading.md b/docs/loading.md deleted file mode 100644 index 069b121..0000000 --- a/docs/loading.md +++ /dev/null @@ -1,86 +0,0 @@ ---- -layout: default -title: Online Use -nav_order: 1 -parent: Loading Web Archives -permalink: /docs/loading_online ---- -# Online Use -{: .no_toc } - -Using [https://replayweb.page](https://replayweb.page), you can select a web archive file from a local file or enter their unique remote URL directly to load into the site. - -
-- -## Browsing Web Archives via the ReplayWeb.page -{: .no_toc .text-delta } - - -1. -{:toc} - -### Loading Local Web Archives - -From [https://replayweb.page](https://replayweb.page), use the `Choose File...` button. You can pick any local archive file from your computer to load for replay. -The click `Load` to load the file. - -The archive is loaded in the browser directly and *not* uploaded anywhere -- it never leaves your computer! - -The archive will be identified by a `file://` URL and is not accessible outside your browser. As expected, -this URL is not shareable. - -{: .fs-3 .pad .bg-grey-lt-100} -This behavior is similar to how [Webrecorder Player](https://github.com/webrecorder/webrecorder-player) worked, but happens entirely in the browser.
The [ReplayWeb.page App](https://github.com/webrecorder/replayweb.page/releases) is also an option for [opening local files when offline](loading_offline#replaywebpage-as-standalone-desktop-electron-app) - -
-
--### Loading Remote Web Archives - -To load a remote archive, simply enter the URL of the archive and click `Load`. - -{: .fs-3 .pad .bg-grey-lt-100} -See [Supported Locations](locations) for details on where archives can be loaded from. - -The archive will be downloaded, either fully or on-demand (if possible) and presented on the archive page. - -The system supports WARC files, as well as several other formats - -{: .fs-3 .pad .bg-grey-lt-100} -See: [Supported Formats](formats) for more details on supported formats. - -
- -### Loading Remote Archives from Google Drive - -ReplayWeb.page also supports loading archives directly from Google Drive. - -You would first need to integrate replayReplayWeb.page, but it's a fairly easy integration. And once integrated, you can use it to open .warc, and .warc.gz files directly from Google Drive using the following steps. - -1. Navigate to any WARC (.warc, .warc.gz) files or WACZ (Web archive collection) files that you have on Google Drive. - -2. When you select the file, you should see an 'Open With ReplayWeb.page' option at the top. - -3. Clicking on this should redirect to ReplayWeb.page and begin loading. Google Drive loaded archives will have a source URL starting with `googledrive://`. - - -#### Install [ReplayWeb.page Google Drive Integration](https://gsuite.google.com/u/2/marketplace/app/replaywebpage/160798412227) -* This is a Google Drive Add-On -* Click install and Google will walk you through the steps of installation -* Once installed, Google will confirm the install and where to find replayweb.page in your drive. - -
-
-
-
--Once an archive has successfully loaded, you will be redirected to the archive view from where you can explore the archive. - -Next: [Exploring the Archive](/docs/exploring) - - - - - - diff --git a/docs/locations.md b/docs/locations.md deleted file mode 100644 index 250d114..0000000 --- a/docs/locations.md +++ /dev/null @@ -1,77 +0,0 @@ ---- -layout: default -title: 'Supported Locations' -nav_order: 2 -description: 'Supported Locations' -parent: Reference -permalink: /docs/locations ---- - -## Supported Location URLs - -ReplayWeb.Page uses URLs to uniquely identify web archives. - -In addition HTTP/S, the system supports loading from additional cloud providers -and via peer-to-peer decentralized services. The goal is to support loading -web archives from any source a web browser can connect to. - -The following is a list of currently supported URL schemes and details below: - - -| Scheme | Status | -|:-----------------|:------------| -| `https://` | Supported | -| `http://` | Supported | -| `file://` | Supported | -| `s3://` | Supported | -| `googledrive://` | Supported | -| `ipfs://` | Supported | -| `hyper://` | Planned | - - -### HTTP and HTTPS - -While `https://replayweb.page` can load from any URL, a site must allow access by enabling [CORS](https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS) -for the web archive. HTTP URLs may not be accessible from `https://replayweb.page` since it requires HTTPS. - -If you need to provide access to web archives stored on your own domain or over HTTP, consider -deploying a copy on your server. See: Running your Own. - -### Local Files - -Local files selected via the UI file chooser, and will be given a `file://` URL. -Unlike other web archive URLs, file URLs can not be shared with others. - - -### Amazon S3 - -The `s3://` URL scheme is provided as a convenience to reference S3 URLs. The URL is converted to an HTTPS URL for access. -The S3 Bucket must be public and also provide access to `https://replayweb.page` via CORS. -Downloading via S3 credentials is not currently supported. - - -### Google Drive - -The `googledrive://` URL scheme is used when accessing a web archive shared over Google Drive. It provides a unique identifier -provided via Google Drive. When accessing a Google Drive URL, the user will be asked to authenticate to be able to access the file. - - -### IPFS -{: .d-inline-block } - -Help Wanted -{: .label .label-yellow } - - -The `ipfs://` URL scheme is provide as a convenience and resolves to using the Cloudflare IPFS Gateway to download archives from IPFS. -A better implementation with IPFS directly is also possible. - - -### DAT/Hyper -{: .d-inline-block } - -Help Wanted -{: .label .label-yellow } - -Support for `hyper://` is planned to allow true peer-to-peer transfer of web archives. - diff --git a/docs/offline.md b/docs/offline.md deleted file mode 100644 index 8c61c42..0000000 --- a/docs/offline.md +++ /dev/null @@ -1,91 +0,0 @@ ---- -layout: default -title: Offline Use -nav_order: 2 -parent: Loading Web Archives -permalink: /docs/loading_offline ---- - -# Offline Use -{: .no_toc } - -## ReplayWeb.page is available for offline use in several ways. -{: .no_toc .text-delta } - - -1. -{:toc} - - -## ReplayWeb.page Offline Cache - -After ReplayWeb.page is loaded, it can automatically be used offline without requiring an Internet connection. - -That's to say, once you have loaded ReplayWeb.page online (with an Internet connection), if an Internet connection is not available later on, all web archives already loaded will continue to be available because it's cached. - -To do this, https://replayweb.page/ in your Chrome browser, and the site will start loading local archives even without an Internet connection. - - -
-
-
-
-## ReplayWeb.page as Progress Web App
-
-ReplayWeb.page can also be installed as [Progressive Web App](https://developer.mozilla.org/en-US/docs/Web/Progressive_web_apps) and be available as a standalone app on desktop and mobile.
-
-These type of apps use the same, current version of Chrome you normally use while searching or browsing the web. This is still experimental, but should provide the same behavior as regular https://replayweb.page/
-
-To do this, on the ReplayWeb.page site, you might have noticed a "new window icon" in your search bar. Clicking on this icon will open the progressive web app on your computer so you can use it offline.
-
-- What's cache? -
-- Cache is a temporary storage space in your browser that keeps certain files. Cached data is used to quickly load an application or website (like replayweb.page!) every time you log on. -
-
-
-
-- What's a Progressive Web App? -
-- Progressive Web Apps can be considered as a cross-platform app: part web and part native. PWAs are built on the web and has web features that make is faster and easier to discover and share. AND it also has native features that makes it better integrated with your operating system and allows you to install on your computer so that it works offline! -
-
-
-
-
-## ReplayWeb.page as Standalone Desktop (Electron) App
-
-ReplayWeb.page is also available as a standalone Electron App. In this mode, ReplayWeb.page packages its own version of Chrome.
-
-To do this, go to our [GitHub repo for replayweb.page](https://github.com/webrecorder/replayweb.page) and scroll down to "what's in this repo" for [app releases](https://github.com/webrecorder/replayweb.page/releases). You can download as a .dmg (for Apple) or .exe (for Windows).
-
-Although the Web Page/Progressive Web App versions have almost the same features, the standalone App is useful for two primary reasons:
-
-1) For viewing web archives that contain Flash. Flash is not supported by default in most browsers and will soon be removed. The ReplayWeb.page App will bundle a Flash plugin to continue to make Flash accessible for web archives.
-
-2) Primary use for viewing offline web archive. If your primary use is to view web archives stored locally,
-the ReplayWeb.page App is a better option, as it provides direct access to the file system.
-Once loaded, ReplayWeb.page App can continue to load web archives from disk.
-The desktop app fully replaces [Webrecorder Player](https://github.com/webrecorder/webrecorder-player)
-
-When using [https://replayweb.page](https://replayweb.page), you may be asked to re-open a local file because a web site can not have persistent access to your file system.
-
-
-
-
- - What's a standalone desktop (Electron) app? -
-- Electron is the framework being used for a native application. A standalone desktop app just means that you can download the application to your desktop and open it up whenever without Internet! -
-
-
--Once an archive has successfully loaded, you will be redirected to the archive view from where you can explore the archive. - -Next: [Exploring the Archive](/docs/exploring) diff --git a/docs/reference.md b/docs/reference.md deleted file mode 100644 index 4782415..0000000 --- a/docs/reference.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -layout: default -title: Reference -nav_order: 6 -has_children: true -permalink: /docs/reference ---- - -## Reference - -A guide to the file formats, specifications, source locations and other APIs to be supported by ReplayWeb.Page diff --git a/docs/sharing.md b/docs/sharing.md deleted file mode 100644 index 192f7ed..0000000 --- a/docs/sharing.md +++ /dev/null @@ -1,73 +0,0 @@ ---- -layout: default -title: Sharing Web Archives -nav_order: 3 -permalink: /docs/sharing ---- - -## Sharing Links - -ReplayWeb.page links are designed to be shareable with others, as long as they have access. - -You can share links to any view, including Page or Page Resources search, or Replay view. - -ReplayWeb.page encodes the web archive source and view parameters in the URL query and hash string. - -Simply copy the entire location URL, and you can share with others! - -ReplayWeb.page will parse the URL, load the source web archive, then load the view or replay from that archive. - -For example, given the following URL: - -``` -https://replayweb.page/?source=s3://webrecorder-builds/warcs/netpreserve-twitter.warc#view=replay&url=https%3A%2F%2Ftwitter.com%2Fnetpreserve&ts=20190603053135 -``` - -ReplayWeb.page will load the web archive from `s3://webrecorder-builds/warcs/netpreserve-twitter.warc`, switch to -the replay view via `https://twitter.com/netpreserve` at timestamp `20190603053135` - -The following URL will load the same web archive, and show the search results for Pages that contain the word 'WARC': - -``` -https://replayweb.page/?source=s3://webrecorder-builds/warcs/netpreserve-twitter.warc#view=pages&query=WARC -``` - -### No Local Files - -For obvious reasons, URLs that have a `source=file://...` can not be shared as they refer to a local file on your machine. -Loading that URL by anyone else will result in an error. - -{: .fs-3 .pad .bg-grey-lt-100} -A future goal is to provide an option to share local web archives peer-to-peer via the DAT `hyper://` protocol. - - -### Sharing Google Drive Archives - -An archive loaded from Google Drive can be shared with others, as long as they have access. - -Google Drive Archive will have a `source=googledrive://...`. - -Due to restrictions of Google Drive, you will likely be required to authorize with your Google Account to access an archive from Google Drive. - -Public files shared from Google Drive can sometimes be loaded without requiring authorization, but this is only supported -for small files. - -To share a Google Drive web archive with colleagues and friends: - -1. Ensure they you have [loaded the web archive from Google Drive](loading_online#loading-remote-archives-from-google-drive) - -2. Ensure that your friends/colleagues also have the [ReplayWeb.page Google Drive Integration](https://gsuite.google.com/u/2/marketplace/app/replaywebpage/160798412227) installed. - -3. Share the file with them via Google Drive. This is the most surefire way to work. - -OR. - -3. You can share the direct ReplayWeb.page URL directly. This does not always work, and you may need to share the Google Drive link instead. - - -In this way, web archives can be shared within private networks using existing access control settings in Google Drive. - -
-Next: Offline Usage Options - - diff --git a/docs/troubleshooting.md b/docs/troubleshooting.md deleted file mode 100644 index ddef3ed..0000000 --- a/docs/troubleshooting.md +++ /dev/null @@ -1,35 +0,0 @@ ---- -layout: page -title: Troubleshooting -nav_order: 5 -description: Troubleshooting ReplayWeb.page -permalink: /docs/troubleshooting - ---- - -This section will be dedicated to common errors and issues that we’ve encountered during web archiving. - -### WARC file is too big / too slow / fails to load. -#### Loading Large Files: Full vs On-Demand Loading - -Due to the nature of the WARC format, the entire file must be read on first use to generate an index. - -For WARC files **>25MB**, only the index is initially stored in the browser, and the actual content is loaded 'on-demand', -when the content is first accessed. This leads to faster loading and saves memory when dealing with large archives, but the entire -file must still be read. - -[Web Archive Collection (WACZ)](wacz-format) are always loaded on-demand, as no indexing is required. -The initial archive view should load almost instantly as a result. - -We recommend converting larger WARC files to WACZ files. - -This is currently possible by using the `wacz` python library, see [py-wacz documentation](https://github.com/webrecorder/py-wacz) -for more info. - -While we've recently made improvements to large WARC loading, for very large files, this may still take some time and occasionally the browser may not load the full file. - -If an archive could not be loaded, an error will be displayed instead of the progress. -Please report any issues encountered via [GitHub Issues](https://github.com/webrecorder/replayweb.page/issues) - -## Have another error or issue? -Since we just launched, we’ll need your help with what we should troubleshoot! Please contact us and ask us questions and show us examples. diff --git a/docs/usage.md b/docs/usage.md deleted file mode 100644 index a3a990c..0000000 --- a/docs/usage.md +++ /dev/null @@ -1,26 +0,0 @@ ---- -layout: default -title: Loading Web Archives -nav_order: 2 -description: How to use ReplayWeb.page -has_children: true -permalink: /docs/usage ---- - -# How to use ReplayWeb.Page -So you generated WARC files after capturing your website. Now what?
-At this point in your web archiving workflow, you'll most likely want to replay these WARC files. - -This is where replayweb.page can be helpful for you! You can view your web archives either from various locations directly in the browser or offline on your computer. - -## Online Use - -Currently the easiest way to view your WARC and/or WACZ files is online through replayweb.page. You can load web archives locally from your computer or from other remote web archives. Check out [Online Use](/docs/loading_online) for more information how to load your web archives. - -
-
-## Offline Use
-You can run replayweb.page on your computer by running it locally. And then you can load web archives the same way you would online. Check out [Offline Use](/docs/loading_offline) for more information how to load your web archives - offline.
-
-
-
diff --git a/docs/wacz-format.md b/docs/wacz-format.md
deleted file mode 100644
index b6d659b..0000000
--- a/docs/wacz-format.md
+++ /dev/null
@@ -1,26 +0,0 @@
----
-layout: default
-title: WACZ Format
-nav_order: 1
-permalink: /docs/wacz-format
-parent: Reference
----
-
-## Web Archive Collection Format Specification
-
-ReplayWeb.page supports a new format for bundling raw web archive data (usually WARC files), indices,
-page lists and other metadata into a single ZIP file.
-
-The full spec for this format is available at: [https://github.com/webrecorder/web-archive-collection-format/blob/master/README.md](https://github.com/webrecorder/web-archive-collection-format/blob/master/README.md)
-
-Files bundled into this format can use the .wacz (web archive collection zipped) file extension.
-
-ReplayWeb.page will recognize this extension (as well as regular .zip) and will also load it from Google Drive when the
-[Google Drive Integration](https://gsuite.google.com/u/2/marketplace/app/replaywebpage/160798412227) is installed.
-
-The key benefit of this format is that large web archive collections can be loaded very quickly, to show the page list
-and other key metadata, by downloading only parts of the WACZ file, as [outlined here](https://github.com/webrecorder/web-archive-collection-format/blob/master/README.md#appendix-a-use-case-random-access-to-web-archives-in-zip)
-
-The actual raw content is loaded on-demand when the user requests each page.
-
-With a WARC file, the entire contents must be loaded or indexed to determine the contents of the web archive collection.
\ No newline at end of file
diff --git a/mkdocs/main.py b/mkdocs/main.py
new file mode 100644
index 0000000..c6eb8c2
--- /dev/null
+++ b/mkdocs/main.py
@@ -0,0 +1,18 @@
+import json
+
+def define_env(env):
+ """
+ This is the hook for the functions (new form)
+ """
+ version = "LATEST"
+ try:
+ with open("../package.json") as fh:
+ version = json.loads(fh.read())["version"]
+ except:
+ pass
+
+ @env.macro
+ def rwp_version():
+ "get the version"
+ return version
+
diff --git a/mkdocs/mkdocs.yml b/mkdocs/mkdocs.yml
new file mode 100644
index 0000000..07e8d3e
--- /dev/null
+++ b/mkdocs/mkdocs.yml
@@ -0,0 +1,110 @@
+site_name: ReplayWeb.page Docs
+docs_dir: ../site/
+watch:
+ - ../site/
+site_dir: _genhtml/
+repo_url: https://github.com/webrecorder/replayweb.page
+repo_name: ReplayWeb.page
+edit_uri: edit/main/site/docs/
+extra_css:
+ - docs/stylesheets/extra.css
+extra_javascript:
+ - docs/js/sw.js
+theme:
+ name: material
+ custom_dir: ../site/docs/overrides
+ features:
+ - navigation.sections
+ - navigation.tabs
+ - navigation.tabs.sticky
+ - navigation.instant
+ - navigation.tracking
+ - navigation.indexes
+ - navigation.footer
+ - content.code.copy
+ - content.action.edit
+ - content.tooltips
+ - search.suggest
+ palette:
+ scheme: webrecorder
+ logo: docs/assets/brand/replaywebpage-icon-white.svg
+ favicon: docs/assets/brand/favicon.svg
+
+ icon:
+ admonition:
+ note: bootstrap/pencil-fill
+ abstract: bootstrap/file-earmark-text-fill
+ info: bootstrap/info-circle-fill
+ tip: bootstrap/exclamation-circle-fill
+ success: bootstrap/check-circle-fill
+ question: bootstrap/question-circle-fill
+ warning: bootstrap/exclamation-triangle-fill
+ failure: bootstrap/x-octagon-fill
+ danger: bootstrap/exclamation-diamond-fill
+ bug: bootstrap/bug-fill
+ example: bootstrap/mortarboard-fill
+ quote: bootstrap/quote
+
+ repo: bootstrap/github
+ edit: bootstrap/pencil
+ view: bootstrap/eye
+
+nav:
+ - docs/index.md
+ - Embedding:
+ - docs/embedding/index.md
+ - docs/embedding/cors-settings.md
+ - User Guide:
+ - docs/user-guide/index.md
+ - docs/user-guide/loading.md
+ - docs/user-guide/locations.md
+ - docs/user-guide/exploring.md
+ - docs/user-guide/sharing.md
+ - docs/user-guide/offline-use.md
+
+markdown_extensions:
+ - toc:
+ toc_depth: 3
+ permalink: true
+ - pymdownx.highlight:
+ anchor_linenums: true
+ - pymdownx.emoji:
+ emoji_index: !!python/name:material.extensions.emoji.twemoji
+ emoji_generator: !!python/name:material.extensions.emoji.to_svg
+ options:
+ custom_icons:
+ - ../site/docs/overrides/.icons
+ - admonition
+ - pymdownx.inlinehilite
+ - pymdownx.details
+ - pymdownx.superfences
+ - pymdownx.keys
+ - def_list
+ - attr_list
+
+extra:
+ generator: false
+ social:
+ - icon: bootstrap/globe
+ link: https://webrecorder.net
+ - icon: bootstrap/chat-left-text-fill
+ link: https://forum.webrecorder.net/
+ - icon: bootstrap/mastodon
+ link: https://digipres.club/@webrecorder
+ - icon: bootstrap/youtube
+ link: https://www.youtube.com/@webrecorder
+copyright: "Creative Commons Attribution 4.0 International (CC BY 4.0)"
+
+plugins:
+ - macros
+ - search
+ - redirects:
+ redirect_maps:
+ 'docs/cors-settings.md': 'docs/embedding/cors-settings.md'
+ 'docs/usage.md': 'docs/user-guide/index.md'
+ 'docs/loading_online.md': docs/user-guide/loading.md
+ 'docs/loading_offline.md': docs/user-guide/offline-use.md
+ 'docs/exploring.md': 'docs/user-guide/exploring.md'
+ 'docs/sharing.md': 'docs/user-guide/sharing.md'
+ 'docs/locations.md': 'docs/user-guide/locations.md'
+ 'docs/formats.md': 'docs/user-guide/index.md#supported-formats'
diff --git a/package.json b/package.json
index 321fe50..88e95af 100644
--- a/package.json
+++ b/package.json
@@ -73,17 +73,17 @@
"index.html"
],
"scripts": {
- "build": "webpack --mode production",
- "build-dev": "webpack --mode development",
- "build-docs": "bundle install; bundle exec jekyll build",
+ "build": "webpack --mode production && cp dist/ui.js site/ui.js",
+ "build-dev": "webpack --mode development && cp dist/ui.js site/ui.js",
+ "build-docs": "yarn run build && cd mkdocs && mkdocs build",
+ "start-prod": "cd site && http-server -p 9990 --cors",
"start-dev": "webpack serve --mode development",
- "start-prod": "http-server -p 9990 --cors",
+ "start-docs": "yarn run build-dev && cd mkdocs && mkdocs serve",
+ "start-electron": "NODE_ENV=development electron ./dist/electron.js $1",
"test-start-embed": "cd tests/embed; http-server -p 8020",
"test-start-sandbox": "cd tests/embed/sandbox; http-server -p 8030",
"pack": "CSC_IDENTITY_AUTO_DISCOVERY=false electron-builder --publish never",
"pack-signed": "electron-builder",
- "start-electron": "NODE_ENV=development electron ./dist/electron.js $1",
- "start-jekyll": "yarn run build-docs && bundle exec jekyll serve",
"dist": "yarn run build && yarn run pack",
"dist-dev": "yarn run build-dev && yarn run pack",
"release": "yarn run build && electron-builder",
diff --git a/plugins/.gitignore b/plugins/.gitignore
deleted file mode 100644
index 7950041..0000000
--- a/plugins/.gitignore
+++ /dev/null
@@ -1,4 +0,0 @@
-*
-!README
-!.gitignore
-
diff --git a/site/CNAME b/site/CNAME
new file mode 100644
index 0000000..21aec4a
--- /dev/null
+++ b/site/CNAME
@@ -0,0 +1 @@
+dev2.replayweb.page
diff --git a/site/docs/assets/brand/favicon.svg b/site/docs/assets/brand/favicon.svg
new file mode 100644
index 0000000..adf1e62
--- /dev/null
+++ b/site/docs/assets/brand/favicon.svg
@@ -0,0 +1,5 @@
+
diff --git a/site/docs/assets/brand/replaywebpage-icon-white.svg b/site/docs/assets/brand/replaywebpage-icon-white.svg
new file mode 100644
index 0000000..9bcce2c
--- /dev/null
+++ b/site/docs/assets/brand/replaywebpage-icon-white.svg
@@ -0,0 +1,10 @@
+
diff --git a/docs/assets/example.wacz b/site/docs/assets/example.wacz
similarity index 100%
rename from docs/assets/example.wacz
rename to site/docs/assets/example.wacz
diff --git a/site/docs/assets/fonts/Inter-Italic.var.woff2 b/site/docs/assets/fonts/Inter-Italic.var.woff2
new file mode 100644
index 0000000..40a2c1d
Binary files /dev/null and b/site/docs/assets/fonts/Inter-Italic.var.woff2 differ
diff --git a/site/docs/assets/fonts/Inter.var.woff2 b/site/docs/assets/fonts/Inter.var.woff2
new file mode 100644
index 0000000..0b74491
Binary files /dev/null and b/site/docs/assets/fonts/Inter.var.woff2 differ
diff --git a/site/docs/assets/fonts/Recursive_VF_1.084.woff2 b/site/docs/assets/fonts/Recursive_VF_1.084.woff2
new file mode 100644
index 0000000..59b07c0
Binary files /dev/null and b/site/docs/assets/fonts/Recursive_VF_1.084.woff2 differ
diff --git a/docs/assets/replaywebpage-googledrive-install.gif b/site/docs/assets/replaywebpage-googledrive-install.gif
similarity index 100%
rename from docs/assets/replaywebpage-googledrive-install.gif
rename to site/docs/assets/replaywebpage-googledrive-install.gif
diff --git a/docs/assets/replaywebpage-online-local.png b/site/docs/assets/replaywebpage-online-local.png
similarity index 100%
rename from docs/assets/replaywebpage-online-local.png
rename to site/docs/assets/replaywebpage-online-local.png
diff --git a/docs/assets/replaywebpage-online.gif b/site/docs/assets/replaywebpage-online.gif
similarity index 100%
rename from docs/assets/replaywebpage-online.gif
rename to site/docs/assets/replaywebpage-online.gif
diff --git a/docs/assets/tweet-example.wacz b/site/docs/assets/tweet-example.wacz
similarity index 100%
rename from docs/assets/tweet-example.wacz
rename to site/docs/assets/tweet-example.wacz
diff --git a/site/docs/embedding/cors-settings.md b/site/docs/embedding/cors-settings.md
new file mode 100644
index 0000000..b5a668f
--- /dev/null
+++ b/site/docs/embedding/cors-settings.md
@@ -0,0 +1,108 @@
+# Configuring CORS
+
+## Primer
+
+ReplayWeb.page can can load files from any web server, but works best when the server hosting the archived content can respond to `Range` requests. That being said, a common obstacle is configuring Cross-Origin Resource Sharing (CORS) settings to give ReplayWeb.page access to the data when content is being hosted from a different server than the webpage where it is embedded.
+
+Browsers typically restrict access to files hosted on a different domain than the website they are trying to load for security reasons.
+
+If you are loading archived items from a different domain (for example, S3 or other cloud storage), the site hosting the file needs to 'allow' the website to load the file using special CORS headers.
+
+These settings apply whenever archived items are hosted on a different domain than the viewer. If you are loading via the [https://replayweb.page](https://replayweb.page) site, you need to grant access to `https://replayweb.page` to load these items.
+
+The minimum requirement is to return: `Access-Control-Allow-Origin: https://replayweb.page` and most CORS settings include at least this option.
+
+## Differences Between Chrome, Firefox, and Safari
+
+Unfortunately, browsers have diverged in how they handle enforcing CORS policies.
+
+Firefox and Safari sometimes send a [pre-flight OPTIONS request](https://developer.mozilla.org/en-US/docs/Glossary/Preflight_request) when handling ReplayWeb.page requests, because the ReplayWeb.page uses the `Range` header.
+
+Chrome currently considers it a 'safe' header, but the other browsers have not yet made this change.
+
+For best compatibility with all browsers, we recommend including all headers as 'allowed headers' for use with ReplayWeb.page, eg.
+by setting `Access-Control-Allow-Headers: '*'` as part of the CORS response in addition to `Access-Control-Allow-Origin`.
+
+## CORS Configurations for Common Hosting Environments
+
+The following section provides suggested CORS configurations for different hosting environments. For more examples of CORS configurations for different servers, see: [enable-cors.org](https://enable-cors.org/).
+
+### S3 and Compatible Configurations
+
+When hosting with Amazon S3 or an S3-compatible service, these CORS bucket configurations _should_ work.
+
+Amazon S3 and other services now support a JSON based bucket policy, as well as an older XML-based policy.
+
+The below examples provide access to `https://replayweb.page` to load your archive. When [embedding replayweb.page](index.md), replace the URL with that of the server (origin) URL where the `#!html See [Loading Errors](#loading-errors) below for more info. + +??? Tip "Best Practice: Controlling ReplayWeb.page versions" + + The above example uses the version number "{{ rwp_version() }}" in paths to load a specific version of ReplayWeb.page from a CDN. These URLs point to a specific version of ReplayWeb.page software released on NPM, meaning that your replay viewer should stay stable, even if ReplayWeb.page is updated. + + - `https://cdn.jsdelivr.net/npm/replaywebpage@{{ rwp_version() }}/ui.js` + - `https://cdn.jsdelivr.net/npm/replaywebpage@{{ rwp_version() }}/sw.js` + + Although ReplayWeb.page strives to remain backwards compatible, this addresses any potential issue of older embeds breaking when the replay system is updated. + + You can also use the latest stable release by omitting the version, and the URL will automatically load the latest published release: + + - `https://cdn.jsdelivr.net/npm/replaywebpage/ui.js` + - `https://cdn.jsdelivr.net/npm/replaywebpage/sw.js` + + This is not recommended for production use as this is more likely to break in case there is a breaking change. + + For testing the very latest, it is possible to simply link to replayweb.page itself, eg. `https://replayweb.page/ui.js` + and `https://replayweb.page/sw.js` but these are updated the most frequently and more likely to break. + + For production use, it is recommended to use a fixed version and explicitly upgrade as necessary. + +## Self Hosting + +Sometimes it can be desirable to self-host the user interface, service worker, and WACZ files. This is useful for preventing tracking by CDNs, or to make it easier to host the content in one place without needing to work out the details of Cross Origin Resource Sharing (CORS). + +Following the same steps above, instead of loading the JavaScript and WACZ file from external locations, [download](https://www.jsdelivr.com/package/npm/replaywebpage) the `ui.js` and `sw.js` JavaScript files and put them on the same server as the HTML that you are publishing. Link them accordingly replacing the CDN links in the guide above. + +### Example + +If you are publishing a page at `https://example.com/path/my-web-archive-embed.html`, adjust the `#!html + +
+
+The archived item is loaded directly in the browser and is _not_ uploaded anywhere — your data never leaves your computer!
+
+Once loaded, the archived item will appear in the list of Loaded Archives and will be identified by its filename with a a `file://` URL as its source. This URL is not shareable as it doesn't exist outside of your computer.
+
+
+ 
+