Site reorg: Remove site/ subpath (#306)
Unfortunately, for proper package pathing, need to remove the `site/` subpath. This includes: - Moving everything back into root - Adding a `copy-site.sh` script to copy resources into `mkdocs/site/` before building the site with mkdocs. - Also update examples to ensure all are loading the local sw.js / ui.js - Consolidate examples in `docs/examples` - Consolidate assets under src/assets/ - Move `favicons` to root - Reformat and relint - Tests: update playwright tests to use mkdocs output, add test for embed in docs - bump to 2.0.0-beta.2
This commit is contained in:
Ilya Kreymer
@@ -0,0 +1,108 @@
|
||||
# Configuring CORS
|
||||
|
||||
## Primer
|
||||
|
||||
ReplayWeb.page can can load files from any web server, but works best when the server hosting the archived content can respond to `Range` requests. That being said, a common obstacle is configuring Cross-Origin Resource Sharing (CORS) settings to give ReplayWeb.page access to the data when content is being hosted from a different server than the webpage where it is embedded.
|
||||
|
||||
Browsers typically restrict access to files hosted on a different domain than the website they are trying to load for security reasons.
|
||||
|
||||
If you are loading archived items from a different domain (for example, S3 or other cloud storage), the site hosting the file needs to 'allow' the website to load the file using special CORS headers.
|
||||
|
||||
These settings apply whenever archived items are hosted on a different domain than the viewer. If you are loading via the [https://replayweb.page](https://replayweb.page) site, you need to grant access to `https://replayweb.page` to load these items.
|
||||
|
||||
The minimum requirement is to return: `Access-Control-Allow-Origin: https://replayweb.page` and most CORS settings include at least this option.
|
||||
|
||||
## Differences Between Chrome, Firefox, and Safari
|
||||
|
||||
Unfortunately, browsers have diverged in how they handle enforcing CORS policies.
|
||||
|
||||
Firefox and Safari sometimes send a [pre-flight OPTIONS request](https://developer.mozilla.org/en-US/docs/Glossary/Preflight_request) when handling ReplayWeb.page requests, because the ReplayWeb.page uses the `Range` header.
|
||||
|
||||
Chrome currently considers it a 'safe' header, but the other browsers have not yet made this change.
|
||||
|
||||
For best compatibility with all browsers, we recommend including all headers as 'allowed headers' for use with ReplayWeb.page, eg.
|
||||
by setting `Access-Control-Allow-Headers: '*'` as part of the CORS response in addition to `Access-Control-Allow-Origin`.
|
||||
|
||||
## CORS Configurations for Common Hosting Environments
|
||||
|
||||
The following section provides suggested CORS configurations for different hosting environments. For more examples of CORS configurations for different servers, see: [enable-cors.org](https://enable-cors.org/).
|
||||
|
||||
### S3 and Compatible Configurations
|
||||
|
||||
When hosting with Amazon S3 or an S3-compatible service, these CORS bucket configurations _should_ work.
|
||||
|
||||
Amazon S3 and other services now support a JSON based bucket policy, as well as an older XML-based policy.
|
||||
|
||||
The below examples provide access to `https://replayweb.page` to load your archive. When [embedding replayweb.page](index.md), replace the URL with that of the server (origin) URL where the `#!html <replay-web-page>` embed is hosted.
|
||||
|
||||
Both the JSON and XML options provide a method of specifying multiple origins via the JSON list or multiple `#!xml <AllowedOrigin>` tags.
|
||||
|
||||
#### JSON-Based Configuration
|
||||
|
||||
```json
|
||||
[
|
||||
{
|
||||
"AllowedOrigins": [
|
||||
"https://replayweb.page"
|
||||
],
|
||||
"AllowedMethods": [
|
||||
"GET",
|
||||
"HEAD"
|
||||
],
|
||||
"AllowedHeaders": [
|
||||
"*"
|
||||
]
|
||||
}
|
||||
]
|
||||
```
|
||||
|
||||
#### XML-Based Configuration
|
||||
|
||||
```xml
|
||||
<CORSConfiguration
|
||||
xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
|
||||
<CORSRule>
|
||||
<AllowedMethod>GET</AllowedMethod>
|
||||
<AllowedMethod>HEAD</AllowedMethod>
|
||||
<AllowedOrigin>https://replayweb.page</AllowedOrigin>
|
||||
<AllowedHeader>*</AllowedHeader>
|
||||
<ExposeHeader>Content-Range</ExposeHeader>
|
||||
<ExposeHeader>Content-Encoding</ExposeHeader>
|
||||
<ExposeHeader>Content-Length</ExposeHeader>
|
||||
</CORSRule>
|
||||
</CORSConfiguration>
|
||||
```
|
||||
|
||||
This policy can be applied using the [s3cmd](https://s3tools.org/usage) command-line tool:
|
||||
|
||||
1. Paste the above snippet into a file, eg. `cors.xml`
|
||||
2. Set the `#!xml <AllowedOrigin>` URL to the site hosting the embed. You can add as many of these as necessary.
|
||||
3. Run `s3cmd setcors ./cors.xml s3://<your-bucket>` to apply the settings from the XML file to the selected S3 bucket.
|
||||
|
||||
See the s3cmd docs for how to configure s3cmd to work with your setup.
|
||||
|
||||
See [S3 Docs](https://docs.aws.amazon.com/AmazonS3/latest/userguide/ManageCorsUsing.html) for more info on how to set this policy.
|
||||
|
||||
Other cloud providers may have a similar settings for configuring CORS.
|
||||
|
||||
### Manual Setup / Nginx
|
||||
|
||||
The recommended setup for nginx is to include the following CORS settings. Like the above, when [embedding replayweb.page](index.md), replace the URL with that of the server (origin) URL where the `#!html <replay-web-page>` embed is hosted.
|
||||
|
||||
```nginx
|
||||
location / {
|
||||
if ($request_method = 'OPTIONS') {
|
||||
add_header 'Access-Control-Allow-Origin' 'https://replayweb.page';
|
||||
add_header 'Access-Control-Allow-Methods' 'GET, HEAD, OPTIONS' always;
|
||||
add_header 'Access-Control-Allow-Headers' '*' ;
|
||||
return 204;
|
||||
}
|
||||
|
||||
if ($request_method = 'GET') {
|
||||
add_header 'Access-Control-Allow-Origin' 'https://replayweb.page' always;
|
||||
add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS' always;
|
||||
add_header 'Access-Control-Allow-Headers' '*' always;
|
||||
add_header 'Access-Control-Expose-Headers' 'Content-Length,Content-Range' always;
|
||||
}
|
||||
}
|
||||
```
|
||||
@@ -0,0 +1,179 @@
|
||||
# Embedding ReplayWeb.page
|
||||
|
||||
## Introduction
|
||||
|
||||
A key goal of ReplayWeb.page is to make embedding archived web content into other sites as easy as embedding other media like images and PDFs.
|
||||
|
||||
To make this possible ReplayWeb.page provides the `#!html <replay-web-page>` HTML [web component](https://developer.mozilla.org/en-US/docs/Web/API/Web_components) to support embedding in the pages where you would like to display web archives. This component works in all modern browsers, and has several configuration options that allow for control over the initial URL or snapshot to display from the archive when the component loads. The component can load WACZ files, WARC files and [other formats](../user-guide/index.md#supported-formats).
|
||||
|
||||
The `#!html <replay-web-page>` web component consists of a "backend" service worker, which emulates a web server, and a "frontend" UI, though in reality both are scripts running in the browser - there is no web server!
|
||||
|
||||
!!! tip "Tip: Serving web archives efficiently with WACZ files"
|
||||
While other web archive filetypes may require ReplayWeb.page to download them in their entirety before viewing, WACZ files allow the service worker to pull individual resources from the file as they are requested by the user. Full retrieval of the WACZ by ReplayWeb.page is *not* required as long as the server delivering the WACZ file supports HTTP range requests. This means that serving archived content from WACZ files is effectively as bandwidth efficient as any other web content!
|
||||
|
||||
## Example
|
||||
|
||||
### Loading the ReplayWeb.page UI (Frontend)
|
||||
|
||||
To embed a WACZ stored at `https://replayweb.page/docs/examples/tweet-example.wacz`, add the following `#!html <script>` tag to your HTML page to load the user interface from the jsDelivr CDN, and use the `#!html <replay-web-page>` component to point to the WACZ:
|
||||
|
||||
```html
|
||||
<script src="https://cdn.jsdelivr.net/npm/replaywebpage@{{ rwp_version() }}/ui.js"></script>
|
||||
|
||||
<replay-web-page source="https://replayweb.page/docs/examples/tweet-example.wacz"
|
||||
url="https://oembed.link/https://twitter.com/webrecorder_io/status/1565881026215219200"></replay-web-page>
|
||||
```
|
||||
|
||||
In this example, the `source` attribute is pointing to the location of the WACZ file (in this case published on AWS S3) and the `url` attribute is used to indicate what URL to display from the archived item after the component loads.
|
||||
|
||||
### Loading the Service Worker (Backend)
|
||||
|
||||
ReplayWeb.page's backend is a [service worker](https://developer.mozilla.org/en-US/docs/Web/API/Service_Worker_API/Using_Service_Workers) which intercepts requests for URLs and looks for them in your loaded archived item. Service workers are just JavaScript files and can be loaded from the CDN (Content Delivery Network), it is necessary to add a service worker path from where the web archive will be served to a local path. We use `/replay` in these examples, but it can be anything.
|
||||
Since the service worker takes over the rendering of this path, it should be a path that is otherwise non-existent (serves a 404) from the actual web server.
|
||||
|
||||
To do this, create a `/replay` subdirectory and in it, a new JavaScript file (`/replay/sw.js`) and copy the following to the file:
|
||||
|
||||
```javascript
|
||||
importScripts("https://cdn.jsdelivr.net/npm/replaywebpage@{{ rwp_version() }}/sw.js");
|
||||
```
|
||||
|
||||
If the HTML above was added to `https://example.com/path/my-web-archive-embed.html`
|
||||
then the `sw.js` should be added such that it is at: `https://example.com/path/replay/sw.js`.
|
||||
|
||||
This has the affect imports the actual server worker from the CDN, serving it on your site `https://example.com/replay/sw.js` and
|
||||
allowing it to render web archives on the `/replay/` path.
|
||||
|
||||
That's it! Loading `https://example.com/path/my-web-archive-embed.html` should now load the web archive.
|
||||
|
||||
Be sure to add width and height styles to the `#!html <replay-web-page>` tag as needed to scale the embed, and replace `https://replayweb.page/docs/examples/example.wacz` with any web archive hosted on your site.
|
||||
|
||||
??? Info "Info: Loading files from different servers"
|
||||
If the file is loaded from a different origin, your site must have Cross Origin Resource Sharing (CORS) access to download the web archive.
|
||||
<br>See [Loading Errors](#loading-errors) below for more info.
|
||||
|
||||
??? Tip "Best Practice: Controlling ReplayWeb.page versions"
|
||||
|
||||
The above example uses the version number "{{ rwp_version() }}" in paths to load a specific version of ReplayWeb.page from a CDN. These URLs point to a specific version of ReplayWeb.page software released on NPM, meaning that your replay viewer should stay stable, even if ReplayWeb.page is updated.
|
||||
|
||||
- `https://cdn.jsdelivr.net/npm/replaywebpage@{{ rwp_version() }}/ui.js`
|
||||
- `https://cdn.jsdelivr.net/npm/replaywebpage@{{ rwp_version() }}/sw.js`
|
||||
|
||||
Although ReplayWeb.page strives to remain backwards compatible, this addresses any potential issue of older embeds breaking when the replay system is updated.
|
||||
|
||||
You can also use the latest stable release by omitting the version, and the URL will automatically load the latest published release:
|
||||
|
||||
- `https://cdn.jsdelivr.net/npm/replaywebpage/ui.js`
|
||||
- `https://cdn.jsdelivr.net/npm/replaywebpage/sw.js`
|
||||
|
||||
This is not recommended for production use as this is more likely to break in case there is a breaking change.
|
||||
|
||||
For testing the very latest, it is possible to simply link to replayweb.page itself, eg. `https://replayweb.page/ui.js`
|
||||
and `https://replayweb.page/sw.js` but these are updated the most frequently and more likely to break.
|
||||
|
||||
For production use, it is recommended to use a fixed version and explicitly upgrade as necessary.
|
||||
|
||||
## Self Hosting
|
||||
|
||||
Sometimes it can be desirable to self-host the user interface, service worker, and WACZ files. This is useful for preventing tracking by CDNs, or to make it easier to host the content in one place without needing to work out the details of Cross Origin Resource Sharing (CORS).
|
||||
|
||||
Following the same steps above, instead of loading the JavaScript and WACZ file from external locations, [download](https://www.jsdelivr.com/package/npm/replaywebpage) the `ui.js` and `sw.js` JavaScript files and put them on the same server as the HTML that you are publishing. Link them accordingly replacing the CDN links in the guide above.
|
||||
|
||||
### Example
|
||||
|
||||
If you are publishing a page at `https://example.com/path/my-web-archive-embed.html`, adjust the `#!html <script>` element to load the ui from a location on your website:
|
||||
|
||||
```html
|
||||
<script src="ui.js"></script>
|
||||
|
||||
<replay-web-page source="example.wacz" url="https://webrecorder.net"></replay-web-page>
|
||||
```
|
||||
|
||||
The following URLs would then need to resolve correctly:
|
||||
|
||||
* https://example.com/path/my-web-archive-embed.html
|
||||
* https://example.com/path/example.wacz
|
||||
* https://example.com/path/ui.js
|
||||
* https://example.com/path/replay/sw.js
|
||||
|
||||
When embedding more than one web archive on your site it can be helpful to centralize the location of the frontend and backend JavaScript, and potentially WACZ files. The `#!html <replay-web-page>` component has a `replayBase` attribute that lets you define the location to load the `sw.js` service worker from. By default, `replayBase` is set to `./replay/` and so the service worker is loaded from `./replay/sw.js`.
|
||||
|
||||
For example, if you publish your JavaScript files at:
|
||||
|
||||
* https://example.com/js/ui.js
|
||||
* https://example.com/js/sw.js
|
||||
* https://example.com/wacz/example.wacz
|
||||
|
||||
Then you must adjust your HTML to reference the new resources:
|
||||
|
||||
```html
|
||||
<script src=""/js/ui.js""></script>
|
||||
|
||||
<replay-web-page replayBase="/js/" source="/wacz/example.wacz" url="https://webrecorder.net"></replay-web-page>
|
||||
```
|
||||
|
||||
## Embed Modes
|
||||
|
||||
ReplayWeb.page offers four different ways to embed the archived content, including with or without the navigation UI, and with an archival information dropdown. The embed mode can be set via the `embed` property:
|
||||
|
||||
- `default` or not set: Show the replay page and the location bar, allowing navigation to other pages and accessing the page list.
|
||||
|
||||
- `full`: Show the full replayweb.page UI and logo.
|
||||
|
||||
- `replayonly`: Show just the replayed page, and no additional UI or nav bar buttons. Useful for embedding a single page.
|
||||
|
||||
- `replay-with-info`: Show the `replayonly` mode, but also add an archive info dropdown, which shows an archival 'receipt' with provenance and verification information (new in 1.7.0)
|
||||
|
||||
??? example "Example: Embed with `replay-with-info` enabled"
|
||||
<script src="/ui.js"></script>
|
||||
<replay-web-page style="height: 600px" loading="eager" embed="replay-with-info" replaybase="/" source="/docs/examples/tweet-example.wacz"url="https://oembed.link/https://twitter.com/webrecorder_io/status/1565881026215219200"></replay-web-page>
|
||||
|
||||
## Embedding Options
|
||||
|
||||
The `#!html <replay-web-page>` tag is a web component that supports a number of additional attributes:
|
||||
|
||||
| Attribute | Description |
|
||||
|:-------------|:-----------------|
|
||||
| `source` | Source URL for the archived item. This should be one of the [supported formats](../user-guide/index.md#supported-formats) loaded from one of the [support locations](../user-guide/locations.md) and is required. |
|
||||
| `url` | The starting URL to load from the archive. If omitted, will start with the page list or URL search view. |
|
||||
| `ts` | The ISO 8601 timestamp of the starting URL to load. If omitted, the latest available version is used. |
|
||||
| `deepLink` | If set, ReplayWeb.page will modify the URL of the page to allow for 'deep linking' to exact pages in the embed. |
|
||||
| `updateFavicons`| If set, ReplayWeb.page will set the favicon of the page to the current webpage being viewed. Only supported in Chrome.|
|
||||
| `embed` | (`default` / `full` / `replayonly` / `replay-with-info` ) - See [Embed Modes](#embed-modes) above. |
|
||||
| `swName` | Service Worker filename (default: `sw.js`). Set if using different name, don't include path, only filename |
|
||||
| `replayBase` | Location of the service worker file (eg. sw.js), defaults to `./replay/` as mentioned above, but can be overridden. |
|
||||
| `coll` | Internal ID for this collection, usually generated automatically. |
|
||||
| `config` | Extra per collection config options (such as custom fuzzy matching rules, TODO add more info!) |
|
||||
| `sandbox` | If set, will iframe in `sandbox`. Provides extra isolation, but prevents PDFs from loading in an embed, and may result in links opening in new windows. |
|
||||
| `noWebWorker`| If set, will not use Web Worker for loading, only Service Worker. May be useful for certain loading edge cases. |
|
||||
| `noCache` | If set, will not cache any loaded content HTTP responses locally, always loading from original source. |
|
||||
| `hideOffscreen` | If set, will unload the embed when it is not visible and reload when scrolled into view. Useful if multiple embeds per-page to avoid loading all at once. |
|
||||
| `newWindowBase` | set base replay URL loaded when a page opens a new window, defaults to `https://replayweb.page/` if `deepLink` not enabled, otherwise, to current page with new link. |
|
||||
| `requireSubdomainIframe` | If set, will only load embed in an iframe loaded from a subdomain, for increased origin isolation. |
|
||||
| `loading="eager"` | If set, will load the entire WACZ file at once (regardless of size), and not attempt on-demand range request loading. |
|
||||
| `useRuffle` | If set, will enable include Ruffle Flash emulator. Must include the `ruffle/` directory in `replayBase`. |
|
||||
|
||||
## Further Examples
|
||||
|
||||
For an example of a site built around multiple archived items, take a look at our [example-webarchive repository](https://github.com/webrecorder/example-webarchive/) and [static website](https://webrecorder.github.io/example-webarchive) hosted on Github pages. It may give you ideas for how to integrate the ReplayWeb.page component into your site. You may also be interested in [Web Replay Gen](https://github.com/webrecorder/web-replay-gen), an 11ty based static site generator for showcasing multiple WACZ files.
|
||||
|
||||
## Common Issues
|
||||
|
||||
Below are some possible issues that you may encounter when embedding and possible workarounds.
|
||||
|
||||
### Embed is too small / doesn't fill page.
|
||||
|
||||
If the `#!HTML <replay-web-page>` tag is the only element on a page, and you want it to use the full window width and height, adding the following
|
||||
CSS should fix the issue:
|
||||
|
||||
```css
|
||||
html, body {
|
||||
width: 100%;
|
||||
height: 100%;
|
||||
}
|
||||
```
|
||||
|
||||
### Loading Errors
|
||||
|
||||
If you see errors related to loading archived items such as `TypeError: failed to load`, the issue may be a result of a CORS error.
|
||||
|
||||
See [CORS Settings](cors-settings.md) for more info on how to configure CORS for ReplayWeb.page
|
||||
Reference in New Issue
Block a user