Security
How WebFrost handles authentication, SSRF, rate limits and untrusted content, and how to report a vulnerability.
WebFrost is a self-hosted web archiving toolkit. It fetches arbitrary URLs supplied by its users and stores what comes back, so the two things it has to get right are who may ask it to fetch and where it may be made to connect. This page describes the controls that exist today, how to configure them, and what is still your job as the operator.
For the structured analysis (assets, trust boundaries, risks by component) see the threat model. For the steps to lock down a server, see Harden a deployment.
Authentication
Every /api/v1/* route requires authentication except POST /api/v1/auth/login, POST /api/v1/auth/logout,
GET /api/v1/setup/status and POST /api/v1/setup (which only works while no admin user exists, and
only with WEBFROST_SETUP_TOKEN when that is set). While no admin exists and no setup token is
configured, the server logs a warning at startup: anyone who reaches it can claim it.
| Mechanism | Details |
|---|---|
| Admin password | bcrypt (cost 10) via Bun.password. 8 to 1024 characters (longer ones are refused before hashing). A failed login takes the same time whether or not the username exists. |
| Session cookie | Named session; HttpOnly, SameSite=Strict, Secure (unless WEBFROST_INSECURE_COOKIES=true). HMAC-SHA256 signed with SESSION_SECRET and backed by a row in the sessions table. Expires after 7 days. |
| API keys | 256-bit random, shown once, stored as SHA-256 hashes. Optional expiry; revocable. Sent as Authorization: Bearer <key>. |
Sessions are revocable:
- Logout deletes the server-side session, so a copied cookie stops working immediately. It clears the cookie even when the session has already expired.
- Logging in again revokes the session in the cookie being replaced.
- Changing the password deletes every session of that user and issues a fresh one to the caller. Use it if you suspect a cookie leaked.
SESSION_SECRET should be at least 32 random characters: openssl rand -hex 32. The Docker
image refuses to start without it (a shorter one starts with a warning). webfrost serve
generates one on first run when it is unset and keeps it in <data dir>/session-secret
(mode 0600). Rotating it invalidates every session.
AUTH_DISABLED=true turns all of the above off. It exists for local development and automated
tests. Never set it on anything reachable from a network you do not fully control. With it the
server binds to 127.0.0.1 unless HOST is set explicitly.
Cross-site requests (CSRF)
SameSite=Strict keeps the cookie off requests started by other sites, but not by other
origins on the same site (a sibling subdomain). So every state-changing request (POST, PUT,
PATCH, DELETE under /api/v1) that does not carry a bearer token is checked before its body is
parsed:
Sec-Fetch-Sitemust besame-originornone;same-siteandcross-siteget403.- A browser that sends no
Sec-Fetch-Sitemust send anOriginequal to theHost. - A request with neither header did not come from a browser and cannot ride on a cookie, so it passes (scripts, the SDK in Node or Bun).
On top of that, every body must be application/json (multipart only for
POST /api/v1/import/file), otherwise 415: an HTML form cannot send JSON. Bearer-token requests
skip the origin check, since a form cannot set an Authorization header.
Public read and private write
WEBFROST_PUBLIC_READ=true lets anonymous visitors read the archive: list, search and open
snapshots, their tags and archived files, and the stats card. Nothing else changes: every write,
and the schedules, imports, settings and API-key routes, still require a session or an API key.
The allowlist is explicit and matched on method and exact path, so a new route is private unless
it is added to it.
No API response includes storage paths (local file paths or S3 keys), and for anonymous requests failed
extractors show extraction failed instead of the raw error, which can name internal hosts and
paths. List pages are capped at 200 items for everyone. See Public read for the exact route list. Remember that
everything you have archived becomes visible to anyone who can reach the server — including
pages that were themselves private when you captured them.
Cookies over plain HTTP
Browsers drop Secure cookies on plain-HTTP origins other than localhost, so logging in through
http://192.168.1.10:3000 appears to succeed and then bounces back to the login page. Put the
server behind TLS, or — for an HTTP-only LAN deployment you accept the risk of — set
WEBFROST_INSECURE_COOKIES=true. (The flag is also dropped when NODE_ENV=development.)
Rate limiting
In-memory, sliding one-minute window, per client IP:
| Scope | Limit |
|---|---|
POST /api/v1/auth/login, /api/v1/auth/change-password, /api/v1/setup | 10 / min |
POST /api/v1/snapshots, /api/v1/snapshots/:id/retry, /api/v1/snapshots/import, /api/v1/import, /api/v1/import/file | 60 / min |
GET / HEAD on /api/v1/* | 300 / min |
A limited request gets 429 with a Retry-After header.
The client is identified by its TCP peer address. X-Forwarded-For is ignored by default because
any client can set it. Behind a reverse proxy every request would then share the proxy's address,
so set WEBFROST_TRUST_PROXY=true there. Proxies append the address they saw to whatever the
client sent, so the left-most entry is the client's choice; the server takes the entry
WEBFROST_TRUSTED_PROXY_HOPS (default 1, one proxy) places from the right instead. IPv6 clients
share a limit per /64, since one host usually controls the whole prefix.
Background pipelines are bounded separately by WEBFROST_CONCURRENCY (default 3), so the write
limit cannot translate into dozens of simultaneous headless browsers.
With public read enabled, anonymous visitors share the same read limit per client address.
SSRF protection
Anyone who can create a snapshot can make the server issue HTTP requests. WebFrost restricts where those requests may go.
What is blocked (for both IPv4 and IPv6, including IPv4-mapped and NAT64 forms): loopback,
RFC 1918 private ranges, carrier-grade NAT (100.64.0.0/10), link-local (169.254.0.0/16, which
covers cloud metadata endpoints), unique-local and site-local IPv6, multicast and reserved ranges.
Only http: and https: URLs are accepted.
Where it is enforced. See also how the browser guard works in the pipeline.
| Path | Enforcement |
|---|---|
| Initial page fetch, RSS/OPML feed fetch, SingleFile asset inlining | safeFetch: resolves the host, rejects blocked addresses, follows redirects manually and re-checks every hop, caps the body at 50 MB, times out after 30 s. |
| Headless Chromium (PDF, screenshot, DOM snapshot, SingleFile) | Every request the page makes — navigation, iframes, images, scripts, fetch/XHR, WebSockets — is checked and aborted if the host is blocked. Service workers are disabled. |
| Redirects inside the browser | Chromium does not allow intercepting a redirect hop. If any hop lands on a blocked host the browser extractors are not run for that snapshot and are recorded as failed, so the response is never captured. The request itself may still have been sent (blind SSRF). |
| After the network settles | Once the page has loaded and its network has gone quiet (up to WEBFROST_SETTLE_MS), every further request is refused before anything is captured, so a script cannot wait out the check and then pull an internal URL into a frame. Every request made before that is still checked. |
Opt-out: WEBFROST_ALLOW_PRIVATE_URLS=true disables the address check everywhere, for
deployments that intentionally archive intranet pages. The http(s)-only rule still applies.
Not an archived URL: webfrost push uploads to the server the user names with --to, usually
their own server on a private address, so it uses a plain fetch rather than safeFetch. It sends
the API key only there: redirects are not followed, and the key is read from WEBFROST_API_KEY,
never from the command line. On the receiving side the import fetches nothing.
Not user-controlled: the optional OpenTelemetry export sends to
the OTEL_EXPORTER_OTLP_*ENDPOINT the operator set in the environment, usually a collector on a
private address, so it does not go through the guard. No request, page or user input can change
where it sends, and WebFrost never adds trace headers to the pages it archives.
Residual risks
- DNS rebinding. The guard resolves a hostname and the HTTP client then resolves it again. An attacker controlling a domain's DNS with a very short TTL can return a public address to the first lookup and a private one to the second.
- Blind requests through browser redirects, as described above.
- yt-dlp makes its own requests and follows its own redirects. It is only invoked for a URL
that already passed
safeFetch, and it is limited as described in Media extraction, but its requests do not go through the guard.
If WebFrost archives URLs from people you do not fully trust, treat the application-level guard as
one layer and add a network-level one: run the container in a network with no route to your
internal ranges and to 169.254.169.254, or apply egress firewall rules.
Media extraction (yt-dlp)
The archived URL is untrusted input to yt-dlp, so every call passes:
--ignore-config: no config file on the host or in$HOMEcan inject options such as--execor change output paths;--no-playlist --playlist-items 1: a playlist or channel URL cannot fan out into hundreds of downloads;--use-extractors default,-generic: only sites yt-dlp explicitly supports. The generic extractor would fetch and scan arbitrary pages for embedded media, a second request outside the SSRF guard.
The URL always comes after --, so it can never be read as an option. yt-dlp runs in its own
process group with a hard timeout (WEBFROST_MEDIA_TIMEOUT_MS); on timeout, cancellation or
shutdown the whole group is killed, ffmpeg included, so no download keeps running outside the
concurrency limit. Its network requests still do not go through WebFrost's guard: isolate the
network at the container or firewall level when archiving untrusted URLs.
Content security
The server sets these headers on every response:
Content-Security-Policy: default-src 'none'; script-src 'self'; style-src 'self' 'unsafe-inline';
img-src 'self' data:; media-src 'self'; connect-src 'self'; frame-src 'self';
frame-ancestors 'none'; base-uri 'self'; form-action 'self'
X-Content-Type-Options: nosniff
X-Frame-Options: DENY
Referrer-Policy: strict-origin-when-cross-originWith the S3 backend, downloads redirect to presigned URLs on the bucket's host, so that one origin
is added to img-src and media-src (and to nothing else). The presigned URL signs the content
type and Content-Disposition: attachment into the response, so an archived HTML page downloads
instead of rendering, with its scripts, on the bucket's origin.
These headers are set on every response, including errors, 404s and requests refused before
they reach a route. Errors are always JSON { error, code }; an unknown /api/v1/* path is a JSON
404, never the single-page app.
Archived content is untrusted:
-
Archived files (HTML, DOM snapshot, SingleFile, PDF, WARC, media) are downloaded with
Content-Disposition: attachment. -
The in-app preview of a DOM snapshot or SingleFile archive is served by a separate endpoint (
GET /api/v1/snapshots/:id/archives/:extractor/preview) with its own policy:Content-Security-Policy: sandbox; default-src 'none'; style-src 'unsafe-inline'; img-src data:; font-src data:; media-src data:; frame-ancestors 'self'sandboxgives the document an opaque origin and disables scripts, forms, popups and top-level navigation, even when the URL is opened directly. Nothing can be loaded from the network, so a previewed page cannot phone home. The UI additionally frames it with<iframe sandbox="">.frame-src 'self'in the app policy exists only for this iframe. -
Search snippets are built from archived text. The server HTML-escapes the text and adds only
<mark>highlights; the UI additionally renders them as text nodes, never as HTML. -
The Reader view renders extracted article text as plain text.
Any new feature that displays archived content must go through the same sandboxed preview —
never add allow-scripts or allow-same-origin to that iframe, and never serve archived HTML
inline from a route that carries the app's own policy.
Input validation
- URLs:
http:/https:only, on the server and in the CLI. GET /api/v1/snapshots/:id/archives/:extractor/download:extractoris checked against an allowlist, and the stored file reference is validated against the storage backend before any read (local path must be insidesnapshots/; S3 keys must be under the configured prefix).- Snapshot bundles (
POST /api/v1/snapshots/import): see below. - All SQL uses bound parameters. Full-text queries are passed to FTS5 as a bound parameter; a syntactically invalid query returns no results.
Snapshot bundles
POST /api/v1/snapshots/import takes a ZIP of files and metadata, which is the riskiest input the
server accepts. Only authenticated admins can send one, and it is still treated as hostile:
- The upload is streamed to a fresh
0700directory underWEBFROST_DIR/tmp, counted, and cut off pastWEBFROST_IMPORT_BUNDLE_MAX_BYTES; it is removed when the request ends. - The ZIP is read from its central directory by a small dedicated reader that refuses encryption, ZIP64, multi-disk archives, methods other than stored/deflate, local headers that disagree with the directory, overlapping entries, duplicate names and more than 64 entries.
manifest.jsonis validated strictly (format version, no unknown fields,http(s)URL, ISO 8601 timestamps, tag names). Extractor names are checked against the known list.- File names are not taken from the bundle: each file-backed extractor has fixed entry names
(
screenshot.png,media.mp4...), the server maps them to storage names itself, and every entry must be referenced by the manifest. Paths,..and unreferenced files are refused. - Declared sizes are checked against the limit (per file and in total) before anything is written. While an entry is decompressed its output is counted and its CRC-32 is checked. Compressed data is fed to the inflater 4 KiB at a time, and deflate expands at most about 1000-fold, so an entry whose data outgrows its declared size is stopped within about 4 MiB of output, before that output is kept or written. A zip bomb never gets past its declared size plus that margin.
- The central directory is read only when its size is plausible for the number of entries (at most 64), so a forged size cannot make the server read a huge directory into memory.
- Content types are derived by the server from the extractor, as for every snapshot. Imported HTML
(
dom-snapshot,singlefile) is rendered only by the sandboxed preview, and downloads are attachments. - Nothing is fetched during an import and no pipeline runs.
At most two imports run at once (a third gets 429), and an upload is closed after 255 seconds
without data, so stalled uploads cannot hold the slots. While its files are copied an
import's snapshot is importing: hidden from the API, and it cannot be retried or deleted. The
server removes spool folders and importing snapshots a crash left behind at startup, once they are
an hour old.
A rejected bundle answers 400 and stores nothing; a failure after the checks removes what was
written.
The listener's body limit is raised to the bundle limit for this route; every other route is held
to the usual limit by the request guard before its body is read, and must declare a
Content-Length (a chunked body is refused with 411).
Resource limits
| Limit | Default | Setting |
|---|---|---|
| Parallel pipelines | 3 | WEBFROST_CONCURRENCY |
| HTTP fetch timeout | 30 s | WEBFROST_FETCH_TIMEOUT_MS |
| HTTP response size | 50 MB | — |
| Browser navigation timeout | 30 s | — |
| Per-extractor timeout | 60 s | WEBFROST_EXTRACTOR_TIMEOUT_MS |
| yt-dlp process timeout | 10 min | WEBFROST_MEDIA_TIMEOUT_MS |
| Browser shutdown | 5 s | — |
Import size (file, content or fetched url) | 10 MiB | WEBFROST_IMPORT_MAX_BYTES |
| URLs per import | 10 000 | WEBFROST_IMPORT_MAX_URLS |
| Snapshot bundle (uploaded and decompressed) | 2 GiB | WEBFROST_IMPORT_BUNDLE_MAX_BYTES |
| List page size | 200 | — |
| Password / username / tag and key name / URL length | 1024 / 128 / 100 / 8192 | — |
| Graceful shutdown wait for running jobs | 20 s | WEBFROST_SHUTDOWN_TIMEOUT_MS |
| Media file size | 500 MB | — |
| Gap between archives of one host | 1 s | WEBFROST_DOMAIN_DELAY_MS |
Running the server
The published image runs as the unprivileged user webfrost (UID 1000) and launches Chromium
without its own sandbox (Playwright's default). The container boundary is the sandbox: do not add
capabilities or mount the host filesystem.
The operator's checklist (secrets, setup token, TLS and proxy headers, egress, file permissions, backups) is a how-to of its own: Harden a deployment.
Dependencies
Run bun audit periodically. The dependencies that parse untrusted input are jsdom and
@mozilla/readability (article extraction), playwright + Chromium (rendering) and yt-dlp
(media). Keep the image up to date to pick up Chromium and yt-dlp fixes.
Reporting vulnerabilities
Please report vulnerabilities privately rather than in a public issue: use Security → Report a vulnerability on the GitHub repository. Include the affected version or commit, reproduction steps and the impact you observed. Allow up to 90 days for a fix before public disclosure.