Skip to Content
WebFrostdocs
Architecture

Security

How WebFrost handles authentication, SSRF, rate limits and untrusted content, and how to report a vulnerability.

WebFrost is a self-hosted web archiving toolkit. It fetches arbitrary URLs supplied by its users and stores what comes back, so the two things it has to get right are who may ask it to fetch and where it may be made to connect. This page describes the controls that exist today, how to configure them, and what is still your job as the operator.

For the structured analysis (assets, trust boundaries, risks by component) see the threat model. For the steps to lock down a server, see Harden a deployment.

Authentication

Every /api/v1/* route requires authentication except POST /api/v1/auth/login, POST /api/v1/auth/logout, GET /api/v1/setup/status and POST /api/v1/setup (which only works while no admin user exists, and only with WEBFROST_SETUP_TOKEN when that is set). While no admin exists and no setup token is configured, the server logs a warning at startup: anyone who reaches it can claim it.

MechanismDetails
Admin passwordbcrypt (cost 10) via Bun.password. 8 to 1024 characters (longer ones are refused before hashing). A failed login takes the same time whether or not the username exists.
Session cookieNamed session; HttpOnly, SameSite=Strict, Secure (unless WEBFROST_INSECURE_COOKIES=true). HMAC-SHA256 signed with SESSION_SECRET and backed by a row in the sessions table. Expires after 7 days.
API keys256-bit random, shown once, stored as SHA-256 hashes. Optional expiry; revocable. Sent as Authorization: Bearer <key>.

Sessions are revocable:

  • Logout deletes the server-side session, so a copied cookie stops working immediately. It clears the cookie even when the session has already expired.
  • Logging in again revokes the session in the cookie being replaced.
  • Changing the password deletes every session of that user and issues a fresh one to the caller. Use it if you suspect a cookie leaked.

SESSION_SECRET should be at least 32 random characters: openssl rand -hex 32. The Docker image refuses to start without it (a shorter one starts with a warning). webfrost serve generates one on first run when it is unset and keeps it in <data dir>/session-secret (mode 0600). Rotating it invalidates every session.

AUTH_DISABLED=true turns all of the above off. It exists for local development and automated tests. Never set it on anything reachable from a network you do not fully control. With it the server binds to 127.0.0.1 unless HOST is set explicitly.

Cross-site requests (CSRF)

SameSite=Strict keeps the cookie off requests started by other sites, but not by other origins on the same site (a sibling subdomain). So every state-changing request (POST, PUT, PATCH, DELETE under /api/v1) that does not carry a bearer token is checked before its body is parsed:

  • Sec-Fetch-Site must be same-origin or none; same-site and cross-site get 403.
  • A browser that sends no Sec-Fetch-Site must send an Origin equal to the Host.
  • A request with neither header did not come from a browser and cannot ride on a cookie, so it passes (scripts, the SDK in Node or Bun).

On top of that, every body must be application/json (multipart only for POST /api/v1/import/file), otherwise 415: an HTML form cannot send JSON. Bearer-token requests skip the origin check, since a form cannot set an Authorization header.

Public read and private write

WEBFROST_PUBLIC_READ=true lets anonymous visitors read the archive: list, search and open snapshots, their tags and archived files, and the stats card. Nothing else changes: every write, and the schedules, imports, settings and API-key routes, still require a session or an API key. The allowlist is explicit and matched on method and exact path, so a new route is private unless it is added to it.

No API response includes storage paths (local file paths or S3 keys), and for anonymous requests failed extractors show extraction failed instead of the raw error, which can name internal hosts and paths. List pages are capped at 200 items for everyone. See Public read for the exact route list. Remember that everything you have archived becomes visible to anyone who can reach the server — including pages that were themselves private when you captured them.

Cookies over plain HTTP

Browsers drop Secure cookies on plain-HTTP origins other than localhost, so logging in through http://192.168.1.10:3000 appears to succeed and then bounces back to the login page. Put the server behind TLS, or — for an HTTP-only LAN deployment you accept the risk of — set WEBFROST_INSECURE_COOKIES=true. (The flag is also dropped when NODE_ENV=development.)

Rate limiting

In-memory, sliding one-minute window, per client IP:

ScopeLimit
POST /api/v1/auth/login, /api/v1/auth/change-password, /api/v1/setup10 / min
POST /api/v1/snapshots, /api/v1/snapshots/:id/retry, /api/v1/snapshots/import, /api/v1/import, /api/v1/import/file60 / min
GET / HEAD on /api/v1/*300 / min

A limited request gets 429 with a Retry-After header.

The client is identified by its TCP peer address. X-Forwarded-For is ignored by default because any client can set it. Behind a reverse proxy every request would then share the proxy's address, so set WEBFROST_TRUST_PROXY=true there. Proxies append the address they saw to whatever the client sent, so the left-most entry is the client's choice; the server takes the entry WEBFROST_TRUSTED_PROXY_HOPS (default 1, one proxy) places from the right instead. IPv6 clients share a limit per /64, since one host usually controls the whole prefix.

Background pipelines are bounded separately by WEBFROST_CONCURRENCY (default 3), so the write limit cannot translate into dozens of simultaneous headless browsers.

With public read enabled, anonymous visitors share the same read limit per client address.

SSRF protection

Anyone who can create a snapshot can make the server issue HTTP requests. WebFrost restricts where those requests may go.

What is blocked (for both IPv4 and IPv6, including IPv4-mapped and NAT64 forms): loopback, RFC 1918 private ranges, carrier-grade NAT (100.64.0.0/10), link-local (169.254.0.0/16, which covers cloud metadata endpoints), unique-local and site-local IPv6, multicast and reserved ranges. Only http: and https: URLs are accepted.

Where it is enforced. See also how the browser guard works in the pipeline.

PathEnforcement
Initial page fetch, RSS/OPML feed fetch, SingleFile asset inliningsafeFetch: resolves the host, rejects blocked addresses, follows redirects manually and re-checks every hop, caps the body at 50 MB, times out after 30 s.
Headless Chromium (PDF, screenshot, DOM snapshot, SingleFile)Every request the page makes — navigation, iframes, images, scripts, fetch/XHR, WebSockets — is checked and aborted if the host is blocked. Service workers are disabled.
Redirects inside the browserChromium does not allow intercepting a redirect hop. If any hop lands on a blocked host the browser extractors are not run for that snapshot and are recorded as failed, so the response is never captured. The request itself may still have been sent (blind SSRF).
After the network settlesOnce the page has loaded and its network has gone quiet (up to WEBFROST_SETTLE_MS), every further request is refused before anything is captured, so a script cannot wait out the check and then pull an internal URL into a frame. Every request made before that is still checked.

Opt-out: WEBFROST_ALLOW_PRIVATE_URLS=true disables the address check everywhere, for deployments that intentionally archive intranet pages. The http(s)-only rule still applies.

Not an archived URL: webfrost push uploads to the server the user names with --to, usually their own server on a private address, so it uses a plain fetch rather than safeFetch. It sends the API key only there: redirects are not followed, and the key is read from WEBFROST_API_KEY, never from the command line. On the receiving side the import fetches nothing.

Not user-controlled: the optional OpenTelemetry export sends to the OTEL_EXPORTER_OTLP_*ENDPOINT the operator set in the environment, usually a collector on a private address, so it does not go through the guard. No request, page or user input can change where it sends, and WebFrost never adds trace headers to the pages it archives.

Residual risks

  • DNS rebinding. The guard resolves a hostname and the HTTP client then resolves it again. An attacker controlling a domain's DNS with a very short TTL can return a public address to the first lookup and a private one to the second.
  • Blind requests through browser redirects, as described above.
  • yt-dlp makes its own requests and follows its own redirects. It is only invoked for a URL that already passed safeFetch, and it is limited as described in Media extraction, but its requests do not go through the guard.

If WebFrost archives URLs from people you do not fully trust, treat the application-level guard as one layer and add a network-level one: run the container in a network with no route to your internal ranges and to 169.254.169.254, or apply egress firewall rules.

Media extraction (yt-dlp)

The archived URL is untrusted input to yt-dlp, so every call passes:

  • --ignore-config: no config file on the host or in $HOME can inject options such as --exec or change output paths;
  • --no-playlist --playlist-items 1: a playlist or channel URL cannot fan out into hundreds of downloads;
  • --use-extractors default,-generic: only sites yt-dlp explicitly supports. The generic extractor would fetch and scan arbitrary pages for embedded media, a second request outside the SSRF guard.

The URL always comes after --, so it can never be read as an option. yt-dlp runs in its own process group with a hard timeout (WEBFROST_MEDIA_TIMEOUT_MS); on timeout, cancellation or shutdown the whole group is killed, ffmpeg included, so no download keeps running outside the concurrency limit. Its network requests still do not go through WebFrost's guard: isolate the network at the container or firewall level when archiving untrusted URLs.

Content security

The server sets these headers on every response:

Content-Security-Policy: default-src 'none'; script-src 'self'; style-src 'self' 'unsafe-inline';
                         img-src 'self' data:; media-src 'self'; connect-src 'self'; frame-src 'self';
                         frame-ancestors 'none'; base-uri 'self'; form-action 'self'
X-Content-Type-Options: nosniff
X-Frame-Options: DENY
Referrer-Policy: strict-origin-when-cross-origin

With the S3 backend, downloads redirect to presigned URLs on the bucket's host, so that one origin is added to img-src and media-src (and to nothing else). The presigned URL signs the content type and Content-Disposition: attachment into the response, so an archived HTML page downloads instead of rendering, with its scripts, on the bucket's origin.

These headers are set on every response, including errors, 404s and requests refused before they reach a route. Errors are always JSON { error, code }; an unknown /api/v1/* path is a JSON 404, never the single-page app.

Archived content is untrusted:

  • Archived files (HTML, DOM snapshot, SingleFile, PDF, WARC, media) are downloaded with Content-Disposition: attachment.

  • The in-app preview of a DOM snapshot or SingleFile archive is served by a separate endpoint (GET /api/v1/snapshots/:id/archives/:extractor/preview) with its own policy:

    Content-Security-Policy: sandbox; default-src 'none'; style-src 'unsafe-inline';
                             img-src data:; font-src data:; media-src data:; frame-ancestors 'self'

    sandbox gives the document an opaque origin and disables scripts, forms, popups and top-level navigation, even when the URL is opened directly. Nothing can be loaded from the network, so a previewed page cannot phone home. The UI additionally frames it with <iframe sandbox="">. frame-src 'self' in the app policy exists only for this iframe.

  • Search snippets are built from archived text. The server HTML-escapes the text and adds only <mark> highlights; the UI additionally renders them as text nodes, never as HTML.

  • The Reader view renders extracted article text as plain text.

Any new feature that displays archived content must go through the same sandboxed preview — never add allow-scripts or allow-same-origin to that iframe, and never serve archived HTML inline from a route that carries the app's own policy.

Input validation

  • URLs: http:/https: only, on the server and in the CLI.
  • GET /api/v1/snapshots/:id/archives/:extractor/download: extractor is checked against an allowlist, and the stored file reference is validated against the storage backend before any read (local path must be inside snapshots/; S3 keys must be under the configured prefix).
  • Snapshot bundles (POST /api/v1/snapshots/import): see below.
  • All SQL uses bound parameters. Full-text queries are passed to FTS5 as a bound parameter; a syntactically invalid query returns no results.

Snapshot bundles

POST /api/v1/snapshots/import takes a ZIP of files and metadata, which is the riskiest input the server accepts. Only authenticated admins can send one, and it is still treated as hostile:

  • The upload is streamed to a fresh 0700 directory under WEBFROST_DIR/tmp, counted, and cut off past WEBFROST_IMPORT_BUNDLE_MAX_BYTES; it is removed when the request ends.
  • The ZIP is read from its central directory by a small dedicated reader that refuses encryption, ZIP64, multi-disk archives, methods other than stored/deflate, local headers that disagree with the directory, overlapping entries, duplicate names and more than 64 entries.
  • manifest.json is validated strictly (format version, no unknown fields, http(s) URL, ISO 8601 timestamps, tag names). Extractor names are checked against the known list.
  • File names are not taken from the bundle: each file-backed extractor has fixed entry names (screenshot.png, media.mp4...), the server maps them to storage names itself, and every entry must be referenced by the manifest. Paths, .. and unreferenced files are refused.
  • Declared sizes are checked against the limit (per file and in total) before anything is written. While an entry is decompressed its output is counted and its CRC-32 is checked. Compressed data is fed to the inflater 4 KiB at a time, and deflate expands at most about 1000-fold, so an entry whose data outgrows its declared size is stopped within about 4 MiB of output, before that output is kept or written. A zip bomb never gets past its declared size plus that margin.
  • The central directory is read only when its size is plausible for the number of entries (at most 64), so a forged size cannot make the server read a huge directory into memory.
  • Content types are derived by the server from the extractor, as for every snapshot. Imported HTML (dom-snapshot, singlefile) is rendered only by the sandboxed preview, and downloads are attachments.
  • Nothing is fetched during an import and no pipeline runs.

At most two imports run at once (a third gets 429), and an upload is closed after 255 seconds without data, so stalled uploads cannot hold the slots. While its files are copied an import's snapshot is importing: hidden from the API, and it cannot be retried or deleted. The server removes spool folders and importing snapshots a crash left behind at startup, once they are an hour old.

A rejected bundle answers 400 and stores nothing; a failure after the checks removes what was written.

The listener's body limit is raised to the bundle limit for this route; every other route is held to the usual limit by the request guard before its body is read, and must declare a Content-Length (a chunked body is refused with 411).

Resource limits

LimitDefaultSetting
Parallel pipelines3WEBFROST_CONCURRENCY
HTTP fetch timeout30 sWEBFROST_FETCH_TIMEOUT_MS
HTTP response size50 MB—
Browser navigation timeout30 s—
Per-extractor timeout60 sWEBFROST_EXTRACTOR_TIMEOUT_MS
yt-dlp process timeout10 minWEBFROST_MEDIA_TIMEOUT_MS
Browser shutdown5 s—
Import size (file, content or fetched url)10 MiBWEBFROST_IMPORT_MAX_BYTES
URLs per import10 000WEBFROST_IMPORT_MAX_URLS
Snapshot bundle (uploaded and decompressed)2 GiBWEBFROST_IMPORT_BUNDLE_MAX_BYTES
List page size200—
Password / username / tag and key name / URL length1024 / 128 / 100 / 8192—
Graceful shutdown wait for running jobs20 sWEBFROST_SHUTDOWN_TIMEOUT_MS
Media file size500 MB—
Gap between archives of one host1 sWEBFROST_DOMAIN_DELAY_MS

Running the server

The published image runs as the unprivileged user webfrost (UID 1000) and launches Chromium without its own sandbox (Playwright's default). The container boundary is the sandbox: do not add capabilities or mount the host filesystem.

The operator's checklist (secrets, setup token, TLS and proxy headers, egress, file permissions, backups) is a how-to of its own: Harden a deployment.

Dependencies

Run bun audit periodically. The dependencies that parse untrusted input are jsdom and @mozilla/readability (article extraction), playwright + Chromium (rendering) and yt-dlp (media). Keep the image up to date to pick up Chromium and yt-dlp fixes.

Reporting vulnerabilities

Please report vulnerabilities privately rather than in a public issue: use Security → Report a vulnerability on the GitHub repository. Include the affected version or commit, reproduction steps and the impact you observed. Allow up to 90 days for a fix before public disclosure.

On this page