Configuration
Every WebFrost environment variable, grouped by area, with defaults and where each one applies.
All configuration is through environment variables. There is no config file. A template with comments is in .env.example.
Numeric variables (WEBFROST_CONCURRENCY and the millisecond settings) ignore values that are not whole numbers and fall back to their default. Boolean variables count only the exact value true.
Archive and logging
| Variable | Default | Description |
|---|---|---|
WEBFROST_DIR | ~/.webfrost (CLI and server), /data (Docker image) | Directory holding archive.db (unless DATABASE_URL points elsewhere) and the snapshots/ files. Use an absolute path: file references in the database are paths. |
WEBFROST_LOG_LEVEL | info | debug, info, warn or error. On the CLI, --verbose and --quiet override it. The server also accepts the older name LOG_LEVEL when this one is unset. |
TZ | system time zone (UTC in Docker) | Time zone used to evaluate cron expressions in schedules. |
Database
| Variable | Default | Description |
|---|---|---|
DATABASE_URL | none: SQLite at $WEBFROST_DIR/archive.db | Where the metadata lives. postgres://user:pass@host:5432/db (or postgresql://) for PostgreSQL 13+, mysql://user:pass@host:3306/db for MySQL 8.0.13+, sqlite:///abs/path.db, sqlite://relative.db or a plain file path for SQLite. A relative SQLite path resolves against WEBFROST_DIR. Archive files always go to the storage backend, whichever database you choose. |
WEBFROST_DB_POOL_MAX | 10 | PostgreSQL/MySQL: connections per process (server, CLI or worker). |
WEBFROST_DB_POOL_IDLE_MS | 30000 | PostgreSQL/MySQL: an idle pooled connection is closed after this many milliseconds. |
Precedence: DATABASE_URL when set, otherwise SQLite at $WEBFROST_DIR/archive.db, exactly as in earlier releases. The schema is created and migrated automatically on first use with each database. Every process that shares an archive (server, CLI, schedule worker) must use the same DATABASE_URL and storage settings. See Deploy for choosing one.
PostgreSQL connection options go in the URL as usual, for example ?sslmode=require, or ?options=-c%20search_path%3Dwebfrost to use a schema other than public. For MySQL, start the server with --innodb-ft-min-token-size=1 (or 2) if searches for one- and two-letter words matter: MySQL's default (3) leaves them out of the full-text index.
Server
| Variable | Required | Default | Description |
|---|---|---|---|
PORT | no | 3000 | HTTP port the server listens on. In Compose, PORT sets the host port only. |
HOST | no | 0.0.0.0; 127.0.0.1 with AUTH_DISABLED=true | Address the server binds to. With auth disabled the server only listens on loopback unless HOST is set explicitly. In a container, where the port is published from outside, set HOST=0.0.0.0. |
SESSION_SECRET | yes | none | Signs session cookies. At least 32 random characters: openssl rand -hex 32. The server refuses to start without it unless AUTH_DISABLED=true; a shorter secret starts with a warning. Rotating it signs everyone out. webfrost serve is the exception: when it is unset it generates one on first run and stores it in <WEBFROST_DIR>/session-secret (mode 0600). |
WEBFROST_SETUP_TOKEN | no | none | When set, first-run setup (POST /api/v1/setup and the setup page) must present this value, so whoever reaches a fresh instance first cannot claim it. When unset, setup stays open until an admin exists and the server logs a warning at startup while there is none. Generate one with openssl rand -hex 16. |
AUTH_DISABLED | no | false | true turns off all authentication. Local development and tests only. Never expose it: the server binds to 127.0.0.1 unless HOST says otherwise. |
WEBFROST_TRUST_PROXY | no | false | true makes rate limits read the client address from X-Forwarded-For. Set only behind a reverse proxy. The right-most entry is used (see WEBFROST_TRUSTED_PROXY_HOPS), never the left-most, which the client controls. |
WEBFROST_TRUSTED_PROXY_HOPS | no | 1 | Number of proxies in front of the server that append to X-Forwarded-For. The client address is taken this many entries from the right: 1 for a single nginx/Caddy/Traefik, 2 for a CDN in front of it. |
WEBFROST_SHUTDOWN_TIMEOUT_MS | no | 20000 | On SIGTERM/SIGINT the server stops accepting requests and the scheduler, then waits up to this long for running archive jobs. Jobs still running at the deadline stay pending and resume on the next start. |
WEBFROST_IMPORT_MAX_BYTES | no | 10485760 (10 MiB) | Largest import accepted: an uploaded file, the content field, or what an import url returns. Larger imports get 413. |
WEBFROST_IMPORT_MAX_URLS | no | 10000 | Most URLs one import may queue. Larger imports get 413 and queue nothing. |
WEBFROST_IMPORT_BUNDLE_MAX_BYTES | no | 2147483648 (2 GiB) | Largest snapshot bundle POST /api/v1/snapshots/import accepts, both as uploaded and once decompressed; at most 4 GiB - 1 (plain ZIP). Larger bundles get 413. The upload is streamed to WEBFROST_DIR/tmp, so that disk needs the room. Every other route keeps the usual body limit (twice WEBFROST_IMPORT_MAX_BYTES plus 1 MiB). |
WEBFROST_PUBLIC_READ | no | false | true lets anonymous visitors browse the archive read-only. See Public read. |
WEBFROST_INSECURE_COOKIES | no | false | true drops the Secure flag from the session cookie, for HTTP-only deployments on a non-localhost address. Prefer TLS. |
WEBFROST_SCHEDULER | no | enabled | false stops the server from running due schedules, for when a separate webfrost schedule worker does it. |
Pipeline
| Variable | Default | Description |
|---|---|---|
WEBFROST_CONCURRENCY | 3 | Archive pipelines running in parallel (server job runner, import, schedule worker). Each running pipeline can launch its own headless Chromium. |
WEBFROST_DOMAIN_DELAY_MS | 1000 | Spaces archives of the same hostname by about this many milliseconds. 0 turns it off. Different hosts never wait on each other. |
WEBFROST_FETCH_TIMEOUT_MS | 30000 | Timeout for the page fetch, redirects included. Also applies to feed and import fetches. 0 disables it. |
WEBFROST_EXTRACTOR_TIMEOUT_MS | 60000 | Timeout for each extractor. One that exceeds it is recorded as failed. 0 disables it. |
WEBFROST_MEDIA_TIMEOUT_MS | 600000 | Timeout for each yt-dlp run. A media archive can run it up to three times, and the media extractor as a whole is allowed three times this value. |
WEBFROST_DEFAULT_THEME | unset | light or dark. The web UI's theme for visitors who have not chosen one; unset, it follows the visitor's system setting. A choice made in the UI is saved in the browser and wins. Does not affect archiving; see WEBFROST_COLOR_SCHEME. |
WEBFROST_COLOR_SCHEME | light | light or dark. The theme the browser archives pages in (screenshot, PDF, Page view) when the request does not choose one: the CLI, schedules and imports. The web UI archives in its own current theme. |
WEBFROST_SETTLE_MS | 5000 | After the page's load event, wait up to this long for the network to go quiet before capturing, so single-page apps finish loading. Requests still go through the SSRF guard. 0 disables the wait. |
WEBFROST_MEDIA_MAX_HEIGHT | 1080 | Video is downloaded at up to this height, preferring VP9 or AV1 with Opus, which Chromium plays even without the patented codecs. 0 means no limit: the best quality, for example 4K AV1. |
WEBFROST_READABILITY_MAX_BYTES | 10485760 | HTML longer than this (10 MB) is cut before the readable-text extraction, which blocks the process while it runs. 0 turns the cap off. |
WEBFROST_SINGLEFILE_CONCURRENCY | 8 | Images, stylesheets and fonts the self-contained HTML downloads at the same time. |
WEBFROST_SINGLEFILE_MAX_BYTES | 104857600 | Total bytes the self-contained HTML may inline (100 MB). Assets past the budget stay as links to the live site. |
WEBFROST_CHROMIUM_PATH | none | Use this Chromium or Chrome binary. Checked first; without it WebFrost uses the Chromium webfrost setup installed, then a system Chrome or Chromium, then Playwright's own. See the lookup order. |
WEBFROST_CACHE_DIR | $XDG_CACHE_HOME/webfrost or ~/.cache/webfrost (Linux), ~/Library/Caches/webfrost (macOS), %LOCALAPPDATA%\webfrost (Windows) | Where webfrost setup installs Chromium. |
WEBFROST_ALLOW_PRIVATE_URLS | false | true allows archiving loopback, private and link-local addresses (intranet archiving). Leave it off when people you do not fully trust can submit URLs. See Security. |
Fixed limits that are not configurable: browser navigation 30 seconds, response body 50 MB, media file 500 MB, browser shutdown 5 seconds, 5 redirects per fetch.
Storage
| Variable | Default | Description |
|---|---|---|
WEBFROST_STORAGE_BACKEND | filesystem | filesystem or s3 |
S3_ENDPOINT | none | Endpoint URL. Required for s3. |
S3_BUCKET | none | Bucket name. Required for s3. |
S3_ACCESS_KEY_ID | none | Access key ID. Required for s3. |
S3_SECRET_ACCESS_KEY | none | Secret access key. Required for s3. |
S3_REGION | us-east-1 | Bucket region |
S3_PATH_PREFIX | snapshots | Object key prefix inside the bucket |
Setup and provider endpoints: Storage.
OpenTelemetry
Off unless an OTLP endpoint is set; without one the OpenTelemetry SDK is not loaded at all. These are the standard OpenTelemetry variables; what is exported and what is never recorded: Observability.
| Variable | Default | Description |
|---|---|---|
OTEL_EXPORTER_OTLP_ENDPOINT | none | OTLP/HTTP receiver, e.g. http://otel-collector:4318. Turns export on. |
OTEL_EXPORTER_OTLP_TRACES_ENDPOINT, ..._METRICS_ENDPOINT, ..._LOGS_ENDPOINT | none | Full URL for one signal; a signal without an endpoint is not exported. |
OTEL_EXPORTER_OTLP_HEADERS | none | key=value,key2=value2 headers for the export requests (collector authentication). |
OTEL_EXPORTER_OTLP_PROTOCOL | http/protobuf | The only protocol supported; grpc and http/json turn the signal off with a warning. |
OTEL_SERVICE_NAME | webfrost | service.name of the exported data |
OTEL_RESOURCE_ATTRIBUTES | none | Extra resource attributes, key=value,... |
OTEL_TRACES_SAMPLER, OTEL_TRACES_SAMPLER_ARG | parentbased_always_on, 1 | Trace sampling; parentbased_traceidratio with 0.1 keeps one trace in ten. |
OTEL_TRACES_EXPORTER, OTEL_METRICS_EXPORTER | otlp | none stops exporting that signal |
OTEL_LOGS_EXPORTER | none | otlp also exports log lines |
OTEL_METRIC_EXPORT_INTERVAL | 60000 | Metric export interval in ms |
OTEL_SDK_DISABLED | false | true turns export off even with an endpoint set |
WEBFROST_OTEL_FULL_URLS | false | true records full URLs of archived pages on spans (default: the host only) |
Docker Compose only
| Variable | Default | Description |
|---|---|---|
IMAGE_TAG | latest | Image tag that compose.yml pulls |
LITESTREAM_S3_BUCKET | none | Bucket the optional litestream service (profile backup) replicates archive.db to. See Backups. |
LITESTREAM_S3_PATH | webfrost/archive.db | Path of the replica inside the bucket |
LITESTREAM_S3_ENDPOINT | none (AWS) | Endpoint of an S3-compatible store (MinIO, R2, B2) |
LITESTREAM_S3_REGION | us-east-1 | Bucket region |
LITESTREAM_S3_FORCE_PATH_STYLE | false | true for MinIO and other stores that need path-style URLs |
LITESTREAM_ACCESS_KEY_ID | none | Access key for the backup bucket |
LITESTREAM_SECRET_ACCESS_KEY | none | Secret key for the backup bucket |
Where to set them
WEBFROST_DIR=$HOME/webfrost-data WEBFROST_LOG_LEVEL=debug webfrost add https://example.com