Troubleshooting
Symptoms, causes and fixes for the problems people hit most when running WebFrost.
Run with WEBFROST_LOG_LEVEL=debug (or --verbose on the CLI) first. Debug output shows each extractor's duration, failures, and why a fetch failed.
Snapshot has no PDF, screenshot, DOM snapshot or SingleFile
Cause: Chromium is missing or not found, navigation failed, or a request hit the SSRF policy. Those four extractors are recorded as failed, and the snapshot still ends done with WARC, Readability and HTML.
GET /api/v1/snapshots/:id shows each failure as status: "failed" with an error in extractorMeta. Read the message.
Fix:
webfrost doctor # is a Chromium found, and which one
webfrost setup # install a headless Chromium if none is
# or use a browser you already have
export WEBFROST_CHROMIUM_PATH=/usr/bin/chromiumThe Docker image already bundles Chromium. A message starting with blocked: means the page redirected to a private address: see Security.
No media for a YouTube or Vimeo URL
Cause: yt-dlp is not installed, or the URL is not a media page. Media extraction is then skipped silently and no failure is recorded. The Docker image bundles yt-dlp and ffmpeg.
Fix: install it (brew install yt-dlp, sudo apt install yt-dlp or pipx install yt-dlp, see Install) and make sure yt-dlp is on PATH; webfrost doctor checks it. If a media page yields metadata but no media file, the download may have exceeded the 500 MB cap or failed; check the debug log.
Snapshots stay pending or end failed
A snapshot is pending while it waits in the job runner or runs. The runner executes at most WEBFROST_CONCURRENCY pipelines (default 3), and archives of the same host start about WEBFROST_DOMAIN_DELAY_MS apart (default 1000 ms), so a burst queues up.
A snapshot ends failed when:
- the page could not be fetched: a timeout (
WEBFROST_FETCH_TIMEOUT_MS, default 30 s), a network error, or an SSRF block; - the server answered with an HTTP status of 400 or higher;
- no extractor produced output;
- it was interrupted three times and given up on.
After a crash or restart, snapshots left pending are re-queued automatically: a snapshot is adopted once its lease (45 seconds) has expired, at the next recovery sweep, which runs at startup and every minute. Give it a minute or two.
Fix: raise WEBFROST_CONCURRENCY if the machine can take it, check the debug log for the failure reason, and set WEBFROST_ALLOW_PRIVATE_URLS=true only if you really intend to archive intranet addresses.
Login bounces back to the login page
Cause: the session cookie is Secure, and browsers drop it on plain-HTTP origins other than localhost. Typical when opening http://<lan-ip>:3000.
Fix: serve WebFrost over TLS, or for an HTTP-only deployment set WEBFROST_INSECURE_COOKIES=true.
429 Too Many Requests
Cause: the API limits requests per client IP (see Rate limits). Behind a reverse proxy every client shares the proxy's address unless WEBFROST_TRUST_PROXY=true.
Fix: wait for the Retry-After period, and set WEBFROST_TRUST_PROXY=true behind a proxy that overwrites X-Forwarded-For.
Server exits with SESSION_SECRET env var is required
The Docker image refuses to start without SESSION_SECRET unless AUTH_DISABLED=true, and compose.yml refuses to start until it is set in .env. (webfrost serve generates one instead.)
export SESSION_SECRET=$(openssl rand -hex 32)A secret shorter than 32 characters starts, with a warning.
Schedules do not run
Cause: the scheduler is off (WEBFROST_SCHEDULER=false) and no webfrost schedule worker is running, or the cron expression is evaluated in a different time zone than you expect. The time zone is TZ, which is UTC in Docker.
Fix: remove WEBFROST_SCHEDULER=false or start a worker, and set TZ. A schedule whose previous run is still going waits until it ends.
Downloads fail with 403 or "file not found"
Cause: the file reference stored in the database no longer matches the storage in use. This happens after switching WEBFROST_STORAGE_BACKEND, after moving the data directory to a different absolute path, or when a relative WEBFROST_DIR is used from a different working directory.
Fix: use an absolute WEBFROST_DIR and keep it stable. See Storage.
database is locked
Cause: several processes write to one SQLite database. Connections wait up to 5 seconds for a competing writer, so a server plus CLI commands or a schedule worker on one archive is fine. The error appears when a writer is blocked longer, or after an unclean shutdown.
Fix: look for a stuck process holding the database open. If it persists after a clean shutdown, checkpoint the write-ahead log:
sqlite3 ~/.webfrost/archive.db "PRAGMA wal_checkpoint(TRUNCATE);" # or $WEBFROST_DIR/archive.dbDo not do this while Litestream replicates the archive: stop Litestream first, and let it take a fresh snapshot afterwards. With many writers on one archive, PostgreSQL or MySQL (DATABASE_URL) avoids the single-writer lock altogether.
Search finds nothing after an import
The full-text index may be out of sync. Rebuild it:
webfrost search:rebuildOnly snapshots that finished done are indexed.
webfrost: command not found
The installer puts the binary in ~/.local/bin, which is not on every PATH. Add it (export PATH="$HOME/.local/bin:$PATH" in your shell profile) and open a new terminal, or install somewhere else with WEBFROST_INSTALL_DIR. See Install.
webfrost doctor reports no Chromium
Run webfrost setup. It installs a headless Chromium into WebFrost's cache directory, unless it finds one. If setup or doctor says the browser does not start, or cannot open a page, a system library is missing: they print the command that installs what it needs on your distribution; run it and then webfrost setup again. Installing Chromium from your package manager (apt install chromium) works too. A WEBFROST_CHROMIUM_PATH that points at a missing file is an error on purpose; unset it or fix it.
Unknown command: --verbose
--verbose and --quiet go after the command: webfrost add https://example.com --verbose. You can also set WEBFROST_LOG_LEVEL=debug.
Authenticated pages
Symptom: a page you can read while signed in is archived as its login page, an error page or a paywall.
Cause: WebFrost archives every page as an anonymous visitor. Each snapshot gets a fresh headless Chromium context with no stored session, and there is no way yet to pass it cookies, a browser profile or custom headers (Authorization included), from the CLI, the API or the web UI. WEBFROST_CHROMIUM_PATH changes which Chromium binary runs, never its profile.
What to do instead:
- Archive a public address of the same content, if there is one: a share link, a public export, a print or cached view.
- Save the page yourself while signed in, for example with the SingleFile browser extension, and keep the file next to your archive. WebFrost cannot import such a file yet.
- Follow the roadmap: cookie and browser-profile support is tracked in ROADMAP.md.
Pages on your own network are a different limitation: private, loopback and link-local addresses are blocked unless WEBFROST_ALLOW_PRIVATE_URLS=true. See SSRF protection.