Introduction
What WebFrost is, what it saves for every page, and where to start.
See it before installing
The live demo is a read-only WebFrost with real pages and a video archived. Browse the cards, open a snapshot, switch between the archived page, the reader, the screenshot and the video, and search.
WebFrost is a self-hosted web archiver. Give it a URL and it saves the page in several formats at once, then makes the text searchable. By default everything lives in one directory you control: a SQLite database plus plain files.
You can drive it from three surfaces over the same archive: a CLI, a web UI and a REST API.
Coding agents can learn to use and run WebFrost: npx skills add gtkatakura/webfrost installs the webfrost and webfrost-self-host skills.
WebFrost is inspired by ArchiveBox and was built for performance: instead of fetching and rendering a page once per format, it fetches the page once and opens it in the browser once, and every format comes from those two visits. See Why WebFrost for the reasoning and Pipeline for how.
What you get for every URL
| Format | File | What it is |
|---|---|---|
| WARC | archive.warc | The page response in the standard archival format |
| Readability | stored in the database | Article title, byline, excerpt and plain text |
| HTML | stored in the database | The raw HTML the server returned |
page.pdf | Full-page PDF from headless Chromium | |
| Screenshot | screenshot.png | Full-page PNG |
| DOM snapshot | dom.html | The rendered DOM after the page loaded |
| SingleFile | singlefile.html | One self-contained HTML file, viewable offline |
| Media | media/ | Video or audio through yt-dlp, when the URL is a media page |
See Extractors for what each format contains and when it is skipped.
Why it works this way
- One browser navigation per snapshot. All browser formats come from the same page load, and the text formats share one HTTP fetch.
- Open formats. WARC, PDF, PNG, HTML and SQLite need no WebFrost to read. If the project disappears, your archive does not.
- One directory. Back it up with
rsync. No database server is required; PostgreSQL, MySQL and S3-compatible storage are options. - Self-hosted. No account, no cloud service. Your data stays on your machine.
- Safe by default. Requests to private networks are blocked, archived content is sandboxed in the UI, and the API is rate limited. See Security.
Start here
Quickstart
Install the binary and archive a page, or run the server with Docker Compose.
Install
The webfrost binary for macOS and Linux, Chromium, yt-dlp and ffmpeg.
Archive your first page
A short tutorial: archive, inspect every format, search.
Deploy to production
Volumes, reverse proxy, TLS, backups and health checks.
REST API
Every route, status code and rate limit.
How it works
Components, the archive pipeline and the storage model.