Disk layout
Where WebFrost puts the database and each snapshot's files on disk and in S3, and how to back them up.
Everything WebFrost stores lives under WEBFROST_DIR (S3 files aside).
Next to screenshot.png there is thumbnail.webp: a small WebP of the first screen, which lists and cards load instead of the full page. In media/, poster.jpg is a still from the video, used by lists and as the player's cover. The html and readability results are stored in the database, not as files. A file exists only if its extractor ran and succeeded, so most snapshots lack some of these.
SQLite also creates archive.db-wal and archive.db-shm next to the database while it is in use (write-ahead logging). Keep them with the database.
tmp/ holds snapshot bundles being uploaded to POST /api/v1/snapshots/import (webfrost push), one private folder (bundle-<uuid>) per request, removed when the request ends; folders a crash left behind are removed at the next server start once they are an hour old. It needs no backup.
Snapshot ids
A snapshot's directory is named after its integer id, matching snapshots.id. The directory is created when the first file is written.
File references are paths
The database refers to each file by the path it was written to: an absolute path with the filesystem backend (WEBFROST_DIR/snapshots/<id>/<file>), and s3:<key> with S3. That has two consequences:
- Use an absolute
WEBFROST_DIR. With a relative one (./data) the references are relative too, and break when you start WebFrost from another directory. - If you restore or move the archive, put it back at the same absolute path (in Docker, always
/data).
S3 layout
With WEBFROST_STORAGE_BACKEND=s3, archive.db stays local and the files go to:
<S3_PATH_PREFIX>/<snapshot-id>/<filename>The default prefix is snapshots, so a screenshot is at snapshots/42/screenshot.png. See Storage.
Backup
Back up the whole directory. With S3, also back up the bucket: the database is what maps snapshots to their files.
rsync -av --progress "$WEBFROST_DIR"/ /backup/webfrost/
# or
tar -czf webfrost-backup-$(date +%Y%m%d).tar.gz "$WEBFROST_DIR"/For a backup while the server runs, use SQLite's online backup for the database (sqlite3 archive.db ".backup file.db") and copy snapshots/ separately. Docker specifics are in Deploy.
Deleting a snapshot
webfrost delete <id> and DELETE /api/v1/snapshots/:id remove the database rows, the search entry and the snapshot's files (the directory, or the S3 objects) on a best-effort basis. There is no trash: deletion is permanent.