Backup and restore

Experimental: backup and restore are incomplete and must not be treated as a production recovery solution.

UMKIM ships hub backup and restore helpers for a single-node management cluster. Test recovery in an isolated lab before relying on the path in production.

Hub backup

scripts/backup-hub.sh writes a timestamped directory under /var/backups/umkim/<UTC-stamp>/:

Component	Default	Contents
K3s cluster state	always	k3s-db/state.db (SQLite) or etcd/*.db (embedded etcd)
WireGuard	always	wireguard/wg.tgz from /etc/wireguard
Hub ACME	on (BACKUP_ACME=true)	host/traefik-acme.tgz from /var/lib/umkim/traefik-acme
Harbor	off (BACKUP_HARBOR=false)	harbor/pvc/*.tgz, harbor/registry.sql, harbor/harbor-pvcs.json

Not included: Ansible inventory, vault password, spoke clusters, or Velero workload backups. Copy inventory and secrets off-host separately.

# Standard hub backup (etcd + WireGuard + ACME)
sudo ./scripts/backup-hub.sh

# Full backup including Harbor (brief Harbor downtime while scaled down)
sudo ./scripts/backup-hub.sh --full

Hub restore

scripts/restore-hub.sh is destructive. It stops K3s, restores etcd, and optionally restores WireGuard, ACME, and Harbor PVC host paths.

sudo ./scripts/restore-hub.sh --dry-run /var/backups/umkim/<stamp>
sudo ./scripts/restore-hub.sh /var/backups/umkim/<stamp>
sudo ./scripts/restore-hub.sh --yes /var/backups/umkim/<stamp>

For seamless spoke reconnect after hub loss, restore the same WireGuard hub keys and keep the same WAN/overlay plan as when the backup was taken.

After restore, run verify-hub-install.sh and verify-fleet.sh. Re-accept CSRs if clusters show Unknown.

Workload recovery

For Helm workloads installed with FleetApp, prefer a chart-aware recovery plan: restore hub/cluster state, then re-install or reconcile the FleetApp. Hub backup does not include spoke etcd or workload data.

Cluster delete defaults to detach: spokes keep running but are no longer hub-managed until re-enrolled. Hub-routed apps (route.mode: hub) are unreachable while the hub is down; site and none apps may remain reachable on their own paths.

Protect backups as carefully as the encrypted vault and WireGuard keys. See secrets and verification.