- Problem
- A local cluster and its storage can share the same failure boundary. Moving recovery data off site introduces separate identity, encryption, retention, cost, and dependency risks, while an uploaded object still does not prove that an application can be recovered.
- Solution
- Terraform defines a compartment-scoped OCI foundation with private Object Storage, a default Vault and software key, restrictive networking, notifications, and a dedicated create/list uploader. Backup confidentiality relies on client-side age encryption before upload; the bucket adds provider-managed encryption rather than a customer-managed key. A hardened Kubernetes job exports selected non-secret resources, creates and validates a PostgreSQL dump, checksums and packages the payload, uploads with no-overwrite semantics, verifies presence, and removes plaintext working data.
- Architecture
- The data path runs from a controlled k3s workload through allowlisted export, database dump, checksums, deterministic packaging, client-side age encryption, a create/list-only identity, and private OCI Object Storage. Recovery uses separate administrator authority, strict archive validation, and an isolated PostgreSQL target. Terraform owns the cloud foundation; Kubernetes manifests and guarded scripts own workload deployment and secret handoff.
- Scope
- Self-managed infrastructure project. Completed evidence is limited to the OCI foundation, one controlled encrypted backup, uploader denial tests, and one isolated PostgreSQL restore; broader recovery and several operator-owned controls remain outside the current claim.
Core capabilities
- Applied an isolated OCI foundation for private storage, key management, restricted networking, notifications, and least-privilege upload
- Executed an encrypted no-overwrite backup of selected Kubernetes configuration and a controlled PostgreSQL database
- Restored the verified archive into a separate namespace and fresh persistent volume without reusing the source database
- Prepared and remotely verified an immutable Linux amd64/arm64 backup image while leaving the live migration review-gated
Validation evidence
- Read-only OCI inventory on 24 July 2026 found one private versioned bucket with five lifecycle rules, one private subnet, one Vault, two key resources, one active secret, one budget, and one notification topic; no compute or managed Kubernetes resource was present
- Applied-foundation report records reviewed plans, successful apply, a zero-drift follow-up plan, and live uploader allow/deny checks
- Manual backup evidence records successful dump validation, deterministic data checks, encryption before upload, remote object verification, and plaintext cleanup
- Isolated-restore evidence records a fresh namespace and volume, exact schema and data verification, source-workload integrity, negative paths, and cleanup
- Current local make test passes ShellCheck, 12 mocked backup paths, 7 archive-validator tests, Terraform safety assertions, Kubernetes and container policies, range checks, quota syntax, recurring-restore contract, and blocker tests
- Both Kubernetes overlays render locally and the immutable Linux amd64/arm64 image publication is remotely verified
- Provider-backed terraform validate passes after the duplicate region declaration was reconciled; public repository CI now runs validation and the safety suite