Self-managed engineering project · Restore evidence plus live OCI inventory verified 24 July 2026
A cost-guarded Oracle Cloud recovery system that encrypts a controlled k3s workload off site, restricts the uploader by design, and proves one backup through an isolated restore.
A local cluster and its storage can share the same failure boundary. Moving recovery data off site introduces separate identity, encryption, retention, cost, and dependency risks, while an uploaded object still does not prove that an application can be recovered.
Terraform defines a compartment-scoped OCI foundation with private Object Storage, a default Vault and software key, restrictive networking, notifications, and a dedicated create/list uploader. Backup confidentiality relies on client-side age encryption before upload; the bucket adds provider-managed encryption rather than a customer-managed key. A hardened Kubernetes job exports selected non-secret resources, creates and validates a PostgreSQL dump, checksums and packages the payload, uploads with no-overwrite semantics, verifies presence, and removes plaintext working data.
The OCI foundation was applied and reported converged, one encrypted backup was verified, uploader denial tests passed, and that backup was restored into a separate namespace and fresh volume with deterministic data checks. A read-only tenancy inventory on 24 July 2026 confirmed the private versioned bucket, five lifecycle rules, private subnet, Vault/key/secret metadata, budget and notification topic, with no compute, OKE, load balancer, NAT gateway, or database resources. The synthetic scope, notification delivery, retention lock, hard quotas, independent key custody, Flux ownership, and recurring restores remain incomplete.
The project treats disaster recovery as a chain of bounded claims: prevent unintended cloud resources, keep secrets out of state, limit uploader authority, validate the archive before use, prove recovery away from the source volume, and leave unresolved operational dependencies visible.
The data path runs from a controlled k3s workload through allowlisted export, database dump, checksums, deterministic packaging, client-side age encryption, a create/list-only identity, and private OCI Object Storage. Recovery uses separate administrator authority, strict archive validation, and an isolated PostgreSQL target. Terraform owns the cloud foundation; Kubernetes manifests and guarded scripts own workload deployment and secret handoff.
Validated and implemented paths are separated from planned and externally blocked controls in this public-safe OCI recovery architecture.
Read the diagram notesThe root configuration composes foundation, network, Object Storage, Vault, notifications, and uploader-IAM modules around an existing isolated compartment and common governance tags. Sensitive identifiers are exposed only through sensitive outputs.
The applied design uses a private versioned Standard-tier bucket with lifecycle and destruction controls, plus a private subnet and service-gateway path that prohibit public instance addressing.
A default software-key Vault and key are destruction-protected, while secret contents are inserted outside Terraform. The notification topic exists, but no delivery subscription is claimed without an approved destination.
Recovery-instance and OKE feature flags default to false and are reinforced by variable validation, Terraform checks, and repository tests; neither resource class is part of the applied foundation.
The investigation found an existing budget, but the project does not present it as a spending barrier or as proof of zero billing.
A compartment-scoped quota proposal covers prohibited compute, storage, networking, and database classes. Its syntax is validated, but live application remains review-gated.
Static checks reject recovery compute, OKE, load balancers, NAT gateways, databases, public buckets, non-default Vaults, non-software keys, unsafe IAM verbs, and missing lifecycle controls.
Object lifecycle rules bound retained daily, weekly, metadata, previous-version, and multipart data. A storage warning threshold supports review but is not presented as hard cost enforcement.
Terraform defines a compartment-scoped OCI foundation with private Object Storage, a default Vault and software key, restrictive networking, notifications, and a dedicated create/list uploader. Backup confidentiality relies on client-side age encryption before upload; the bucket adds provider-managed encryption rather than a customer-managed key. A hardened Kubernetes job exports selected non-secret resources, creates and validates a PostgreSQL dump, checksums and packages the payload, uploads with no-overwrite semantics, verifies presence, and removes plaintext working data.
Off-site ciphertext does not require another always-on VM or cluster. This reduces cost and attack surface, while accepting that recovery compute must be provisioned separately during a real event.
The recovery identity is inserted and checked through guarded scripts so private key material never becomes a Terraform value or state attribute. The trade-off is explicit operator custody and recovery responsibility.
The scheduled workload can create a unique ciphertext and inspect its metadata, but cannot download, overwrite, delete, or access Vault. Recovery therefore uses separate administrator authority.
Backup metadata correctly records that the historical archive was not restored at creation time. A later isolated drill supplies separate evidence without rewriting the original artifact's claim.
The archive is encrypted inside the restricted backup Pod before any OCI upload begins; only the public recipient is tracked, and plaintext workspace data is ephemeral and cleaned on exit.
OCI policy limits the uploader to the intended bucket prefix and create/inspect operations. Namespaced Kubernetes RBAC allows only the selected reads and denies Secret access, mutation, role escalation, and impersonation.
The backup container runs as a non-root user with a read-only root filesystem, dropped capabilities, bounded temporary storage, and egress limited to required cluster, database, and OCI paths.
The restore validator accepts only the expected schema and regular-file path set, bounds extraction, requires complete checksum coverage, and rejects traversal, links, devices, duplicates, corruption, and non-empty restore targets.
live OCI inventory: private/versioned bucket, five lifecycle rules, private subnet
paid runtime: no compute, OKE, load balancer, NAT gateway, or database found
backup: encrypted upload verified
uploader: create/list allowed; read/delete/key access denied
restore: isolated fresh-volume data check passed
notification subscription: not configured
retention lock: not configured
local safety suite and terraform validate: passedChecks shown here are project-specific; no portfolio-wide claim is applied automatically.
The first manual job stopped before export or upload because a CLI tried to create its default home directory. The workload now points HOME into the size-limited ephemeral volume, after which the backup completed.
The OCI identity domain required an email-shaped value and ambient administrator authentication initially overrode uploader tests. A reserved non-routable address and explicit API-key selection made the test boundary deterministic.
A duplicate region declaration caused current validation to diverge from the earlier applied evidence. The provider now consumes the canonical root variables, provider-backed validation passes, and public repository CI guards the repaired path.
Independent offline key custody, an approved notification endpoint, clean Flux ownership, and secure recurring-restore credentials require separate operator or external-system action.
The project has a deployed private OCI backup foundation, a verified encrypted backup, and a successful isolated restore for the controlled PostgreSQL workload. The 24 July 2026 live inventory agrees with the cost-guarded design and the public repository passes provider-backed Terraform validation and its safety suite. The evidence still does not prove cluster-wide or production recovery.
Links are limited to public-safe source material and sanitized evidence. Credentials, state, plans, identifiers, private topology, and backup contents are intentionally excluded.
Terraform, Kubernetes, automation, tests, runbooks, and sanitized reports.
Public-safe component, status, trust-boundary, and evidence notes for the portfolio diagram.
Assets, actors, abuse cases, controls, and explicitly excluded recovery claims.
Always Free target, lifecycle, plan gates, budgets, and the unapplied quota boundary.
Sanitized encrypted-upload, denial-test, and plaintext-cleanup evidence.
Sanitized isolated-restore, negative-path, data-verification, and cleanup evidence.
Complete independent key custody and notification delivery before enabling the already validated recurring-restore template.