Korab CenajEngineering case studyAll case studies

Open engineering project · Undated plan-only snapshot

Kubernetes Capacity and Placement Optimizer

A Go simulator that tests whether workloads can move and whether a node can leave under real scheduling, storage, disruption, availability, and capacity constraints before an operator accepts a consolidation plan.

Project type
Infrastructure Tool
Domain
Capacity and efficiency
Language
Go
Status
Implemented

Executive summary

Operational problem

Low average utilization is not a maintenance or consolidation plan. PDBs, taints, host ports, topology, persistent-volume locality, Longhorn replica placement, and endpoint loss can make a “quiet” node impossible to remove without reducing availability or stranding state.

Implemented solution

The optimizer combines requests and limits with Metrics API samples, Prometheus history, optional energy signals, scheduling rules, storage state, and failure-domain headroom. Its simulator attempts candidate placements, records the constraint that rejects each move, and produces a reviewable consolidation plan without cordoning, evicting, scaling, changing storage, or powering off nodes.

Outcome

The output shows which constraint blocks a move, how much headroom remains under the scenario, and where a recommendation depends on fallback data. It turns utilization into one input rather than the decision itself.

Technical and operational context

The simulator combines placement, storage, availability, historical demand, and measurement confidence, showing why an efficient-looking utilization target can still produce an unusable consolidation plan.

Architecture

Kubernetes inventory and metrics feed workload models; optional Prometheus, Longhorn, KEDA, Kepler, and Flux data enrich them. A scheduling simulator then produces right-sizing, consolidation, energy, and rollback notes.

  1. Kubernetes API + metrics
  2. Resource model
  3. Scheduling simulation
  4. Storage + availability checks
  5. Consolidation plan
  6. Capacity + energy result

Implementation

The optimizer combines requests and limits with Metrics API samples, Prometheus history, optional energy signals, scheduling rules, storage state, and failure-domain headroom. Its simulator attempts candidate placements, records the constraint that rejects each move, and produces a reviewable consolidation plan without cordoning, evicting, scaling, changing storage, or powering off nodes.

Key decisions and trade-offs

Simulate before changing capacity

The optimizer produces a static plan instead of cordoning nodes or evicting workloads itself. This separates analysis from execution, gives an operator a review gate, and limits blast radius; the tradeoff is that recommendations remain proposals until someone validates assumptions and performs the approved change.

Prefer distributions to point samples

Prometheus history supports burst-aware planning better than a single Metrics API reading, so the model prefers percentiles and trends when they exist. When history is missing or sparse, it lowers the recommendation's confidence instead of extrapolating a demand curve that the available measurements cannot justify.

Keep storage in the placement model

A CPU-efficient placement can still strand an attached volume, violate affinity, or reduce the replicas available during maintenance. Persistent-volume topology, attachments, locality, and Longhorn health therefore enter the same model as compute demand, accepting a more complex plan to avoid an operationally unusable one.

Treat energy as optional evidence

Kepler measurements can add an energy dimension when the cluster exposes reliable power data, but they are not required for a capacity recommendation. If those signals are absent, the optimizer reports the gap and avoids converting a modeled consolidation into an unsupported claim about electricity or cost savings.

Security, reliability, and observability

Plan-only execution

The optimizer does not cordon, evict, scale, modify storage, or power off hosts. It produces a reviewable scenario and keeps the change authority outside the assessment.

Availability constraints

PDBs, endpoints, scheduler rules, host ports, persistent-volume topology, attachments, and Longhorn replica placement remain part of the same consolidation decision.

Measurement confidence

Historical distributions are preferred over point samples; sparse history, fallback metrics, and absent energy signals lower confidence and suppress unsupported savings claims.

Infrastructure concepts demonstrated

Published evidence

Sanitized node-shutdown simulation

Sanitized live output
Decision: BLOCKED
Effects modeled: 14 pods / DaemonSets
PDBs allowing no disruption: 1
Longhorn volumes below required replicas away: 4
Remaining CPU headroom: 67.8%
Remaining memory headroom: 81.6%
No mutation was performed.

Validation

Checks shown here are project-specific; no portfolio-wide claim is applied automatically.

Evidence and links

Links are limited to public-safe source material and sanitized evidence. Credentials, state, plans, identifiers, private topology, and backup contents are intentionally excluded.

Lessons learned

Limitations

  • Scheduler simulation does not exactly reproduce the Kubernetes scheduler or every plugin.
  • Energy savings are omitted when measured or configured power ranges are unavailable.
  • Historical metrics describe observed demand and cannot predict every future burst.

Next improvement

Add topology-scenario comparison across multiple candidate node sets with a concise Pareto view of risk, headroom, and measured energy.