Public engineering project · Hosted CI and four-image pipeline verified 23 July 2026
A queue-based inference platform that routes vision work toward an Intel OpenVINO worker and language-model work toward an AMD Vulkan worker while keeping API state, model storage, metrics, and hardware-validation boundaries explicit.
Heterogeneous GPUs do not share one runtime, workload shape, or scheduling contract. A useful platform has to route jobs by capability, retain job state, expose progress and failure, isolate model artifacts from images, and avoid claiming hardware performance that hosted CI cannot prove.
A React dashboard submits work to a FastAPI gateway backed by PostgreSQL. Celery dispatches jobs through Redis to separate vision and LLM queues; the workers select OpenVINO, Vulkan, or documented CPU fallbacks. Prometheus and Grafana cover service signals, while Compose and Kustomize describe local and Kubernetes deployment paths.
The latest public commit passed GitHub Actions for four API/queue contract tests, Python lint and compilation, the frontend build, Compose validation, and Kustomize rendering. A separate successful matrix built and published four container images. GPU execution, model provenance, throughput, multi-user security, and a live Kubernetes rollout remain outside the verified result.
The project connects API design, asynchronous orchestration, persistence, heterogeneous compute, container builds, Kubernetes configuration, observability, and evidence-led release boundaries in one reviewable system.
The control plane separates request handling, durable job state, queue dispatch, and status events from the device-specific data plane. Vision jobs target an Intel/OpenVINO worker, LLM jobs target an AMD/Vulkan worker, model files remain on runtime volumes, and metrics feed an operator dashboard.
The control plane, device-specific worker paths, observability, and hosted-CI evidence boundary for the public inference-platform project.
Read the diagram notesA React dashboard submits work to a FastAPI gateway backed by PostgreSQL. Celery dispatches jobs through Redis to separate vision and LLM queues; the workers select OpenVINO, Vulkan, or documented CPU fallbacks. Prometheus and Grafana cover service signals, while Compose and Kustomize describe local and Kubernetes deployment paths.
Separate vision and LLM queues keep device assumptions and failure modes visible. The trade-off is more deployment configuration and explicit capacity planning for each worker class.
Images stay smaller and model acquisition remains an operator decision. Deployments must separately verify model provenance, hashes, licensing, availability, and persistent-volume behavior.
CI validates routing, API behavior, frontend compilation, configuration rendering, and image construction without downloading models or pretending hosted runners contain the target GPUs. Hardware execution needs a separate signed benchmark and rollout record.
Kubernetes deployment requires an operator-created Secret; repository manifests do not contain deployable passwords. Shared environments still need an approved secret-management controller.
Worker containers disable privilege escalation and drop Linux capabilities while mounting only the required render devices. Host device groups remain a trusted deployment boundary.
The validation workflow has read-only repository permissions, while the image workflow receives package-write permission only for publishing its matrix outputs.
GitHub Actions CI: passed
hosted API/queue tests: 4 passed
live-cluster smoke tests: 3 present, opt-in, not counted
frontend build: passed
Compose and Kustomize validation: passed
container image matrix: 4 passed
GPU runtime and throughput benchmark: not recordedChecks shown here are project-specific; no portfolio-wide claim is applied automatically.
Links are limited to public-safe source material and sanitized evidence. Credentials, state, plans, identifiers, private topology, and backup contents are intentionally excluded.
API, workers, frontend, Compose, Kubernetes, monitoring, tests, and release workflows.
Control-plane, worker, model-storage, observability, and evidence boundaries.
Hosted API tests, lint, frontend build, Compose validation, and Kustomize rendering.
Secret, image, node-label, storage, and lab-deployment prerequisites.
Deployment scope, reporting path, and current dependency exception.
Record a reproducible Intel/AMD hardware benchmark and promote signed immutable images only after full worker dependency audit, model provenance checks, authentication, TLS, and a reviewed live rollout.