# Heterogeneous GPU inference architecture

## Purpose and scope

The diagram describes the public repository’s implemented control-plane,
queue, worker, model-storage, and observability relationships. It is a
configuration and contract view, not proof of GPU execution, model quality,
throughput, production security, or a live Kubernetes rollout.

![Heterogeneous GPU inference architecture](../../assets/diagrams/dual-gpu-ai-orchestrator.svg)

## Boundaries and data flow

The React dashboard submits work to a FastAPI gateway. PostgreSQL retains job
state, while Celery and Redis route vision and language-model jobs to separate
worker queues. The vision worker prefers an Intel OpenVINO provider and the
language-model worker is configured for an AMD Vulkan path; both document CPU
fallback behavior. Model files remain on runtime volumes rather than being
embedded in container images.

Prometheus-compatible metrics and Grafana describe the operator-facing signal
path. The repository also provides Compose and Kustomize deployment
descriptions, but a shared environment still requires operator-managed
secrets, reviewed storage, authenticated TLS ingress, immutable images, and
properly labeled GPU nodes.

## Evidence boundary

On 23 July 2026, the latest public GitHub Actions run passed four hosted API and
queue-contract tests, Python lint and compilation, the frontend build, Compose
validation, and Kustomize rendering. A separate matrix successfully built and
published four container images. Three live-cluster smoke tests exist but are
opt-in and were not counted as hosted evidence.

No repeatable Intel/AMD execution result or throughput benchmark is published.
The current dependency audit covers the API requirements rather than every
worker dependency. The LLM stack also retains a documented transitive advisory
without a fixed upstream release. Those gaps remain explicit prerequisites for
any promotion beyond a lab.