# Observability signal flow

## Purpose and scope

This diagram assigns ownership from signal production to operator action. It
describes the logical flow for metrics, logs, traces, probes, dashboards, and
alerts; it does not claim complete telemetry or successful notification
delivery.

- **Source:** `sources/observability.mmd`
- **Generated output:** `generated/observability.svg`

![Sanitized observability signal flow](generated/observability.svg)

## Components and ownership boundaries

Workloads and nodes produce signals. Prometheus stores metrics and evaluates
alert rules; Alertmanager groups, routes, and attempts delivery of firing
alerts. Loki owns logs; Tempo owns traces; Cilium/Hubble adds network-flow
context; Grafana owns presentation. Backup jobs emit status and size logs. Cloud
budget/monitoring configuration and notification destinations cross the Google
Cloud/account boundary and require separate verification.

## Data flow and failure paths

Signals flow from systems → collectors/stores → queries, dashboards, or alert
policies → channels → operator action. Missing instrumentation, scrape or ingest
failure, stale queries, unavailable retention, an unconfigured channel, and
failed delivery are distinct gaps. Absence of a signal remains unknown; it is
not a healthy result. Kepler is an optional capacity input and no energy saving
is inferred without a measured range.

## Omissions and verification

Measurements, tenant names, endpoints, retention values, notification IDs, and
alert recipients are omitted. Prometheus, Alertmanager, Grafana, Loki, and Tempo
workloads plus relevant Flux releases were Ready in the read-only snapshot on
16 July 2026. Terraform monitoring configuration/state evidence is dated
15 July 2026. Read-only GCP inventory on 24 July confirmed three log metrics,
three alert policies, one dashboard, and one uptime check. Notification-channel
configuration and end-to-end delivery were not verified.

See [operations documentation](../../hybrid-cloud/docs/operations.md) for the
ownership and monitoring context.

Regenerate and validate with `npm run diagrams && npm run diagrams:check`.