How it works
How ifa works.
A read-only pipeline that reads what your cluster already exposes and reports which of 19 rules fired, with evidence — or reports nothing measured.
Current status
The vLLM and Triton adapters have been validated against live CPU-backed servers and NVIDIA L4 GPU hardware, and NVIDIA's official DCGM exporter has been validated on an L4, each with authentic idle, loaded and recovery captures. IFA has also been deployed with Helm into a real GPU-enabled K3s cluster, where it discovered a vLLM workload, scraped live vLLM and DCGM metrics, and produced diagnostic findings through its REST API. That run was a single node with one GPU. Not yet validated: a real TimescaleDB, multi-GPU behavior, per-pod GPU attribution, webhook delivery reliability, KV-cache preemption inside Kubernetes, and any external-user validation.
The latest validation evidence is merged into main and postdates the published v0.2.0-alpha.1 release.
The pipeline
Kubernetes discovery → vLLM metrics + DCGM metrics → IFA collector → telemetry store → rule engine → REST API findings
Discover
An informer-based Kubernetes watcher finds inference workloads by runtime label and reads model identity from an annotation.
Scrape
In-cluster Service networking reaches each workload's Prometheus endpoint over a plain GET — no request-path insertion.
Parse
Runtime-specific adapters parse the exposition format for vLLM, Triton, or DCGM Exporter into typed metrics.
Store
Parsed telemetry is stored in-memory. When TimescaleDB is configured it also receives a copy as an additional historical sink — the in-memory store is always present and is not a fallback.
Evaluate
19 rules across 7 families are evaluated on a sustain window against shipped-default thresholds, with suppression between related rules.
Serve
Findings are served over a REST API and the ifa CLI, each with the metric, observed value, threshold, and comparison attached.
Alert
When a webhook URL is configured, state-transition notifications are delivered outbound — firing, escalation, resolution, and suppression. Delivery uses a bounded queue and retries. This is an alert-delivery integration, not a routing or escalation platform.
Stack facts
- Runtimes supported
- vLLM, Triton, DCGM Exporter
- Rule families
- 7 (19 rules total)
- vLLM 0.28.0
- CPU and NVIDIA L4 validated
- Triton 25.12
- CPU and NVIDIA L4 validated
- DCGM
- Official NVIDIA exporter validated on an L4
- Kubernetes (CI)
- kind 1.31 integration validated
- Kubernetes (GPU)
- K3s with vLLM, DCGM and IFA validated end to end on an L4 (single node, single GPU)
- Dynamic multi-scrape
- Validated using synthetic metrics
- TimescaleDB
- Implemented, real-database test pending
Action model
ifa never mutates your cluster. It holds only the read verbs listed on the security page. Findings are surfaced through the API and CLI; a human decides what, if anything, to change.
Repository structure
Air-gapped operation
There is no "air-gapped mode" flag. Air-gapped operation is a property of the deployment — no egress, mirrored images, a NetworkPolicy — not a runtime setting. A configuration field that claimed to enforce it while enforcing nothing would be security theatre.
ifa communicates only with the endpoints it is configured to use: the inference metrics endpoints it scrapes, DCGM exporters, and the Kubernetes API. It can additionally deliver webhook alerts when you explicitly configure a destination: IFA_ALERTING_WEBHOOK_URL sets it and can remain unset in air-gapped deployments. ifa performs no licence check, no usage telemetry, no update check, and no crash reporting. The only things that need to cross the boundary are the container image and the Helm chart. scripts/package-offline.sh mirrors both for transfer, and docs/DEPLOYMENT.md documents the full procedure.