# CX Triage

Turns the Infrahub error-alert firehose into a short list of things that
actually need doing — then helps you do them.

Python/FastAPI backend, React frontend, PostgreSQL for case state.

## What it does

1. **Pulls** the alert queue from Prometheus.
2. **Screens** every alert by re-checking its condition against live state, so
   noise and already-resolved alerts drop out.
3. **Diagnoses** what is left using the CX runbooks, reconciling Infrahub
   against OpenStack through the CX-Tools collectors.
4. **Drafts** the customer email with contacts resolved from Infrahub, and the
   Jira escalation with the evidence attached.
5. **Tracks** each case — who owns it, what was done, what was sent.

On live data this takes roughly **2,650 firing alerts down to ~20** that need a
decision.

## Findings that shaped it

Validated against production, not assumed:

- **`Suspected Rogue VM` is measuring spare capacity.** `In_Use_Gpus` equals the
  physical GPU count on 71 of 75 firing hosts, so the rule reduces to "this host
  has a free GPU". Checked against OpenStack on 10 hosts: Infrahub and OpenStack
  agreed exactly on all of them. Those alerts are flagged as a rule defect.
- **`Exists in Infrahub but does not exist in OpenStack` matches every VM**,
  because `openstack_nova_server_status` returns no series. Excluded outright.
  The same gap means `Suspected Orphan VM` cannot fire at all.
- **Prometheus alert ages are unreliable here.** The `Resources` metric drops
  most of its series several times a day; every alert alive at the time resolves
  and re-fires, resetting `activeAt`. Ages are recovered from `ALERTS` history
  instead.

## Read-only by design

The app queries and advises. It never deletes, shelves, or edits a VM — those
stay copy-a-command. The only thing it can send is a Zendesk ticket or a Jira
issue, behind three gates and a confirm step. See
[docs/INTEGRATIONS.md](docs/INTEGRATIONS.md).

## Run it

```bash
cp .env.example .env
docker compose up --build
```

More: [docs/DEPLOYMENT.md](docs/DEPLOYMENT.md) ·
[docs/LINKAGE.md](docs/LINKAGE.md) · [docs/PLAN.md](docs/PLAN.md)

## Layout

```
backend/
  app/            FastAPI: config, db, models, auth, delivery, routers
  triagelib/      the triage engine (screening, runbooks, comms, linkage)
  tests/          pure-logic tests — no network, no CX-Tools
frontend/         React + TypeScript + Vite
deploy/k8s/       manifests, rendered by the pipeline
.gitea/workflows/ test → build → deploy
```

## Tests

```bash
cd backend && python tests/test_runbooks.py && python tests/test_screening.py
```