Quality, UX, cost, and safety metrics for the Fabrick.

Metrics & SLAs

UX

  • E2E latency (p50/p95): prompt → first token → ready
  • Continuation rate: % tasks requiring resume
  • SLA adherence by tier (light/heavy)

Quality

  • QA pass rate by role
  • Rework count per task
  • Bug regression rate (post-merge)

Cost

  • $ / successful task
  • GPU minutes per heavy run
  • Tokens per role

Safety

  • PII leakage flagged (target 0)
  • High-risk action confirmations captured
  • Policy violations per 100 runs