QA Results — Scenario C (Hallucinated guidance causes customer harm)
Date: 2026-02-24 Pack version under test: v1.0
Scenario: Support uses AI to draft a response; model suggests wrong troubleshooting steps; customer experiences outage/harm.
Expected classification
- Data: D1–D2 (no restricted data)
- Output: O1 (external drafted)
- Criticality: C1 → Conditional
Checks (per 10-qa-scenarios.md)
1) Classification in 02a-ai-use-case-matrix.md
- Result: PASS (Conditional is appropriate for O1/C1; ensure “no D3” is explicit)
- Patch needed: add an explicit worked example entry for support drafting (optional).
2) Use‑Case Card
- Result: PASS — created
use-cases/scenario-c-support-hallucination-card.md
3) Risk register (03-people-harm-risk-register.md)
- Result: PASS — hallucination harm scenario exists (R1) and includes kill‑switch/rollback control (C‑I3).
4) Controls map (04-controls-map.md)
- Result: PASS — HITL (C‑H1), QA sampling (C‑Q1), logging (C‑L1), incident response (C‑I1/C‑I2), and kill switch/rollback (C‑I3) exist.
5) 30‑day plan (05-30-day-implementation-checklist.md)
- Result: PASS — includes incident channel, HITL workflow, QA sampling.
- Gap: ensure QA reports and escalation playbook are called out as evidence.
6) Training deck (06-training-deck-outline.md)
- Result: PARTIAL — outline exists.
- Patch: add a 10‑question quiz + 1 exercise (applied).
Patches applied in this QA run
- Added scenario-specific Use‑Case Card file.
- Added quiz + exercise to training outline.
- Tightened controls map incident response wording to make kill-switch ownership explicit.