RAG Evidence Coverage Auditor
Audit whether outputs are evidence-backed; list unsupported claims; propose retrieval queries to close gaps.
AI governance, retrieval-augmented generation quality, audit trails, and compliance frameworks.
Audit whether outputs are evidence-backed; list unsupported claims; propose retrieval queries to close gaps.
Identify contradictions; resolve by evidence weighting and recency; if unresolved, mark UNKNOWN with next steps.
Rewrite outputs to enforce strict citation; remove or label any unsupported assertions.
Generate adversarial questions to probe weak points; require evidence-backed answers; flag failures.
Define rules to treat docs as data; detect malicious instructions; sanitize retrieval; log injections.
Define provenance fields: source IDs, timestamps, claim links, version hashes.
Specify evaluation tests: factuality, coherence, bias, compliance, reproducibility; define pass/fail thresholds.
Audit for biased framing; propose neutral rewrites; ensure sovereignty-respecting language.
Draft MRM policy: acceptable use, review gates, incident response, documentation, retraining cadence.
Define instruction library governance: approvals, versioning, access control, change logs.
Define minimization rules, retention, access logging, and export gating by sensitivity.
Define how UNKNOWNs are tracked, verified, and resolved; include decision trigger linkage.
Generate reproducibility metadata: inputs, instruction versions, retrieval scope, assumptions, output hashes.
Define evidence weighting rubric; apply to retrieved sources; document conflicts.
Plan multilingual retrieval (Khmer/Thai/Vietnamese/Chinese/English where available); define translation QA.
Generate 20–40 targeted queries by topic; include synonyms and institutional terminology.
Validate output conforms to schema; list missing sections; regenerate only missing parts with evidence.
Check requests for illicit content; refuse unsafe parts; provide safe alternatives; log rationale.
Assign confidence per section tied to evidence density; explain what would raise confidence.
Ensure scenario branches don’t contradict baseline facts; reconcile; label uncertainties.
Ensure outputs present options and triggers; avoid unauthorized “decide for user”; enforce human gates.
Add export checklist: approvals, redactions, attribution checks, evidence verification.
Define rubric: accuracy, provenance, clarity, feasibility, compliance, risk controls; score outputs and justify.
Link each claim to evidence; list orphan claims; propose retrieval or removals.
Compare versions; explain changes, drivers, evidence updates, and impacts on recommendation.
Produce a concise exec summary that references evidence and clearly states uncertainties.
Select 12 best service configurations from this catalog for the scenario; order them for workflow; explain why.
Define automation hooks: run triggers, audit logging, approvals, export; include security controls.
Propose KB updates: new tags, deprecations, contradiction merges, and governance approvals.
Execute deep scan: retrieval plan, contradiction resolution, scenario testing, compliance gates, and audit appendix; fail closed on missing evidence.