Skip to content

Epistemic Audit

Services → diagnosis

An Epistemic Audit tells you what will go wrong before it does.

The core question is simple: Are you structurally ready to deploy agentic AI without unacceptable hallucination risk? We answer it with evidence, not optimism.

Start a Conversation Architecture Blueprint Baseline stack (LLM+Tool+RAG)

What we assess

Data reality

PDFs, SQL, spreadsheets, KBs, tickets, wikis — and the mismatch between them.

Failure modes

Fabrications, drift, inconsistent answers, policy edge cases, silent uncertainty.

Ontology gaps

Missing concepts and relations that cause retrieval to return “relevant” but unusable evidence.

Decision boundaries

What must never be wrong, what can be approximate, and where abstention and escalation must trigger.

Governance requirements

Audit obligations, traceability, constraint needs, approval workflows, and abstention rules.

Evaluation reality

What you can measure today, what you’re not measuring, and how to build go/no-go gates that survive model churn.

Diagram: what an audit maps

flowchart TB
%% Styles (brModel Standard)
classDef i fill:#D3D3D3,stroke-width:0px,color:#000;
classDef p fill:#B3D9FF,stroke-width:0px,color:#000;
classDef r fill:#FFFFB3,stroke-width:0px,color:#000;
classDef o fill:#C1F0C1,stroke-width:0px,color:#000;
classDef s fill:#FFB3B3,stroke-width:0px,color:#000;

I_Dec(["🎯 Target decision(s) + unacceptable errors"]):::i
I_Data(["📥 Data reality (PDF, SQL, KB, policies)"]):::i

G_Data{"Access available?"}:::s

P_Test("🧪 Probe retrieval and reasoning"):::p
R_Beh(["🔎 Behavior map (what it retrieves, what it misses)"]):::r

G_Gaps{"Safety-critical gaps?"}:::s

P_Fail("⚠️ Elicit failure modes"):::p
R_Modes(["🧾 Failure taxonomy (hallucination, drift, policy edges)"]):::r

P_Gov("🔒 Derive governance requirements"):::p
R_Constr(["🔒 Constraint scope + escalation rules"]):::r

P_Eval("📏 Define evaluation + go/no-go gates"):::p
R_Meas(["📊 Measurement plan (metrics, tests, drift signals)"]):::r

P_Road("🗺️ Prioritize into staged roadmap"):::p
O_Out(["✅ Audit outputs (report + roadmap + next step)"]):::o

O_Stop(["🛑 Stop / rescope (missing access or scope)"]):::s
O_Blue(["📐 Proceed to Blueprint (design)"]):::o

I_Dec --> G_Data
I_Data --> G_Data
G_Data -->|"no"| O_Stop
G_Data -->|"yes"| P_Test

P_Test --> R_Beh --> G_Gaps
G_Gaps -->|"yes"| P_Fail
G_Gaps -->|"no"| P_Road

P_Fail --> R_Modes --> P_Gov --> R_Constr --> P_Eval --> R_Meas --> P_Road --> O_Out
O_Out --> O_Blue

%% Clickable nodes
click P_Gov "/methodology/constraints/" "Constraints & SHACL"
click P_Test "/methodology/llm-tool-rag/" "Baseline stack"
click P_Eval "/reasoners/governance/" "Governance approach"

🔎 This diagram makes the audit more decision-grade: we first gate on access (if data/policies aren’t accessible, we must rescope), then test behavior and decide whether gaps are safety-critical. Only then do we convert findings into 🔒 constraints, 📏 go/no-go gates, and a 🗺️ roadmap that cleanly hands off into blueprint design.

Deliverables (decision-grade, not slide-grade)

Readiness report

A candid assessment of reliability, risk, and what must change before production.

Prioritized risks

Top failure modes with severity, likelihood, and concrete mitigations.

Quick wins

Low-effort fixes that reduce hallucinations fast (schema, provenance, constraints, evaluation).

Roadmap

Staged plan with measurable milestones and explicit “go/no-go” gates.

Constraint scope

A first-pass map of the rules that must be enforceable, plus the highest-risk violations to block first.

Evaluation plan

A concrete test strategy: what to measure, how to red-team, and how to detect drift and regression over time.

Diagram: from audit to blueprint

flowchart TB
%% Styles (brModel Standard)
classDef i fill:#D3D3D3,stroke-width:0px,color:#000;
classDef p fill:#B3D9FF,stroke-width:0px,color:#000;
classDef r fill:#FFFFB3,stroke-width:0px,color:#000;
classDef o fill:#C1F0C1,stroke-width:0px,color:#000;
classDef s fill:#FFB3B3,stroke-width:0px,color:#000;

P_Audit("🔎 Audit"):::p
R_Find(["🧾 Findings: risks + gaps + priorities"]):::r
R_Scope(["🧩 Scope: ontology + constraints + trace needs"]):::r
G_Scope{"Scope stable?"}:::s
P_Blue("📐 Blueprint"):::p
R_Arch(["📐 Reference architecture + gates"]):::r
G_Go{"Proceed?"}:::s
P_Impl("🧑‍💻 Implementation"):::p
O_Ship(["✅ Governed build in production"]):::o

S_Back(["🛑 Go back: gather missing inputs"]):::s

P_Audit --> R_Find --> R_Scope --> G_Scope
G_Scope -->|"no"| S_Back --> P_Audit
G_Scope -->|"yes"| P_Blue --> R_Arch --> G_Go
G_Go -->|"no"| S_Back
G_Go -->|"yes"| P_Impl --> O_Ship

%% Clickable nodes
click P_Blue "/services/blueprint/" "Architecture Blueprint"
click P_Impl "/services/implementation/" "Implementation"
click R_Scope "/methodology/core-primitives/" "Core primitives"

🚦 This diagram adds the missing gates: audit findings become scope, but we only enter blueprint if the scope is stable enough to design. After blueprint, we explicitly decide whether we’re ready to build — otherwise we loop back to collect what’s missing instead of forcing an implementation.

Best fit

  • Hallucination is unacceptable (legal, medical, financial, safety-critical).
  • Audits or compliance matter.
  • Your data is messy and multi-source.
  • You need a plan that survives model churn.

Start a Conversation