> ## Documentation Index
> Fetch the complete documentation index at: https://docs.exterview.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# AI Observability and Evals

> How Exterview keeps its AI outputs trustworthy in production: critic-agent evaluations, an observability layer over every AI step, and full auditability, with human review owning every decision.

## Purpose

Every AI output in Exterview is itself evaluated, traced, and made auditable before a person acts on it. This is where Responsible AI stops being a policy statement and becomes a running layer over production.

## The critic layer (coming soon)

Exterview is building a critic-agent evaluation to run over the outputs its agents produce — a separate evaluation pass that checks each report before it reaches a reviewer. Once live, an output that fails these checks would be flagged for human review rather than passed through silently.

<CardGroup cols={2}>
  <Card title="Grounding">
    Claims in a report must trace to evidence from the session, not to unsupported assertion.
  </Card>

  <Card title="Rubric adherence">
    The score has to reflect your configured rubric, not drift away from the
    criteria you signed off.
  </Card>

  <Card title="Consistency">
    Comparable evidence produces comparable scores across candidates and sessions.
  </Card>

  <Card title="Safety and quality">
    Outputs are checked for quality and safety issues before they surface to a person.
  </Card>
</CardGroup>

<Note>
  This is still being built and is not running yet. We'll update this page when
  it becomes available.
</Note>

## Observability over every step

Every agent run is instrumented today. Exterview records the inputs, the steps, and the output of each evaluation, so you can see not just what was produced but how the system behaved.

<CardGroup cols={2}>
  <Card title="Traced end to end">
    Each evaluation carries a trace of what the agent saw and produced.
  </Card>

  <Card title="Operational monitoring">
    Run success, latency, and cost are tracked against real production traffic
    today.
  </Card>

  <Card title="Quality signals (coming soon)">
    We're building automatic quality scoring for every run, so weak outputs become
    visible rather than hidden.
  </Card>

  <Card title="Drift and regression watch (coming soon)">
    We're building ongoing monitoring of AI performance over time, so a regression can be caught rather than discovered later.
  </Card>
</CardGroup>

## Auditability

Because every AI step and every evaluation is logged, the chain is reviewable after the fact: what the agent saw, what it produced, what was flagged, and which person approved the outcome. That trail is the evidence a compliance reviewer needs. See [Auditability](/trust/auditability).

## Responsible AI, operationally

Responsible AI is enforced here, in Monitor, not only asserted. Content-based scoring, explainable outputs, mandatory human review, and no autonomous adverse decisions are checked and recorded at this layer. See [Responsible AI](/trust/responsible-ai) for the principles; this page is where they run.

## FAQs

<AccordionGroup>
  <Accordion title="Does a person still make the decision?">
    Yes. This layer improves the quality and traceability of AI outputs; it never replaces the human who owns the outcome. No adverse decision is made autonomously.
  </Accordion>

  <Accordion title="What happens when an output is flagged?">
    It is routed to human review rather than used silently. A person sees the
    flag, the evidence, and the trace before acting.
  </Accordion>

  <Accordion title="Can we see the evidence for an audit?">
    Yes. The trace and the critic's checks are recorded and reviewable, and feed the audit trail. See [Auditability](/trust/auditability).
  </Accordion>
</AccordionGroup>

## Related

<CardGroup cols={2}>
  <Card title="Calibration" href="/monitor/calibration">
    How scoring stays reproducible as evaluations run.
  </Card>

  <Card title="Outcome Intelligence" href="/monitor/outcome-intelligence">
    Post-hire signals that feed evaluation quality.
  </Card>

  <Card title="Responsible AI" href="/trust/responsible-ai">
    The principles this layer enforces.
  </Card>

  <Card title="Auditability" href="/trust/auditability">
    The traceable record across actions and results.
  </Card>
</CardGroup>
