Human Oversight & HITL Architecture

Human-in-the-Loop (HITL) Architecture

AI systems do not govern themselves; they inherit the governance culture of the organization that deploys them. As AI transitions from static, predictive models to agentic, autonomous tools that plan, reason, and alter external system states, passive auditing or simple post hoc monitoring is no longer sufficient.

Rather than relying on vague, narrative assurances, the AI Governance Evidence Infrastructure (AGEI) treats human oversight as a first-class, machine-verifiable control loop. It shifts the core question from a binary "is there a human involved?" to a disciplined framework: When is human oversight required, who has the authority to act, and what cryptographic evidence proves it?

1. The Risk-Proportionate Oversight Spectrum

There is no universal, fixed frequency for human oversight. Instead, AGEI enforces a risk-tiered model where oversight intensity rises proportionally with an AI's authority, context access, and potential operational or safety impact:

Tier 1: Low Risk (Automated & Human-on-the-Loop)

  • Context: Read-only operations, public web searches, or draft summaries.
  • Oversight Mechanism: Continuous, automated monitoring with human-on-the-loop (HOTL) vigilance.
  • Evidence Standard: Continuous generation of lightweight lifecycle receipts containing minimal metadata, signatures, and cryptographic hashes to confirm execution stayed within approved bounds.

Tier 2: Medium Risk (Human-on-the-Loop + Exception Review)

  • Context: Internal read access, knowledge retrieval, or drafting tickets based on proprietary data.
  • Oversight Mechanism: Automated processing by default, but with structured exception reviews (HOTL) triggered by anomalous telemetry, policy warnings, or out-of-bounds metrics.
  • Evidence Standard: Detailed rule evaluations linked to action-request and execution-result receipts in the tampered-evident vault.

Tier 3: High Risk (Human-in-the-Loop Before Action)

  • Context: External communications (e.g., sending emails), modifying customer records, or submitting transaction forms.
  • Oversight Mechanism: Human-in-the-loop (HITL) gatekeepers. The system executes a default-deny policy, pausing execution until a designated reviewer inspects the context, parameters, and recommendations.
  • Evidence Standard: A structured pre_action_proof_bundle containing the human reviewer's signature, authority context, and the corresponding policy evaluation receipt.

Tier 4: Critical Risk (Dual Approval & Escalation)

  • Context: Financial transactions exceeding high thresholds, secrets access, or irreversible database/infrastructure changes.
  • Oversight Mechanism: Mandatory dual-control approvals (four-eyes principle) and multi-level human escalation pathways.
  • Evidence Standard: Linked Merkle-batched receipts representing the full delegation chain, multi-signature approval blocks, and sealed evidence payloads.

2. Regulatory Alignment Crosswalk

Global regulations are rapidly codifying human oversight from a soft recommendation into a binding architectural constraint:

Regulatory InstrumentCore Oversight ThemeAGEI Operationalization Fit
European Union - EU AI ActRequires high-risk AI systems to be designed so that natural persons can oversee them, proportionate to the system's risk, autonomy, and context.Strong Fit. Operationalized via gate_definitions and gate_evaluations that enforce mandatory pauses and human review hooks before execution.
GDPR (Article 22)Grants individuals the right not to be subject to solely automated decisions significantly affecting them, mandating human intervention and contestability.Strong Fit. Ensures that every automated path has a corresponding, verifiable contestability/escalation workflow tracked in data_subject_requests.
NIST AI RMF 1.0Focuses on risk governance, calling for context-driven HOTL or HITL mechanisms to monitor and manage system actions.Strong Fit. Links operational human approvals directly to versioned policy rules (policy_rules) and registers them in the Evidence Vault.
ISO/IEC 42001Obligates organizations to define, document, and continually improve oversight controls in their AI Management System (AIMS).Strong Fit. Converts manual compliance checklists into verifiable audit packs (audit_packs) proving that oversight controls operated as designed.

3. Pre-Action Escalation & State Machine

When a consequential action or tool call crosses a policy-defined risk threshold, the AGEI Policy Engine triggers an Escalation Path rather than a flat allow/deny decision.

The Pre-Action Authorization Sequence:

  1. Request Initiation: An agent attempts to invoke a high-risk tool or execute an external transition.
  2. Gate Evaluation: The gate evaluates the transaction parameter hashes, context, and actor permissions against versioned policy rules.
  3. Escalation Fired: If a rule fails or highlights a high-risk scenario, the gate outcome resolves to escalate.
  4. Escalation Receipt Emitted: The system writes a signed escalation_receipt detailing the pending action, evaluated rules, risk class, and the designated reviewer pool.
  5. Human Review: The system registers the event in a secure, human-in-the-loop review queue.
  6. Resolution Outcomes:
    • Deny: The action is blocked, leaving the target resource completely unchanged. The system emits a cryptographically signed denial receipt. This proves that the control actively restricted runtime behavior (denials are counted as control successes, not failures).
    • Approve: The human reviewer signs the action payload. The system generates a Just-In-Time (JIT) elevation token and packages the contextual hashes into a pre_action_proof_bundle.
  7. Tool Execution: The secure tool wrapper verifies the signature on the proof bundle and executes the action, writing a post-execution receipt.

4. Override Handling: Transparent Exceptions

In production environments, emergency bypasses, patch-releases, or administrative overrides are sometimes necessary to maintain operational continuity.

AGEI enforces a strict No Silent Override policy. Overrides do not erase historical gate failures; instead, they are captured as first-class, immutable, and fully attributable governance actions in the database:

+------------------------------------------------------------+
|                       gate_evaluations                     |
|------------------------------------------------------------|
| - gate_outcome: deny                                       |
| - is_overridden: TRUE                                      |
| - override_reason: "Emergency hotpatch for critical bug"   |
| - override_by: "principal:usr:denzil_greenwood"            |
| - override_at: "2026-08-12T12:00:00Z"                      |
| - evaluation_hash: sha256(canonical_payload_metadata)      |
+------------------------------------------------------------+

By maintaining first-class override columns in the gate_evaluations table, auditors can instantly query every bypass event, reconstruct the exact timeline during forensics, and ensure full human accountability for high-risk manual interventions.

5. Relational Schema Contract

The following database tables from your relational schema contract govern human oversight, escalations, and overrides:

Table 1: gate_evaluations (Policy and Gate Enforcement Family)

Tracks the exact control outcome, human escalations, and manual overrides.

  • id (UUID, PK): Unique identifier for this gate outcome.
  • gate_definition_id (UUID, FK): Links back to the policy-derived gate rule.
  • gate_outcome (VARCHAR): Resolves to approve, deny, escalate, or inspect.
  • is_overridden (BOOLEAN): Flag indicating if a human manually bypassed a restriction.
  • override_reason (TEXT): Compulsory justification text explaining the bypass.
  • override_by (UUID, FK): References the authorized principal who executed the override.
  • override_at (TIMESTAMPTZ): Cryptographically signed timestamp of the bypass.
  • evaluation_hash (VARCHAR): SHA-256 content hash of the evaluation metadata to prevent modification.

Table 2: agent_delegations (Agentic Governance Family)

Establishes the human-to-agent authority link, proving who signed the agent's "permission slip".

  • id (UUID, PK): Unique delegation transaction ID.
  • delegating_principal_id (UUID, FK): References the human principal who delegated authority.
  • agent_principal_id (UUID, FK): References the specific child agent principal.
  • authority_scope (JSONB): Declares exact limits on tools, funding, or data access.
  • validity_period (TSTZRANGE): Start and expiration times of the delegated authority window.
  • delegation_token_hash (VARCHAR): Cryptographic hash protecting delegation token integrity.

6. Verification and Forensic Auditing

When a third-party verifier or internal auditor compiles an audit_pack to evaluate human oversight, the system runs a deterministic verification_job to check the cryptographic proofs:

  1. Signature Verification: Recomputes hashes over the canonicalized human approval payloads and verifies the Ed25519 signatures against the reviewer's public key in the vault.
  2. Lineage Resolution: Traverses the graph via receipt_links and ai_lifecycle_object_links to prove the exact sequence: from the human's delegation ticket, through the agent's escalated gate check, to the resulting tool execution receipt.
  3. Zero-Repudiation Check: Confirms that no override occurred silently by checking the consistency of is_overridden states against the signed batches in vault_objects.