Building AI agents with real human oversight

The dream of AI agents is compelling: you describe a goal, the agent figures out how to reach it, and it just… does it. No micromanagement. No step-by-step instructions.

The nightmare version of the same story: the agent interprets your goal slightly differently than you intended, and by the time you notice, it has already dropped your production database, emailed your entire customer list, or committed a "refactoring" that rearranged half your codebase.

The difference between dream and nightmare is a single, unsexy thing: human checkpoints.

The current state of AI agent safety

Most AI agent frameworks today treat safety as a post-hoc concern. You build the agent, give it tools, run it — and then you look at logs afterward to see what it did. Some frameworks offer "confirmation" flows, but these are often optional, easy to bypass, and not wired into real approval infrastructure.

A few patterns you'll see in the wild:

• Dry-run mode: The agent prints what it would do. You approve mentally, re-run for real. Risky for multi-step tasks where step N assumes step N-1 ran.
• Sandbox first: Run in a container, check output, promote to prod. Good for code; terrible for agents that interact with external APIs, send emails, or mutate state.
• Human-in-the-loop callbacks: The agent asks the framework, the framework asks the user. Works until the user is asleep, away, or the latency budget is tight.

None of these give you a durable, language-agnostic, audit-trail-backed approval gate that blocks execution until a real human says yes. That's the gap expacti fills.

How expacti works (five sentences)

Your agent calls client.run("rm -rf /tmp/build"). Instead of executing, the command is sent over WebSocket to the expacti backend. The backend queues it and sends a notification to the reviewer (browser UI, Slack, Teams, email, push). The reviewer sees the command, context, and risk score — then clicks Approve or Deny. Only then does client.run() return, with the decision.

The agent blocks synchronously. No polling. No callbacks to wire up. The reviewer is never racing the agent — they're in the critical path.

Seven SDKs, same protocol

Pick your language:

All SDKs speak the same wire protocol over WebSocket. All return the same data structure: whether the command was approved, the latency from submission to decision, and any reviewer notes. All raise typed exceptions on denial or timeout.

Wiring it up in Python

The synchronous case is five lines of real code:

from expacti import ExpactiClient, CommandDeniedException

client = ExpactiClient(url="wss://api.expacti.com/shell/ws", token="YOUR_TOKEN")

decision = client.run("docker build -t myapp:latest .")
print(f"Approved after {decision.latency_ms}ms — executing")

For LangChain agents, wrap it as a tool:

from expacti.langchain import ExpactiTool

tool = ExpactiTool(client=client)

# LangChain agent — every action it takes goes through expacti
agent = create_react_agent(llm, tools=[tool, ...])
agent.invoke({"input": "deploy the latest build to staging"})

The agent never changes. You drop in the tool, and suddenly every shell invocation the agent makes requires your explicit approval before it runs.

TypeScript / Vercel AI SDK

Same pattern for TypeScript, with first-class Vercel AI SDK support:

import { ExpactiClient } from 'expacti';
import { expactiTool } from 'expacti/vercel-ai';

const client = new ExpactiClient({ url: 'wss://api.expacti.com/shell/ws', token: process.env.EXPACTI_TOKEN });

const result = await generateText({
  model: openai('gpt-4o'),
  tools: { shell: expactiTool(client) },
  prompt: 'Run the test suite and fix any failures.',
});

The model can call shell as a tool — but every call blocks until you approve it. No command runs unsupervised.

Go: idiomatic context cancellation

client, _ := expacti.New(expacti.Options{
    URL:   "wss://api.expacti.com/shell/ws",
    Token: os.Getenv("EXPACTI_TOKEN"),
})
defer client.Close()

ctx, cancel := context.WithTimeout(context.Background(), 5*time.Minute)
defer cancel()

decision, err := client.Run(ctx, "terraform apply -auto-approve")
if err != nil {
    var denied *expacti.CommandDeniedError
    if errors.As(err, &denied) {
        log.Printf("Reviewer denied: %s", denied.Command)
        return
    }
    log.Fatal(err)
}
log.Printf("Applied — approved in %dms", decision.LatencyMs)

Context cancellation works as expected: if the context expires before the reviewer responds, Run returns a TimeoutError. The backend auto-denies the pending command.

Designing your approval UX

How you structure approval requests matters as much as the SDK you pick. A few things we've learned:

1. Group related commands into a single session

Don't submit 50 individual client.run() calls for a deployment pipeline. Instead, define a session around a goal ("deploy v2.3.1 to staging") and let the reviewer see the full context — current step, remaining steps, what's already run.

2. Build a whitelist, not a gate

The first few times an agent runs docker ps, you'll approve it. After that, it should be whitelisted — expacti will pass it through without a human review. The approval gate is for new or risky actions, not routine operations.

expacti's built-in AI suggestions engine watches your sessions and proposes whitelist patterns: "You've approved rm /tmp/build-* 12 times — want to whitelist this glob?" You accept once, and that class of command flows freely thereafter.

3. Risk score everything

Not all commands are equally dangerous. Reading a file vs. deleting one. Querying a DB vs. dropping a table. expacti assigns a risk score (0–100) to every command based on 8 anomaly rules. Route high-risk commands to senior reviewers; low-risk ones can be approved by anyone on the team.

4. Set reviewer SLA expectations

An agent that can't get approval in 30 seconds will timeout and fail. That's fine during a workday. It's not fine at 2 AM. Configure your agents to know when human oversight is available — and what to do when it isn't (graceful degradation, not silent skipping).

What the reviewer sees

The reviewer UI shows the command, the session context (what's already run, what's pending), the risk score with anomaly badges, the agent's identity, and the target server. Approve and Deny are a click away — or keyboard shortcuts A/D if you're moving fast.

On mobile, it's swipe-right to approve. You can review pending commands from your phone while the agent runs on a server somewhere, patiently waiting.

If you use Slack, the approve/deny buttons appear directly in the Slack message — no tab switching. Same for Microsoft Teams.

The audit trail

Every approval and denial is recorded with: who reviewed, how long it took, what the risk score was, whether it was a whitelist hit or a manual review. Sessions are recorded in asciinema format and are playable in the browser.

If something goes wrong, you have a complete record of every decision — human and automated. SOC 2 and ISO 27001 compliance reports are one API call away.

Getting started

The fastest path to a running instance:

# 1. Pull and start
git clone https://github.com/kwha/expacti
cd expacti
make eval  # starts backend + Caddy + demo config

# 2. Open reviewer UI
open http://localhost:3001/app
# Login: reviewer-token

# 3. Connect your agent
export EXPACTI_TOKEN=demo-shell-token
export EXPACTI_URL=ws://localhost:3001/shell/ws

python3 -c "
from expacti import ExpactiClient
c = ExpactiClient(url='$EXPACTI_URL', token='$EXPACTI_TOKEN')
d = c.run('echo hello from agent')
print('approved:', d.approved)
"

Approve the command in the UI. Watch it execute. That's the loop.

The thesis behind expacti is simple: AI agents are only as trustworthy as the oversight mechanisms around them. Not oversight after the fact — oversight in the loop, before consequential actions execute.

We built the SDKs to make that easy to adopt. Pick your language, add three lines, and your agent becomes one that asks before it acts.

Source: github.com/kwha/expacti