Monday, 9:00 AM — Slack: “Book me for NYC tomorrow — flight, hotel, and tell the customer I’ll be there.”

TIMEWHAT THE AGENT DID
9:01:00Agent books the flight. Corporate card charged $1,200.
9:01:30Hotel booked. $400. Confirmed.
9:02:00Agent emails the customer: “See you tomorrow at 10 AM.” Calendar blocked.
9:02:15Hotel webhook: Overbooked. Booking cancelled.

Your agent says: “Hotel failed. Want me to try another?”

But the flight is non-refundable, the customer expects you, and your calendar is already blocked. The workflow looked successful until one completed step was invalidated. The other side effects remain. There is no transaction to roll back.

In Part 1, inference looked like an execution engine. Now the agent has to act.

The runtime must choose an execution plan, then preserve correctness across its side effects. External tools rarely share an ACID boundary, so recovery may require compensation rather than rollback.

How the optimizer derives the plan

Part 1 proposed a physical optimizer. How would it actually choose? Start with one inference request whose reusable KV state sits on a busy worker.

Filter eligible plansEstimate latency and costChoose within constraintsExecute and measureUpdate statistics

Planning happens in two passes: feasibility, then costing. Model capability, context capacity, privacy, available memory and compatible KV state constrain the choices. Live queue depth, cache reuse, measured throughput, transfer bandwidth and execution cost help compare them. These decisions can be revisited as conditions change.

Plan Trade-off Est. latency Est. cost Decision
Stay with KV Reuse state; wait for busy worker 1.6 s $0.003 Too slow
Transfer KV Move state to available compute 1.0 s $0.004 Choose
Recompute Rebuild state on another worker 1.4 s $0.006 Eligible, costs more
Illustrative estimates for the same request. Choose the lowest-cost eligible plan within a 1.5-second latency budget.

The cheapest worker is not necessarily the cheapest valid plan. Here, locality loses to queueing; transfer beats recomputation. If the network slows, recomputation may win instead. These are hypothetical estimates, not benchmark results or provider prices.

NVIDIA Dynamo illustrates this direction through routing that considers cache reuse and worker load. The example above extends that reasoning across placement, transfer and recomputation. EXPLAIN ANALYZE closes the loop by comparing estimates with actual latency and cost, then updating the statistics. The decision changes with the workload; authorization and privacy remain constraints.

From an inference plan to an agent task

For an agent, the same reasoning applies to the whole task: weigh expected cost and latency within the task budget and the constraints on models, tools and actions. Include likely retries and tool delays, and replan when outcomes change. A plan can outlive its approval or source state, so recheck before consequential actions.

EXPLAIN INFERENCEPlanModel · Placement · KV reuse · Estimated latency / costWhy this execution plan?
EXPLAIN ANALYZEActualsQueue · Transfer · Tokens · Latency · CostWhere did estimates miss?
EXPLAIN AGENT TASKTrajectoryMemory · Tools · Authority · Commits · RecoveryWhat happened across the whole task?

Six Database Responsibilities Agents Rediscover

Once agents act on external systems, a cheap plan is useful only if it executes correctly. The runtime takes on six familiar database responsibilities:

Agent problem Database responsibility What the runtime needs
1. The payment times out. Retrying charges twice.
The hotel later cancels, but the flight and customer email remain.		 Transactions Durable intent and identity, receiver-enforced idempotency, commit boundaries, replay-safe operations, sagas and compensation. A timeout can mean COMMIT_UNKNOWN — treat it as an outstanding resource commitment that remains visible to the governor / admission control. Reconcile before retrying; if reconciliation cannot resolve the outcome, escalate to HIL. Track later cancellations and failed compensation; repair cannot undo every committed effect.
2. A document tells the agent to send private data—and it does.
Retrieved content is mistaken for authority.		 Authorization / trust boundary Separate instructions from retrieved content, scope permissions to the task and record provenance. Bind approval to the concrete action and its limits; recheck authority at dispatch when plans change or approvals expire. A model’s claim of permission cannot authorize a tool call.
3. You correct a fact. The agent keeps acting on the old summary.
A model inference can persist as if you stated it.		 Persistent / versioned state Separate user-stated facts, model-derived conclusions, historical versions and temporary context. Preserve sources and derivation links so corrections invalidate stale summaries and dependent plans. Scope retrieval to the user and task; persistent memory must not leak across tenants. The context window is the working set, not the database.
4. Two agents book the same trip. Both report success.
Separate idempotency keys do not prevent the duplicate intent.		 Concurrency control Record read versions, detect lost updates and validate preconditions at the commit boundary. Claim shared operations atomically and fence stale workers at dispatch. Deduplication does not resolve conflicting intent, and an unguarded check-then-act still races.
5. The agent restarts and repeats a completed action.
It may also resume work you already cancelled.		 Recovery Resume from checkpoints and recorded outcomes, preserving completed tool/model results. Calling the model again is a new attempt, not deterministic replay. Recheck deadlines, cancellation, approval and remaining budget before resuming. Cancellation stops future dispatch; committed effects and in-flight calls still need reconciliation or repair.
6. Sub-agents each spend the full task budget.
Retries multiply cost while background work delays your live request.		 Workload management Priorities, quotas, admission control and checkpointing across tasks and sub-agents. Enforce a shared task budget so branches cannot each spend the full allowance; bound retries and fan-out. Protect interactive latency while throttling background work, with enough reserved capacity to avoid starving recovery and maintenance.

Together, these responsibilities define what the agent's execution record must capture. EXPLAIN AGENT TASK should make them visible across the full task trajectory — what state was read, what was authorized, what committed and how the runtime recovered.

What EXPLAIN AGENT TASK Might Look Like

Here is an ambiguous commit as a task trace: the payment executes, but its acknowledgement is lost.

ILLUSTRATIVE — condensed from a run of the reference implementation below
EXPLAIN AGENT TASK  task-7422

Goal           Complete approved purchase
Trajectory     4 steps / 1 attempt shown
Models         gpt-5.2 (plan)
Memory         1 lookup
               └─ preferred vendor → HIT
Tools          payment.charge
Policy         purchase policy → PASSED
Checkpoint     cp-05 (before the side effect)
Transaction    payment: COMMIT_UNKNOWN
Retry          blocked pending commit check
Idempotency    pay_task-7422_01
Cost           $0.0019 (estimated)
Elapsed        3.4 sec

I built a small open-source reference implementation of these ideas.

Open-source reference implementation · Pythoninference-explainAn open-source systems prototype that makes real calls to OpenAI, Gemini and Anthropic and projects actual inference, tool, memory and transaction execution into EXPLAIN INFERENCE, EXPLAIN ANALYZE and EXPLAIN AGENT TASK views. Not a production system: the payment service is a local stand-in that moves no money. Recorded runs replay without API keys.github.com/vanuganti/inference-explain ↗

Its transaction demo creates the difficult case described above: the external action commits, the acknowledgement is lost, the runtime enters COMMIT_UNKNOWN, blocks a blind retry, and reconciles the original commit from durable state.

From inference runtime to agent state runtime

The task trace exposes an architectural boundary: inference can finish while the agent’s work remains unresolved. Something must own the task’s state, authority and recovery across model calls and external actions.

Workflow engines such as Temporal already provide persistence and recovery; databases and retrieval systems supply other pieces. An agent state runtime coordinates them:

Agent / application
Agent framework
Agent state runtimeMemory · Transactions · Versioned state · Concurrency · Checkpoints / recovery · Identity / authority · Provenance / governance
Inference runtimeKV / Cache · Scheduling · Routing · OptimizationGPU / NPU / CPU · HBM / DRAM
Tools / external systemsAPIs · Payments · SaaS · DevicesWorld state

The state runtime spans inference and tools, recording intent, outcomes and compensation across their separate failure boundaries.

Context assembly connects the two runtimes: durable state becomes a selected working set, then inference materializes KV state. HBM and ordinary DRAM remain volatile; persistence and recovery belong to the state layer.

The bottom line

The optimizer chooses how a task should execute. The state runtime preserves what it read, what it was authorized to do and what actually committed.

Inference is rediscovering the database execution engine. Agentic AI is beginning to rediscover the transaction manager, recovery system, security model and workload governor.

For database and systems engineers

If this feels familiar, that's the point. AI infrastructure is rediscovering problems you spent decades solving: memory management, scheduling, optimization, transactions, concurrency and recovery.

Your experience belongs at the center of the inference and agentic stack.

Next, even closer to home: distributed state, replication, consistency and conflict resolution.

Next: every agent succeeds, and the system still gets it wrong

Your phone remembers “no red-eye flights.” Your laptop books one from yesterday’s preferences. Both operations commit successfully.

You tell the cloud agent to forget a private fact. An offline device reconnects and puts it back into memory.

A plant’s operating limit changes in the cloud. The disconnected edge agent keeps acting under the old limit.

No retry policy fixes these failures. Each runtime can execute correctly against its own state while the overall system violates your intent.

Your agent has a transaction manager. Now it has three copies of the truth.

Part 3: when agent state is distributed and replicated, which version can an agent trust—and what must it stop doing when it cannot know?