AI Agent Observability

AI agent observability answers three questions for every run: what did it cost, how long did it take, and what did it actually do? agentFast records all of it as a structured step tree — so you can prove ROI, debug a failure, and catch a cost regression before it hits a bill.

Every run, as a step tree with cost and latency
AI agent observability: every run is a step tree with per-step tokens, cost and latencyEach run renders as an ordered tree of LLM calls, tool calls, memory recalls and human-in-the-loop pauses, each with its own cost, latency and prompt-cache accounting.run_8f21 · step treelatencycostllmplan$0.004toolkb_search$0.000memoryrecall$0.000llmdecide$0.006toolorder_lookup$0.000hitlapproval: refund—llmanswer$0.003total $0.013 · cache read 4.2k · cache write 1.1kOTEL · LangSmith export
What you get on every run

Every run is a step tree

An ordered tree of LLM calls, tool calls, memory recalls, guardrail interventions and HITL pauses — rendered in the dashboard.

Cache-aware cost accounting

Per-step cost against a configurable pricing table, counting prompt-cache reads and writes separately — the line item that dominates long runs.

Canonical telemetry record

Each run rolls up: complexity, total cost, cache tokens, LLM vs tool latency, tool errors, guardrail interventions, PII redactions and eval outcome.

OpenTelemetry export

Full span-tree export to any OTLP backend, with deterministic span timing and attributes.

First-class LangSmith

A real LangSmith exporter following its project/api-key conventions — not just a generic OTLP endpoint.

Redacted by construction

Traces are written after the guardrail pipeline, so PII never lands in a stored span in the clear.

Cost you can actually trust
Most naive trackers miss prompt-cache accounting entirely, then under-report the bill on exactly the long, tool-heavy runs where it matters most. agentFast splits cache reads and writes out per step, so the number you see is the number you pay.

Observability pairs with evals to turn a prompt change from a gamble into a measurement, and with guardrails so every intervention is a visible step. It ships on every SDK — start with the OpenAI Agents SDK or LangGraph.

Stop rebuilding the plumbing.

agentFast is the production layer — memory, 50 tools, observability, evals, guardrails, human-in-the-loop — for LangGraph, CrewAI, the Claude & OpenAI Agent SDKs & Vanilla. Own it for life.

Get agentFast — from $89
Keep reading
The Production Layer Every AI Agent Needs (and Why LangGraph Doesn't Give It to You)
9 min read
LangGraph vs CrewAI vs a Vanilla Loop: Which Should You Actually Use in 2026
11 min read
How to Add Human-in-the-Loop Approval to Any AI Agent (With Durable Execution)
10 min read