span-level replay is live · apache-2.0 · open source

the real debugger
for agents

trace, replay, eval — every signal in one place.
open a broken run, edit it, re-run it, and diff what changed.

start self-hosting
$docker compose up -d
lp
langprobe ai 2:43am
likely cause: web.search timeout is 5s but p95 latency is 6.2s. replay with 30s?
replay it dismiss
replay diff · rep_c11d
web.search error→ok
determinism 9/10 stable
acme-prod/runs/run_9f3ab2
error
replay
spans · 7
chain agent.run 12.4s
llm claude-sonnet-4 1.82s
retr vectordb.query 88ms
tool web.search 5.0s
chain subagent.plan 4.1s
llm claude-sonnet-4 2.31s
tool code.exec 1.4s
0s3s6s9s12s
span_04 ToolTimeout: web.search exceeded 5s deadline replay with edits →
no proprietary sdk — plain otlp/http from the frameworks you already use
CrewAI DSPy Pydantic AI OpenAI Agents LlamaIndex LangChain OpenTelemetry Anthropic CrewAI DSPy Pydantic AI OpenAI Agents LlamaIndex LangChain OpenTelemetry Anthropic
replay

edit. re-run.
diff.

the debugger you reach for at 2am. change a prompt, model, or tool config and re-run against a real model — see exactly what changed, span by span.

span-level diffs — outputs, latency, tokens, cost, side by side.
determinism verdict — know if the fix is real or a lucky sample.
eval it — score the replay with the same judges as production.
replay diff rep_c11d vs run_9f3ab2 deterministic 9/10
span_04 · tool.config
- timeout_s: 5
+ timeout_s: 30
spanoriginalreplay
web.search error ok
agent.run error ok
claude · mcp: langprobe
▸ runs.search {"status":"error","last":"1h"}
  → run_9f3ab2 · agent.run · error · 48,213 tok
▸ runs.read {"id":"run_9f3ab2","budget":2000}
  → salient slice: 6 spans · web.search timeout at span_04
▸ replay.edit {"span":"span_04","timeout_s":30}
  → replay rep_c11d queued
▸ replay.diff {"id":"rep_c11d"}
  → 1 span changed · error → ok · deterministic 9/10
agent-first · mcp

an agent can debug an agent

every read is a token-budgeted, llm-legible projection over rest and mcp. a 48k-token trace becomes a 2k-token salient slice — find, read, replay, diff, hands-free.

rest + mcp surface — same api for humans, ci, and coding agents.
token-budgeted views — salient slices, never raw dumps.
replay → diff → eval — the agent’s debug loop, as api calls.
eval rigor
judge your judges

schema adherence, test-retest stability, inter-judge agreement — before you trust a score.

test-retest
.91
inter-judge κ
.76
migration
leaving langsmith?

ingestion is langsmith-compatible. change one env var; history backfills automatically. migration guide →

- ENDPOINT=api.smith...
+ ENDPOINT=probe.acme.dev
self-host
your vpc, your data

postgres + clickhouse + redis, one compose file. helm chart and operator for kubernetes.

$ docker compose up -d
✓ langprobe ready on :3000

common questions

how langprobe drops into your stack

01 how is this different from langsmith or langfuse? −

those are dashboards for humans. langprobe adds two things they don’t do: replay — edit a prompt, model, or tool config, re-run, and diff span by span with a determinism verdict — and an agent-legible surface over rest and mcp, so an agent can debug an agent.

02 do i need a proprietary sdk? +
03 can i migrate from langsmith? +
04 what does self-hosting require? +
05 is there a hosted version? +

debug the next broken run
properly

in your vpc, on your data, apache-2.0.
free to start — it’s open source.

star on github
$docker compose up -d