What is LLM tracing?
An agent gives a wrong answer after nine tool calls. Without a trace, you are re-running the whole chain and guessing which step lied.
LLM tracing
agent tracing
LLM tracing is the practice of recording every step of a multi-call AI run as one connected record, called a trace. That includes prompts, retrieved documents, tool calls and token counts. Each step in the record is a span, and spans nest to show which call triggered which.
A single prompt-and-response pair barely needs tracing. You can read both and see what happened. An agent is a different problem. It retrieves a document, calls two tools, and re-prompts itself based on the result. The final answer sits five or six decisions removed from the input, and any one of them could be where it went wrong.
A trace fixes that by keeping the full path. Open one and you see the exact prompt sent at each step. You see the tool that ran, its arguments, the tokens it cost, and how long it took. Nothing is reconstructed after the fact. It is the record of what actually ran.
Tracing turns debugging into reading, not guessing
Without a trace, a bad answer sends someone back to the start of the pipeline. They re-run the same input and hope to spot the fault by eye. That works for a two-step chain. It falls apart once an agent branches, calls tools conditionally, or retries a failed step on its own.
A trace turns that search into a read. Open the run and find the span where the document was empty, or the tool returned the wrong field. You know the cause in minutes, not a re-run in the dark. The same record answers a cost question just as fast: which step burned the tokens, and which one is worth swapping to a cheaper model.
It also gives support and compliance teams something to point to. Someone asking why the system decided a thing gets the actual sequence of calls. Not a description of how the system is supposed to behave.
- 01Trace every run in production, not just the ones that fail. You do not know it failed until someone reads the trace.
- 02Nest spans by call, so a retry or a branch shows up as a child of the step that triggered it.
- 03Keep the retrieved content and tool arguments in the trace, not just the final text. The failure is usually upstream of the last call.
- PromptThe instruction sent to the model.
- RetrieveDocuments pulled, and what they contained.
- Tool callArguments in, result out.
- Re-promptModel reasons over the tool result.
- AnswerFinal output, tied to every step above.
Each node is a span. The trace is the whole chain, kept together.
Common questions
01How is LLM tracing different from normal application logging?
Application logs record events in a flat timeline. A trace connects every step of one run into a single nested record, so you can see which tool call happened inside which prompt. Tracing tools also capture prompt and completion content, which general logging usually strips out.
02Does every LLM feature need tracing?
A single prompt with no tools or retrieval rarely needs it, since the input and output are the whole story. Tracing earns its keep once a run has more than one call. That means retrieval plus generation, a tool the model can invoke, or an agent that decides its own next step.
03What is a span, exactly?
A span is one recorded step inside a trace: a single model call, a single tool invocation, or a single retrieval. Spans carry a start time, an end time, and the inputs and outputs that matter for that step. They nest under the call that triggered them.
04Does tracing slow down an agent in production?
Recording a trace adds the cost of writing structured data alongside a call the system makes anyway. That cost is small next to the model call itself. The bigger cost is storage over time, so most teams sample or trim older traces rather than skip tracing altogether.
