Best LLM observability tools in 2026
The week after launch, someone asks why the agent said what it said. These four tools are how you answer without guessing.
The short answer
Langfuse is the strongest default for most teams, because it is open source, self-hostable, and framework neutral. LangSmith is the better pick if you already build on LangChain or LangGraph and want a managed platform that matches your stack.
Helicone wins on setup speed if the system is a single LLM call behind a proxy. Braintrust earns its place when observability and evaluation need to share one dataset.
We instrument client agents before they reach production, then keep watching after. This page covers runtime monitoring only. Pre-launch evaluation tools are ranked separately.
How this ranking was made
Verified
We ranked on what each tool captures about a live request, and how much a team can act on without building their own pipeline. Tracing nobody looks at again is not observability. So we weighted retention, search and alerting over raw feature counts.
Capabilities were read from each vendor's own documentation on 14 August 2026 and are linked in each entry. We instrument client AI systems in production, and that operational need sets the criteria below.
Generic application performance monitoring tools were excluded. They track latency and errors, not what a model retrieved or replied, and that gap is exactly the failure this page is about.
- Full trace capture
- Whether it records the input, retrieved context, tool calls and reply as one linked record, instead of only the final answer.
- Self-hosting
- Whether the traces can stay inside your own infrastructure, which matters the moment they hold customer data.
- Framework fit
- How much wiring it takes to instrument a stack that is not LangChain.
- Path to evaluation
- Whether a traced failure can become a test case in the same tool, or has to move somewhere else.
The four compared
Read from vendor documentation on 14 August 2026.
| Tool | Shape | Self-hostable | Pick it when |
|---|---|---|---|
| Langfuse | Open source platform | Yes | You want tracing you control end to end |
| LangSmith | Hosted platform | No | You already build on LangChain or LangGraph |
| Helicone | Proxy-based logging | Yes | You want logging with almost no code change |
| Braintrust | Hosted platform | No | Observability and evals need to share one dataset |
The ranking
Ordered by how many production teams each one suits as a starting point.
Open-source tracing you can run yourself
Start here if your stack is not LangChain, or if traces might hold data you cannot send to a third party. Langfuse's documentation describes tracing for any LLM app, plus prompt management and evaluation scoring in the same product.
Being open source and self-hostable is the real differentiator. A team in insurance or healthcare can run it inside their own network and never let a customer document leave their infrastructure. The hosted version exists too, for teams who would rather not run it.
The trade-off is that self-hosting is a service you now operate. Someone has to patch it, back it up and watch its own uptime, which is a real cost even when the license is free.
Best for
- Teams that cannot send traces to a third-party platform
- Stacks built outside LangChain that still want full tracing
- Anyone who wants prompt management and evaluation in one place
Not for
- Teams with no appetite to run and patch another service
- Projects wanting zero setup on day one
- License
- Open source, self-host or hosted
- Also does
- Prompt management, evaluation scoring
The managed platform built for LangChain and LangGraph
Pick this when your agent already runs on LangChain or LangGraph. LangSmith's documentation covers tracing every run as a tree of the calls, tools and retrieval steps involved. That lines up with how those frameworks structure a request.
Instrumentation is close to automatic if you are already in that ecosystem, which is the whole appeal. It also supports online evaluation on live traffic, so a trace can be scored without you writing that pipeline.
Outside LangChain, the automatic wiring goes away. You are instrumenting by hand like any other tool, and Langfuse's open licensing becomes the stronger argument.
Best for
- Teams already built on LangChain or LangGraph
- Products wanting online evaluation on live traffic without wiring it themselves
Not for
- Stacks with no LangChain in them
- Teams that cannot send traces to a third-party platform
- Shape
- Hosted platform
- Strongest fit
- LangChain, LangGraph
Proxy-based logging with almost no code change
Reach for this when the system is close to a single LLM call and you want logging live today. Helicone's documentation describes routing requests through its proxy, which captures the request and response without instrumenting your code by hand.
That proxy model is also the limit. A multi-step agent with several tool calls needs a trace of the whole sequence, not a log of one request. A single proxied call cannot show you that.
It is the fastest of the four to turn on. That is the right trade for an early product, or a single feature that just needs someone watching it.
Best for
- A single LLM call or simple chat feature
- Teams wanting logging live in an afternoon
Not for
- Multi-step agents with tool calls and retrieval
- Teams needing a full trace tree, not a request log
- Setup
- Proxy, minimal code change
- Best fit
- Single-call or simple chat features
Observability and evaluation sharing one dataset
Choose this when you want a trace you did not like to become a test case with one click. Braintrust's documentation ties logging and evaluation to the same dataset, so a production failure and a regression test live in the same place.
That link is the reason to pick it over a pure observability tool. Most teams find their best test cases in production traces anyway. Braintrust removes the export step between finding one and using it.
It is hosted only, with no self-hosted option. That raises the same data-residency question that rules LangSmith out for some teams.
Best for
- Teams who want production failures to feed evaluation directly
- Products iterating on prompts against a growing dataset
Not for
- Teams that cannot send traces to a third-party platform
- Anyone wanting observability with no evaluation features attached
- Shape
- Hosted platform
- Also does
- Evaluation, sharing the same dataset
- InputThe request, and who sent it.
- RetrievedPassages or records the model used.
- Tool callsWhat the agent did, in order.
- OutputThe reply, the model, the token count.
- OutcomeAccepted, edited, escalated or disputed.
Teams that log only the output can prove something happened. They cannot prove why.
When observability alone will not save you
A trace tells you what happened. It does not tell you whether that was correct, and the two questions get confused often enough to name separately.
Pairing every trace with a definition of a good answer is what evaluation does, and it belongs before launch, not after. Our roundup of evaluation tools covers that half, scored on a different set of criteria than this page.
- 01No alerting on a traced tool means someone still has to remember to look.
- 02A trace with no retention policy becomes a customer-data liability nobody decided on.
- 03Watching traces after a bad answer ships is monitoring, not prevention. Pair it with evaluation before launch.
A system where every AI decision had to be reconstructable
“I am extremely happy with the results and would highly recommend Hashlogics to anyone.”
Daniel Khin · CEO, PremiumAudit.io
Questions teams ask
01Is LangSmith or Langfuse the better starting point?
LangSmith if your agent already runs on LangChain or LangGraph and you want tracing that matches the framework with little setup. Langfuse if you want an open-source, self-hostable option that works with any stack. Data residency is usually the deciding factor.
02Do we need observability if we already run evals before launch?
Yes. Evals check a fixed set of cases before you ship. Observability watches live traffic, where the questions and documents drift in ways no pre-launch test set predicted. They answer different questions and neither replaces the other.
03What is the minimum we should log on every request?
The input, what was retrieved, and the final reply, linked as one record. That triplet lets you tell whether a wrong answer came from bad retrieval or bad reasoning, which is the first fork in almost every investigation.
04Can we self-host if data cannot leave our network?
Langfuse and Helicone both support self-hosting, based on their published documentation. LangSmith and Braintrust are hosted only. Check each vendor's current terms before committing, since hosting options change.
05Why isn't this page about evaluation tools too?
Because they answer different questions and mixing them hides both. This page covers watching production. Our separate roundup of LLM evaluation tools covers testing before launch, which needs its own criteria.
Related reading
- Best LLM evaluation tools →Testing before launch, ranked on a different set of criteria.
- What LLM observability is →The full definition and why ordinary monitoring misses this failure.
- How do you know if an AI agent is production ready →The checklist buyers should be asking for, observability included.
