Best LLM observability tools (2026)
The week after launch, someone asks why the agent said what it said. These four tools are how you answer without guessing.
The short answer
Langfuse is the best LLM observability tool for most teams, because it's open source, self-hostable and framework neutral. LangSmith is the better pick if you already build on LangChain or LangGraph.
Helicone wins on setup speed if the system is a single LLM call behind its gateway. Braintrust earns its place when observability and evaluation need to share one dataset. Arize Phoenix is the open-source alternative built on OpenTelemetry.
We instrument client agents before they reach production, then keep watching after. This page covers runtime monitoring only. Pre-launch evaluation tools are ranked separately.
Key takeaways
3 things that decide this
- 01Langfuse is the best LLM observability tool for teams that need traces to stay inside their own network. LangSmith wins for teams already on LangChain or LangGraph.
- 02We ranked five tools on trace capture, self-hosting, framework fit and the path to evaluation. We read each vendor's docs on 11 November 2026.
- 03Before you pick one, check that a single trace links the input, the retrieved context, every tool call and the reply.
How we ranked these LLM observability tools
Verified
We ranked on what each tool captures about a live request, and how much your team can act on without building its own pipeline. Tracing nobody looks at again isn't observability, so we weighted retention, search and alerting over raw feature counts.
For every entry we read the vendor's own docs on 11 November 2026, linked in each entry. We checked how it traces, how it self-hosts, and how a trace becomes an eval.
Generic application performance monitoring tools were left out. They track latency and errors, not what a model retrieved or replied, and that gap is exactly the failure this page is about.
Hashlogics publishes this page. We build custom software and are not one of the tools ranked here. No company paid for its place.
- Full trace capture
- Whether it records the input, retrieved context, tool calls and reply as one linked record, instead of only the final answer.
- Self-hosting
- Whether the traces can stay inside your own infrastructure, which matters the moment they hold customer data.
- Framework fit
- How much wiring it takes to instrument a stack that is not LangChain.
- Path to evaluation
- Whether a traced failure can become a test case in the same tool, or has to move somewhere else.
The five compared
Read from vendor documentation on 11 November 2026.
| Tool | Shape | Self-hostable | Pick it when |
|---|---|---|---|
| Langfuse | Open source platform | Yes | You want tracing you control end to end |
| LangSmith | Hosted platform | Enterprise add-on only | You already build on LangChain or LangGraph |
| Helicone | Gateway-based logging | Yes | You want logging with almost no code change |
| Braintrust | Hosted platform | Data plane only (hybrid) | Observability and evals need to share one dataset |
| Arize Phoenix | Open source, OpenTelemetry-based | Yes | You want open tracing on OpenTelemetry |
The ranking
Ordered by how many production teams each one suits as a starting point. Each entry was checked on the vendor's own docs on 11 November 2026.
Open-source tracing you can run yourself
Start here if your stack is not LangChain, or if traces might hold data you cannot send to a third party. Langfuse's documentation describes tracing for any LLM app, plus prompt management and evaluation scoring in the same product.
Being open source and self-hostable is the real differentiator. A team in insurance or healthcare can run it inside their own network and never let a customer document leave their infrastructure. The hosted version exists too, for teams who would rather not run it.
Self-hosting is a service you now operate, and that is the trade-off. Someone has to patch it, back it up and watch its own uptime, which is a real cost even when the license is free.
Best for
- Teams that cannot send traces to a third-party platform
- Stacks built outside LangChain that still want full tracing
- Anyone who wants prompt management and evaluation in one place
Not for
- Teams with no appetite to run and patch another service
- Projects wanting zero setup on day one
- License
- Open source, self-host or hosted
- Also does
- Prompt management, evaluation scoring
The managed platform built for LangChain and LangGraph
Pick this when your agent already runs on LangChain or LangGraph. LangSmith's documentation covers tracing every run as a tree of the calls, tools and retrieval steps involved. That lines up with how those frameworks structure a request.
Instrumentation is close to automatic if you are already in that ecosystem, which is the whole appeal. It also supports online evaluation on live traffic, so a trace can be scored without you writing that pipeline.
Outside LangChain, the automatic wiring goes away. You are instrumenting by hand like any other tool, and Langfuse's open licensing becomes the stronger argument.
Best for
- Teams already built on LangChain or LangGraph
- Products wanting online evaluation on live traffic without wiring it themselves
Not for
- Stacks with no LangChain in them
- Teams that need self-hosting without an enterprise contract
- Shape
- Hosted platform
- Strongest fit
- LangChain, LangGraph
Proxy-based logging with almost no code change
Reach for this when the system is close to a single LLM call and you want logging live today. Helicone's quick-start routes requests through its AI gateway by changing the base URL, which captures each request and response without hand instrumentation.
The gateway captures the model calls with no extra work. To see a multi-step agent as one trace, you add Helicone session IDs to each step and log the tool and retrieval calls yourself. That's the instrumentation work the gateway was meant to save you.
It is the fastest of the four to turn on. That is the right trade for an early product, or a single feature that just needs someone watching it.
Best for
- A single LLM call or simple chat feature
- Teams wanting logging live in an afternoon
Not for
- Multi-step agents where you do not want to instrument tool and retrieval calls by hand
- Teams that want a full trace tree with no code changes
- Setup
- Gateway, minimal code change
- Best fit
- Single-call or simple chat features
Observability and evaluation sharing one dataset
Choose this when you want a trace you did not like to become a test case with one click. Braintrust's documentation ties logging and evaluation to the same dataset, so a production failure and a regression test live in the same place.
That link is the reason to pick it over a pure observability tool. Most teams find their best test cases in production traces anyway. Braintrust removes the export step between finding one and using it.
Self-hosting is hybrid: the data plane runs in your own AWS, GCP or Azure account, while the UI and authentication stay with Braintrust. The management plane stays with Braintrust, which is the same shape of question an enterprise LangSmith deployment raises.
Best for
- Teams who want production failures to feed evaluation directly
- Products iterating on prompts against a growing dataset
Not for
- Teams whose policy rules out any vendor-run control plane, even with data kept in their own cloud
- Anyone wanting observability with no evaluation features attached
- Shape
- Hosted platform
- Also does
- Evaluation, sharing the same dataset
Open-source tracing and evals built on OpenTelemetry
Phoenix is Arize's open-source tracing and evaluation tool. It takes traces over OpenTelemetry, so your instrumentation isn't tied to one vendor, and its docs cover self-hosting with Docker.
It ranks fifth because its source license is the Elastic License 2.0 rather than a permissive one, and its managed path leads into Arize's wider platform. If self-hosting is the deciding factor, compare it directly with Langfuse.
Best for
- Teams standardising on OpenTelemetry across their whole stack
- Teams that want tracing and evals in one open tool
Not for
- Teams whose policy requires a permissive open-source license
- License
- Elastic License 2.0, self-hostable
- Tracing
- OpenTelemetry
- InputThe request, and who sent it.
- RetrievedPassages or records the model used.
- Tool callsWhat the agent did, in order.
- OutputThe reply, the model, the token count.
- OutcomeAccepted, edited, escalated or disputed.
Teams that log only the output can prove something happened. They cannot prove why.
When observability alone will not save you
A trace tells you what happened. It does not tell you whether that was correct, and the two questions get confused often enough to name separately.
Pairing every trace with a definition of a good answer is what evaluation does, and it belongs before launch, not after. Our roundup of evaluation tools covers that half, scored on a different set of criteria than this page.
Dashboards show the failure. Fixing the retrieval or prompt that caused it is engineering, the part our RAG development team owns on production systems.
- 01No alerting on a traced tool means someone still has to remember to look.
- 02A trace with no retention policy becomes a customer-data liability nobody decided on.
- 03Watching traces after a bad answer ships is monitoring, not prevention. Pair it with evaluation before launch.
A system where every AI decision had to be reconstructable
“I am extremely happy with the results and would highly recommend Hashlogics to anyone.”
Daniel Khin · CEO, PremiumAudit.io
Need every AI decision traceable before launch?
We wire tracing into the agent while we build it, not after a customer asks why it did something. Scoping calls cost nothing.
Questions teams ask
01What is the best LLM observability tool?+
Langfuse is the best LLM observability tool for most teams, because it's open source, self-hostable and works with any stack. LangSmith is the better pick if you already build on LangChain or LangGraph.
02Is LangSmith or Langfuse the better starting point?+
LangSmith if your agent already runs on LangChain or LangGraph and you want tracing that matches the framework with little setup. Langfuse if you want an open-source, self-hostable option that works with any stack. Data residency is usually the deciding factor.
03Do we need observability if we already run evals before launch?+
Yes. Evals check a fixed set of cases before you ship. Observability watches live traffic, where the questions and documents drift in ways no pre-launch test set predicted. They answer different questions and neither replaces the other.
04What is the minimum we should log on every request?+
The input, what was retrieved, and the final reply, linked as one record. That triplet lets you tell whether a wrong answer came from bad retrieval or bad reasoning, which is the first fork in almost every investigation.
05How do I compare LLM observability tools fairly?+
Instrument the same agent run in each tool and break one step on purpose. Then see how fast you can find the failed tool call, what it retrieved, and whether you can turn that trace into a test case.
06Can we self-host if data cannot leave our network?+
Langfuse, Helicone and Arize Phoenix all document self-hosting. LangSmith self-hosts only as an Enterprise add-on. Braintrust runs its data plane in your own cloud and keeps its UI managed. Check each vendor's current terms before you commit, since hosting options change.
07Why isn't this page about evaluation tools too?+
Because they answer different questions and mixing them hides both. This page covers watching production. Our separate roundup of LLM evaluation tools covers testing before launch, which needs its own criteria.
08How were these LLM observability tools ranked?+
We ranked five tools on trace capture, self-hosting, framework fit and the path to evaluation, read from vendor docs on 11 November 2026. Hashlogics publishes this page and builds custom software, not one of these tools. No company paid for its place.
09Why isn't Datadog or another APM tool on this list?+
APM tools track latency and errors, not what a model retrieved or replied. Several now add LLM features, but this page ranks tools built around the LLM trace itself.
