Observability & traceability
Apache-2.0 platform from Comet that records every LLM call, tool invocation and agent step as traces, monitors cost, latency, error rates and feedback scores, scores incoming traces with online rules, and converts problem traces into test cases. The full backend can be self-hosted; RBAC and audit logs are not documented.
hybrid · generally available · Research snapshot 2026-09-06
Visit the official product source ↗Observability & traceability · Evaluation & testing
Useful conversation with: AI engineer, ML platform lead.
Show me a self-hosted deployment where every tool invocation in an agent run is logged and automatically scored by an online evaluation rule.
Support labels reflect the supplied research. Documentation and vendor claims are not independent product tests. “Not established” means the researcher did not find support; it does not prove a capability is absent.
Documented by provider
Opik records every LLM call, tool invocation and agent step, providing a complete log of every request and response, with dashboards for cost, latency, error rates and feedback scores and token cost tracking.
Limit: Docs do not use span terminology or describe multi-agent handoff modelling; retention behaviour is unstated.
Documented by provider
Online evaluation rules automatically score incoming traces using LLM-as-a-judge and heuristic metrics, and a problematic trace can be turned into a test case added via UI or SDK and run as a suite.
Limit: No documented approval workflow or gate tied to those scores.
Documented by provider
The repository states the full platform including server backend, web application, tracing, datasets, experiments, evaluations, prompt management, online evaluation and agent optimization components is Apache-2.0 licensed and free to self-host with no data leaving the environment, and includes Opik Guardrails.
Limit: No RBAC, SSO, SCIM, audit log, retention or PII-masking features are stated in the repository README.
Listing does not imply partnership, supplier status, a working DutyGraph integration, or a compliance certification.
Suggest a correction