Alphabetical, not ranked. Membership includes primary and secondary research categories. These products have different scopes; inspect the evidence profile before comparing capabilities.
Also covers this layer
Open-source proxy that carries service, LLM-provider, MCP and agent-to-agent traffic in one data plane, letting platform teams route agent tool calls through a single enforcement and audit point. Project documentation covers MCP proxying and routing configuration; tool-level RBAC and token exchange are described mainly on vendor product pages.
open_source · Research snapshot 2026-09-06
- Ask for a demonstration
- Show me an MCP server proxied through agentgateway with tool-level RBAC denying one specific tool call, and the audit record it produces.
Read sources and limitations →
Primary category
MIT-licensed monitoring platform for AI agents that records each program execution as a session and visualises a waterfall of LLM calls, action events, tool calls and errors, with prompt and completion inspection, event-type breakdowns, timings and LLM spend tracking. Self-hostable; no documented access control or retention.
hybrid · Research snapshot 2026-09-06
- Ask for a demonstration
- Show me the session waterfall for one agent execution, including tool calls, errors and the exact prompt and completion for a chosen LLM call.
Read sources and limitations →
Primary category
Commercial AI engineering platform that captures traces of agent runs including inputs, outputs, tools and costs, groups them into sessions, visualises agent trajectories, and runs online and offline evaluations with human annotation. Arize also maintains Phoenix, a local-first open-source tracing and evaluation package.
hybrid · Research snapshot 2026-09-06
- Ask for a demonstration
- Show me an agent trajectory visualisation for a failed multi-step run and the trace-level evaluation scores attached to it.
Read sources and limitations →
Also covers this layer
Registry and monitoring console within Boomi Agentstudio where organisations register, tag, monitor and manage AI agents across providers - agents built in Boomi and third-party environments such as Amazon Bedrock - via configured provider connections. Agents appear because a provider account is connected, so unconnected environments remain invisible.
commercial · Research snapshot 2026-09-06
- Ask for a demonstration
- Show me registering an Amazon Bedrock provider account and then disabling one of its agents directly from Agent Control Tower, including the trust level and tags applied.
Read sources and limitations →
Also covers this layer
Developer platform for scoring AI application output: teams log traces from their app, curate datasets from human feedback, run experiments against scorers, then watch production logs. Documentation covers project-level access control and a customer-controlled data plane, but audit logging and retention behaviour are not described.
commercial · Research snapshot 2026-09-06
- Ask for a demonstration
- Show me an experiment run comparing two prompt versions against a dataset built from human feedback, and who in my organization can view the underlying logs.
Read sources and limitations →
Also covers this layer
Model registry inside the open-source ClearML MLOps toolchain: models are logged automatically or manually from training tasks, catalogued per project with metadata and metrics columns, queryable by name, tag or metadata, with lineage and CI/CD triggers on registry events such as tagging or publishing.
hybrid · Research snapshot 2026-09-06
- Ask for a demonstration
- Demonstrate an automatic model registration from a training task, then trigger a CI/CD pipeline when that model is tagged for production.
Read sources and limitations →
Also covers this layer
Hosted quality platform from the DeepEval maintainers. It captures each LLM call as a trace with inputs, outputs, tool calls, latency, token cost and metadata, converts flagged traces into evaluation datasets, and runs metric-based regression tests on pull requests. Governance controls are not documented.
hybrid · Research snapshot 2026-09-06
- Ask for a demonstration
- Show me how a failing production trace becomes a dataset case that then blocks a pull request when the metric regresses.
Read sources and limitations →
Also covers this layer
Apache-2.0 gateway, registry and proxy from IBM that federates MCP servers, A2A agents and REST or gRPC APIs behind one endpoint, adding authentication, rate limiting, input validation and OpenTelemetry tracing. It centralises tool discovery for MCP clients but does not itself provide threat detection or sandboxed execution.
open_source · Research snapshot 2026-09-06
- Ask for a demonstration
- Show me ContextForge federating two MCP servers plus a REST API behind one endpoint, with a user-scoped OAuth token and rate limit applied to one tool.
Read sources and limitations →
Also covers this layer
Simulation and evaluation platform for voice and chat agents. It generates large volumes of simulated callers with accents, interruptions, noise and policy traps, scores production calls in real time on resolution and safety, detects regressions after prompt or model changes, and routes failures to human reviewers.
commercial · Research snapshot 2026-09-06
- Ask for a demonstration
- Demonstrate a pre-launch simulation of a thousand callers against my voice agent, then show the production scoring that flags a regression after a prompt change.
Read sources and limitations →
Also covers this layer
Open-source Python framework for orchestrating role-based agent crews, paired with CrewAI AMP, a hosted platform that deploys crews, exposes them via REST, streams events and shows execution traces. Human review is a framework-level pattern; platform documentation reviewed showed no approval console or audit-log feature.
hybrid · Research snapshot 2026-09-06
- Ask for a demonstration
- Demonstrate a crew deployed on CrewAI AMP where a task requires human review before the next agent runs, and show the execution trace for that run.
Read sources and limitations →
Also covers this layer
Telemetry-collection layer of CrowdStrike's AI Detection and Response product. Collectors capture AI activity from the Falcon sensor (browser extension plus network inspection of desktop AI apps and coding assistants), standalone browser extensions, application SDK calls, and supported API gateways. Browser collectors log detections in report-only mode for model responses.
commercial · Research snapshot 2026-09-06
- Ask for a demonstration
- Demonstrate registering a Falcon endpoint collector and show which desktop AI applications and coding assistants network inspection surfaced that the browser extension missed.
Read sources and limitations →
Also covers this layer
Runtime governance layer inside the Databricks platform that routes and controls requests to models, agents, MCP services and AI tools using Unity Catalog privileges, service policies, rate limits and spend caps. Controls are strongest for assets governed by Unity Catalog rather than arbitrary external agent traffic.
commercial · Research snapshot 2026-09-06
- Ask for a demonstration
- Show me Unity Gateway restricting an agent to an approved subset of tools on an MCP service, with the service policy that blocks a request and the resulting inference log.
Read sources and limitations →
Primary category
Datadog's AI observability product, documented as span kinds for LLM, workflow, agent, tool, task, embedding and retrieval. LLM spans record model and provider, session ID, inputs and outputs, tool calls, results and definitions, token and cost metrics, metadata and errors, inside an enterprise platform with audit trail, SAML SSO and access control.
commercial · Research snapshot 2026-09-06
- Ask for a demonstration
- Show me an agent span tree with tool calls and token cost metrics, and how retention plus RBAC apply to the captured prompt and response data.
Read sources and limitations →
Also covers this layer
Open-source platform for building agents, agentic workflows, chatbots and RAG pipelines that can draw on private data and ship as web apps or APIs, with LLMOps monitoring and third-party tracing integrations. Documentation reviewed showed no approval workflow, role-based access control or audit-log capability.
hybrid · Research snapshot 2026-09-06
- Ask for a demonstration
- Show me a Dify agentic workflow published as an API, and explain exactly what record exists of each model and tool call it made.
Read sources and limitations →
Also covers this layer
US NIST-built test platform for assessing trustworthy characteristics of AI models, providing a REST API, web interface and Python client to design, run and track reproducible experiments including adversarial red-team scenarios. Intended to support the Measure function of the AI Risk Management Framework rather than to certify systems.
open_source · Research snapshot 2026-09-06
- Ask for a demonstration
- Demonstrate reproducing an adversarial robustness experiment in Dioptra and exporting the tracked evidence for an internal AI risk review.
Read sources and limitations →
Also covers this layer
MIT-licensed gateway from Docker that acts as a proxy between MCP clients and MCP servers, running each catalogued local server in an isolated container with restricted privileges, network access and resources, injecting credentials centrally and applying per-profile tool allowlists. It provides no semantic inspection of prompts or tool arguments.
open_source · Research snapshot 2026-09-06
- Ask for a demonstration
- Show me the MCP Gateway running two catalogue servers in isolated containers with a profile tool allowlist that hides one tool, plus the call trace for a blocked request.
Read sources and limitations →
Also covers this layer
Apache-2.0 Python library plus self-hostable platform for evaluating and monitoring data and AI systems, including tabular data quality and drift tests with statistical methods, declarative test suites usable in CI/CD, and dashboards tracking metrics and test results over time for deployed models.
open_source · Research snapshot 2026-09-06
- Ask for a demonstration
- Show me a drift test suite failing in CI for a production model's input data, and how the result appears on the monitoring dashboard over time.
Read sources and limitations →
Primary category
Monitoring platform spanning traditional ML models, LLM applications and multi-agent systems. For LLM applications customers publish prompts, prompt context, responses and retrieved source documents; Fiddler generates trust and safety metrics, embeddings with UMAP visualisation and drift detection to support alerting and root-cause analysis.
commercial · Research snapshot 2026-09-06
- Ask for a demonstration
- Show me drift and trust-and-safety metrics for a RAG application, including the retrieved source documents behind a flagged response.
Read sources and limitations →
Also covers this layer
Google Cloud's renamed Vertex AI agent stack, combining a managed agent runtime with sessions, a memory bank, example store and evaluation service, and support for ADK, LangGraph, LlamaIndex and A2A agents. Governance evidence centres on registration through the Gemini Enterprise app and built-in tracing rather than approval workflows.
commercial · Research snapshot 2026-09-06
- Ask for a demonstration
- Demonstrate deploying an ADK agent to the managed runtime, registering it in Gemini Enterprise, and inspecting its traces in Cloud Trace.
Read sources and limitations →
Primary category
Apache-2.0 gateway and observability service that proxies LLM requests and logs them for inspection, with model routing across many providers. Capture is request/response oriented rather than agent-span oriented, and the pages reviewed document no RBAC, retention or redaction controls despite claiming SOC 2 and GDPR compliance.
hybrid · Research snapshot 2026-09-06
- Ask for a demonstration
- Show me the logged request and response for a specific model call routed through the gateway, and what data I can export via the API.
Read sources and limitations →
Primary category
Agent observability and evaluation platform aimed at enterprises running production agents. Distributed tracing captures every step including tool calls, prompts, retries, loops and handoffs between sub-agents across long trajectories, with step-by-step replay, online evals on live traffic and annotation queues that turn expert review into datasets.
commercial · Research snapshot 2026-09-06
- Ask for a demonstration
- Replay a multi-day agent trajectory step by step, showing every tool call and sub-agent handoff plus the evaluation scores attached to it.
Read sources and limitations →
Also covers this layer
IBM's AI governance offering combining a tracked model and prompt-template inventory (AI Factsheets) with monitors for fairness, drift, model health and generative-AI output risks. Suited to enterprises already on watsonx; evidence reviewed covers monitoring and inventory mechanics, not regulatory outcomes.
commercial · Research snapshot 2026-09-06
- Ask for a demonstration
- Demonstrate how a deployed model's factsheet, fairness and drift monitors, and alert thresholds appear in the shared model inventory for a risk reviewer.
Read sources and limitations →
Also covers this layer
Infosys implementation and advisory suite of more than ten offerings arranged as Scan, Shield and Steer, including regulatory watchtower monitoring, maturity and risk assessments, a responsible AI audit offering, and a control centre for compliance telemetry. Delivered as consulting plus proprietary assets; component depth is not documented publicly.
service · Research snapshot 2026-09-06
- Ask for a demonstration
- Demonstrate the Responsible AI Control Center on a live AI portfolio: which compliance signals it ingests and what a violation alert looks like end to end.
Read sources and limitations →
Also covers this layer
Gateway layer from Kong that proxies LLM provider traffic and remote MCP traffic, exposes existing APIs as MCP tools, and applies authentication, access control and rate limiting plugins to those calls. Documentation emphasises traffic governance; semantic guardrails and agent threat detection are not covered on the fetched pages.
hybrid · Research snapshot 2026-09-06
- Ask for a demonstration
- Show me an internal REST API exposed as an MCP tool through Kong, protected by the AI MCP OAuth2 plugin, with a rate limit and traffic log for each tool call.
Read sources and limitations →
Primary category
Apache-2.0 OpenTelemetry-native platform for AI agents that traces LLM calls, tool use, custom functions and parallel sub-agents, presenting runs as transcripts rather than span trees. It records and replays runs in a debugger, builds evaluation datasets from production traces, and runs evals in CI. No documented access control.
hybrid · Research snapshot 2026-09-06
- Ask for a demonstration
- Replay a recorded agent run in the debugger, swap the prompt, and show the side-by-side comparison plus the eval that would catch the regression in CI.
Read sources and limitations →
Primary category
German open-source AI engineering platform whose traces record LLM and non-LLM calls, retrieval, embeddings and API calls, grouped into sessions for multi-turn conversations and agent graphs. Paid tiers add audit logs, fine-grained RBAC, SSO/SCIM, retention management, client-side masking and EU, US or JP data regions.
hybrid · Research snapshot 2026-09-06
- Ask for a demonstration
- Show me a session trace for a multi-turn agent conversation, then show the audit log of who accessed or changed that project.
Read sources and limitations →
Primary category
LangChain's hosted platform for tracing LLM and agent applications. Traces record what agents did in production and feed evaluation datasets, online evaluation rules and annotation queues. Enterprise plans add role-based access control with workspace roles and custom organization roles; audit logging is not documented.
commercial · Research snapshot 2026-09-06
- Ask for a demonstration
- Show me a production trace for a failed agent run and demonstrate which workspace roles can view or edit that trace.
Read sources and limitations →
Also covers this layer
LangChain's runtime for running agents in production alongside the open-source LangGraph library. It provides durable execution, threads and runs, and middleware that pauses tool calls for human approval, edit or rejection. Governance beyond approvals and tracing is developer-configured rather than an administrative control surface.
hybrid · Research snapshot 2026-09-06
- Ask for a demonstration
- Show me an agent deployed on LangSmith Deployment pausing on a database-write tool call, and a reviewer editing the arguments before it resumes.
Read sources and limitations →
Primary category
Dutch open-source LLMOps platform that automatically tracks every LLM call, tool usage and user interaction as traces, spans and metadata, adds prompt versioning, dataset experiments and multi-turn agent testing against a simulated user. Self-hosted and enterprise editions document audit logs, RBAC and SSO/SCIM.
hybrid · Research snapshot 2026-09-06
- Ask for a demonstration
- Show me the full conversation trace for one agent session, then show the audit log entry for the user who changed the evaluation configuration.
Read sources and limitations →
Also covers this layer
AI governance platform aimed at regulated industries that replaces manual risk reviews with workflow-driven assessments, evaluates models across a large metric set, and adds monitoring plus guardrails. Public evidence is largely vendor-authored, including a 2024 UK government assurance-technique listing.
commercial · Research snapshot 2026-09-06
- Ask for a demonstration
- Show me which of the 200+ model metrics are computed automatically for a deployed model and how results feed the risk assessment workflow.
Read sources and limitations →
Also covers this layer
India-founded platform combining agent simulation, evaluation and production logging. Teams simulate multi-turn interactions across scenarios and personas, score runs with AI, programmatic or statistical evaluators plus human review pipelines, curate datasets from real interactions, and analyse production logs with distributed tracing.
commercial · Research snapshot 2026-09-06
- Ask for a demonstration
- Show me a simulation run across several user personas with evaluator scores per version, and the production logs for the same agent.
Read sources and limitations →
Also covers this layer
Admin control plane for AI agents in Microsoft 365. Its registry lists Microsoft, partner-built, org-published and user-shared agents available to a tenant, and a preview Shadow AI view detects unapproved local agents on managed devices using Defender for Endpoint, with extra usage metadata from Global Secure Access. Registry coverage is Microsoft-ecosystem centric.
commercial · Research snapshot 2026-09-06
- Ask for a demonstration
- Demonstrate the Shadow AI page detecting an unapproved local agent on a Defender-enrolled device and show which governance action removes its access.
Read sources and limitations →
Also covers this layer
Managed Microsoft platform for building, deploying and scaling agents, offering prompt agents configured in a portal, hosted container agents from frameworks such as Agent Framework and LangGraph, and direct Responses API use. Each hosted agent receives a dedicated Microsoft Entra identity plus managed endpoint, scaling and observability.
commercial · Research snapshot 2026-09-06
- Ask for a demonstration
- Show me a hosted agent deployed from a container that receives its own Entra identity, and the end-to-end trace of one of its runs.
Read sources and limitations →
Also covers this layer
Managed MCP gateway for enterprises that curates which MCP servers and tools each role may reach, issues per-agent credentials, and logs every tool call. An agent monitor captures file reads, command execution and tool calls so teams can write rules that block risky behaviour. Self-hosting requires vendor contact.
commercial · Research snapshot 2026-09-06
- Ask for a demonstration
- Show me two role-based MintMCP endpoints where a destructive tool is switched off for one role, plus the agent monitor rule that blocks a risky command.
Read sources and limitations →
Also covers this layer
Open-source component of MLflow providing a central model store with named registered models, versions, aliases, tags and lineage back to the producing run, used by ML teams as the promotion and rollback backbone. It provides mechanics for staging, not policy, risk assessment or regulatory mapping.
open_source · Research snapshot 2026-09-06
- Ask for a demonstration
- Demonstrate promoting a model version via alias with lineage back to its run, and show what audit information the registry retains after rollback.
Read sources and limitations →
Also covers this layer
Open-source Python library for post-deployment monitoring that estimates a model's performance when ground-truth labels are delayed or missing, using confidence-based estimation for classification and direct loss estimation for regression, and links univariate and multivariate drift alerts to performance impact.
open_source · Research snapshot 2026-09-06
- Ask for a demonstration
- Demonstrate estimated versus realised ROC AUC on a tabular classifier with delayed labels, and show which drift alerts were linked to the performance change.
Read sources and limitations →
Also covers this layer
Oasis places an access layer between AI agents and SaaS, cloud, on-prem and data systems: it converts each agent request into a structured intent, evaluates it against policy with optional human escalation, and provisions ephemeral least-privilege session identities with prompt-level audit trails. Enforcement depends on agents routing through Oasis.
commercial · Research snapshot 2026-09-06
- Ask for a demonstration
- Demonstrate an agent request converted into structured intent, denied by policy, then approved with human-in-the-loop and executed under a just-in-time session identity.
Read sources and limitations →
Primary category
Apache-2.0 set of OpenTelemetry extensions from Traceloop that instrument GenAI applications non-intrusively in Python, TypeScript or Go and export traces either to Traceloop or to an existing observability backend. It is instrumentation plumbing: no access control, retention or evaluation features are part of the library.
open_source · Research snapshot 2026-09-06
- Ask for a demonstration
- Demonstrate exporting GenAI traces from my agent to my own OpenTelemetry backend, showing the workflow, task, agent and tool annotations.
Read sources and limitations →
Primary category
Apache-2.0 platform from Comet that records every LLM call, tool invocation and agent step as traces, monitors cost, latency, error rates and feedback scores, scores incoming traces with online rules, and converts problem traces into test cases. The full backend can be self-hosted; RBAC and audit logs are not documented.
hybrid · Research snapshot 2026-09-06
- Ask for a demonstration
- Show me a self-hosted deployment where every tool invocation in an agent run is logged and automatically scored by an online evaluation rule.
Read sources and limitations →
Also covers this layer
Commercial platform built on the Conductor durable orchestration engine. Conductor Agents are authored in code or imported from frameworks, compiled into versioned workflow definitions, and composed with branching, schedules, human approval and cancellation, with each LLM call and tool invocation visible in the UI and API.
hybrid · Research snapshot 2026-09-06
- Ask for a demonstration
- Show me a Conductor Agent deployed at a pinned version, invoked from a production workflow, with each LLM call and tool invocation visible in the execution view.
Read sources and limitations →
Also covers this layer
Vendor offering managed evaluators plus simulation environments for agent testing: hosted judges score hallucination and unsafe output, red-teaming algorithms probe for weaknesses, and simulated digital workflows exercise long-horizon agent tasks. Public pages document scoring and simulation but not audit records, retention or access control.
commercial · Research snapshot 2026-09-06
- Ask for a demonstration
- Demonstrate a simulated multi-step workflow run where my agent is scored for hallucination and unsafe output, and show what evaluation evidence I can export.
Read sources and limitations →
Also covers this layer
Palo Alto Networks security service that scans prompts and model responses via API or network enforcement to detect prompt injection, sensitive data leakage and malicious content, with agent-focused detections such as MCP threat detection, tool chaining attack analysis and privilege misuse. Delivered as a managed enterprise service.
commercial · Research snapshot 2026-09-06
- Ask for a demonstration
- Show me Prisma AIRS scanning an agent request through the AI Runtime API, flagging a tool chaining attack and a prompt injection, and the security profile that blocked it.
Read sources and limitations →
Also covers this layer
Securiti's pipeline product for building enterprise AI systems on governed data. It catalogs unstructured sources, redacts or masks sensitive values inside AI pipelines, loads permission-aware embeddings into vector databases, and tracks lineage from source file to embedding. Detailed technical documentation was not publicly reachable during review.
commercial · Research snapshot 2026-09-06
- Ask for a demonstration
- Show me a Gencore AI pipeline where a retrieval answer changes because the requesting user lacks entitlements on the underlying SharePoint file.
Read sources and limitations →
Also covers this layer
Teleport issues short-lived certificates, JWTs and SPIFFE-compatible identities to bots, CI jobs, workloads and AI agents via its tbot agent, then enforces role-based access and records every command an agent runs against SSH hosts, Kubernetes clusters, databases and MCP servers. Agent use cases are positioned around design partnerships.
hybrid · Research snapshot 2026-09-06
- Ask for a demonstration
- Show an AI agent issued its own Teleport identity, restricted to read-only Kubernetes access by RBAC, with the full session recording of its commands.
Read sources and limitations →
Also covers this layer
Durable execution platform used to run stateful agent loops that call models and tools, wait on human decisions for days, and resume exactly where they stopped after failures. It supplies reliability primitives and full execution history rather than AI-specific policy, review consoles or data controls.
hybrid · Research snapshot 2026-09-06
- Ask for a demonstration
- Demonstrate an agent workflow that pauses for multi-day human approval via a Signal, survives a worker crash, and replays its full execution history.
Read sources and limitations →
Also covers this layer
UiPath's cloud orchestration layer that coordinates AI agents, RPA robots and people in BPMN, Flow or case-management models, with DMN decision tables. Governance is process-centric: role-based access, versioning and auditing of process and decision changes, rather than per-tool-call approval policy for agents.
commercial · Research snapshot 2026-09-06
- Ask for a demonstration
- Demonstrate a Maestro BPMN process where an agent step hands an exception to a person, and show the versioned, audited history of the decision table change.
Read sources and limitations →
Also covers this layer
Orchestration environment in Verint's CX platform for building and running a mixed workforce of human and AI agents: prebuilt and custom AI agents are connected to customer-experience workflows with explicit routing to humans when judgment is needed, plus centralized prompt and model governance. Scope is contact-centre work, not general enterprise task delegation.
commercial · Research snapshot 2026-09-06
- Ask for a demonstration
- Demonstrate a CX workflow where an AI agent hands off to a human agent, and show how the handoff condition and the governing prompt policy are configured.
Read sources and limitations →
Also covers this layer
Weights & Biases registry that curates versioned artifacts — models and datasets — into organisation-level collections with aliases, lineage tracking and access control, giving ML teams a single record of what is in production. Governance here means access and provenance, not risk or regulatory workflow.
commercial · Research snapshot 2026-09-06
- Ask for a demonstration
- Show me how a model artifact is linked into a production collection, who can access it, and how lineage lets us reproduce that exact version.
Read sources and limitations →
Primary category
Weights & Biases product for tracking LLM calls and application logic with automatic tracing and cost tracking, scorer-based evaluation and comparison tools, plus pre- and post-response safeguards. Platform controls include role-based access at team or project level, SSO via OIDC, SCIM provisioning and scoped service accounts.
commercial · Research snapshot 2026-09-06
- Ask for a demonstration
- Show me a traced LLM application with cost tracking, and demonstrate restricting project access to a named team using SSO-provisioned users.
Read sources and limitations →
Also covers this layer
Runtime security layer for AI agents that analyses full execution sequences, including chained tool calls, retrievals and agent-to-agent handoffs, to detect direct and indirect prompt injection, unauthorised tool invocations and sensitive data leaving through agent activity, with inline blocking. Evidence is drawn from vendor platform pages.
commercial · Research snapshot 2026-09-06
- Ask for a demonstration
- Show me AIDR detecting a slow-building indirect injection across several turns, blocking the unauthorised tool call it triggers, and the agent-to-agent handoff record.
Read sources and limitations →
