Alphabetical, not ranked. Membership includes primary and secondary research categories. These products have different scopes; inspect the evidence profile before comparing capabilities.
Also covers this layer
Open, community-contributed catalogue of real-world AI harms and near-harms maintained by the Responsible AI Collaborative, searchable on the web and queryable through a public read-only GraphQL endpoint, with taxonomies for classification. Records are curated press-based reports, so coverage is uneven and not an authoritative incident register.
open_source · Research snapshot 2026-09-06
- Ask for a demonstration
- Show me querying the GraphQL endpoint for incidents involving autonomous agents or LLM assistants and how you would use them in our AI risk taxonomy.
Read sources and limitations →
Also covers this layer
Commercial AI engineering platform that captures traces of agent runs including inputs, outputs, tools and costs, groups them into sessions, visualises agent trajectories, and runs online and offline evaluations with human annotation. Arize also maintains Phoenix, a local-first open-source tracing and evaluation package.
hybrid · Research snapshot 2026-09-06
- Ask for a demonstration
- Show me an agent trajectory visualisation for a failed multi-step run and the trace-level evaluation scores attached to it.
Read sources and limitations →
Also covers this layer
AI governance, risk and security platform formed when Canada's Fairly AI acquired Sweden's anch.AI in June 2025 and rebranded as Asenion. It markets automated controls that continuously assess and test AI systems and agents against frameworks including the EU AI Act, ISO/IEC 42001 and NIST AI RMF.
commercial · Research snapshot 2026-09-06
- Ask for a demonstration
- Demonstrate one automated control test running against a production agent, and show the tamper-resistant assurance record it produces.
Read sources and limitations →
Also covers this layer
Boutique firm offering independent third-party AI audits and responsible AI consulting, plus a five-course AI and Algorithm Auditor Certification for practitioners ending in a capstone and exit exam. The firm states its audits follow assurance-engagement style practice but names no accrediting body for either the audits or the credential.
service · Research snapshot 2026-09-06
- Ask for a demonstration
- Demonstrate how a BABL independent third-party audit engagement is scoped and what assurance wording appears in the final report you issue to a client's stakeholders.
Read sources and limitations →
Primary category
Developer platform for scoring AI application output: teams log traces from their app, curate datasets from human feedback, run experiments against scorers, then watch production logs. Documentation covers project-level access control and a customer-controlled data plane, but audit logging and retention behaviour are not described.
commercial · Research snapshot 2026-09-06
- Ask for a demonstration
- Show me an experiment run comparing two prompt versions against a dataset built from human feedback, and who in my organization can view the underlying logs.
Read sources and limitations →
Also covers this layer
Apache-2.0 Python tool from Cisco's AI Defense group that scans MCP servers, their tools, prompts and resources plus server source code, combining YARA rules, LLM-based analysis and Cisco's hosted inspection API, and audits dependencies and bundled binaries. Full detection depth depends on optional third-party services.
open_source · Research snapshot 2026-09-06
- Ask for a demonstration
- Show me a scan of an untrusted MCP server package that flags a docstring-versus-implementation mismatch, and which findings required the Cisco AI Defense API.
Read sources and limitations →
Primary category
Hosted quality platform from the DeepEval maintainers. It captures each LLM call as a trace with inputs, outputs, tool calls, latency, token cost and metadata, converts flagged traces into evaluation datasets, and runs metric-based regression tests on pull requests. Governance controls are not documented.
hybrid · Research snapshot 2026-09-06
- Ask for a demonstration
- Show me how a failing production trace becomes a dataset case that then blocks a pull request when the metric regresses.
Read sources and limitations →
Primary category
Simulation and evaluation platform for voice and chat agents. It generates large volumes of simulated callers with accents, interruptions, noise and policy traps, scores production calls in real time on resolution and safety, detects regressions after prompt or model changes, and routes failures to human reviewers.
commercial · Research snapshot 2026-09-06
- Ask for a demonstration
- Demonstrate a pre-launch simulation of a thousand callers against my voice agent, then show the production scoring that flags a regression after a prompt change.
Read sources and limitations →
Primary category
Open-source Python framework for testing LLM applications: it runs metric-based scoring on test cases and on captured agent trajectories, including individual LLM calls, tool use, retrieval and sub-agent handoffs. It executes locally in developer or CI environments and provides no access control or audit records itself.
open_source · Research snapshot 2026-09-06
- Ask for a demonstration
- Show me a CI test run that scores my agent's full trajectory, including whether the right tools were called with the right arguments.
Read sources and limitations →
Also covers this layer
Named specialist assurance offering from Deloitte member firms that reviews client algorithms and the controls around them: definition, identification, classification, assessment, code review, stress testing, and monitoring. Aimed at audit committees and risk functions. Pages describe activities and objectives without naming an assurance standard or reporting format.
service · Research snapshot 2026-09-06
- Ask for a demonstration
- Demonstrate how you classify an inventory of production algorithms and what your third-party code review report on one high-risk model actually concludes.
Read sources and limitations →
Primary category
US NIST-built test platform for assessing trustworthy characteristics of AI models, providing a REST API, web interface and Python client to design, run and track reproducible experiments including adversarial red-team scenarios. Intended to support the Measure function of the AI Risk Management Framework rather than to certify systems.
open_source · Research snapshot 2026-09-06
- Ask for a demonstration
- Demonstrate reproducing an adversarial robustness experiment in Dioptra and exporting the tracked evidence for an internal AI risk review.
Read sources and limitations →
Also covers this layer
Apache-2.0 Python library plus self-hostable platform for evaluating and monitoring data and AI systems, including tabular data quality and drift tests with statistical methods, declarative test suites usable in CI/CD, and dashboards tracking metrics and test results over time for deployed models.
open_source · Research snapshot 2026-09-06
- Ask for a demonstration
- Show me a drift test suite failing in CI for a production model's input data, and how the result appears on the monitoring dashboard over time.
Read sources and limitations →
Primary category
Apache-licensed LLM vulnerability scanner maintained by NVIDIA. It fires static, dynamic and adaptive probes at a model or dialog system to test for jailbreaks, prompt injection, toxicity, data leakage and misinformation, logs each generation and detector verdict, and outputs a report with failure rates and hit logs.
open_source · Research snapshot 2026-09-06
- Ask for a demonstration
- Show me a garak scan of my chatbot with the probe-by-probe failure rates and the hit log for successful jailbreaks.
Read sources and limitations →
Also covers this layer
Google Cloud's renamed Vertex AI agent stack, combining a managed agent runtime with sessions, a memory bank, example store and evaluation service, and support for ADK, LangGraph, LlamaIndex and A2A agents. Governance evidence centres on registration through the Gemini Enterprise app and built-in tracing rather than approval workflows.
commercial · Research snapshot 2026-09-06
- Ask for a demonstration
- Demonstrate deploying an ADK agent to the managed runtime, registering it in Gemini Enterprise, and inspecting its traces in Cloud Trace.
Read sources and limitations →
Primary category
French vendor pairing an open-source Python testing library with a delivered assessment service. Automated and expert-led testing probes conversational agents for prompt injection, data disclosure, sycophancy, hallucination and inappropriate refusals, returning a severity-ranked vulnerability report and a signed go/no-go deployment recommendation.
hybrid · Research snapshot 2026-09-06
- Ask for a demonstration
- Show me a full assessment report for my customer-facing agent, with vulnerabilities ranked by severity and the go/no-go deployment recommendation.
Read sources and limitations →
Also covers this layer
Independent AI audit engagements from Holistic AI covering bias, privacy, efficacy, robustness and explainability, plus regulation-specific assessments and a separate independent audit service for Digital Services Act due-diligence obligations. Audits produce reports and mitigation strategies; no accreditation or certification mark is claimed on the pages reviewed.
service · Research snapshot 2026-09-06
- Ask for a demonstration
- Show me an anonymised AI audit report structure and the specific tests you ran for bias and robustness on a deployed decisioning model.
Read sources and limitations →
Also covers this layer
Enterprise platform that inventories AI systems, models, agents and pipelines, then rates each system across risk verticals such as bias, robustness, privacy and transparency on a traffic-light dashboard. Marketed for internal builds and procured AI. Risk ratings are vendor-defined scores, not audits or certifications.
commercial · Research snapshot 2026-09-06
- Ask for a demonstration
- Demonstrate how a procured third-party model gets an inherent-risk rating and what evidence sits behind the red-amber-green score.
Read sources and limitations →
Also covers this layer
Agent observability and evaluation platform aimed at enterprises running production agents. Distributed tracing captures every step including tool calls, prompts, retries, loops and handoffs between sub-agents across long trajectories, with step-by-step replay, online evals on live traffic and annotation queues that turn expert review into datasets.
commercial · Research snapshot 2026-09-06
- Ask for a demonstration
- Replay a multi-day agent trajectory step by step, showing every tool call and sub-agent handoff plus the evaluation scores attached to it.
Read sources and limitations →
Primary category
MIT-licensed evaluation framework from the UK AI Security Institute for running model and agent evaluations. It executes dataset samples through solvers, supports tool calling, multi-turn dialog, multi-agent primitives and sandboxed execution in Docker or Kubernetes, and writes evaluation logs for analysis. No hosted service or access control.
open_source · Research snapshot 2026-09-06
- Ask for a demonstration
- Demonstrate running one of the pre-built agentic evaluations against my model in a sandboxed environment and show the resulting evaluation log.
Read sources and limitations →
Also covers this layer
Apache-2.0 OpenTelemetry-native platform for AI agents that traces LLM calls, tool use, custom functions and parallel sub-agents, presenting runs as transcripts rather than span trees. It records and replays runs in a debugger, builds evaluation datasets from production traces, and runs evals in CI. No documented access control.
hybrid · Research snapshot 2026-09-06
- Ask for a demonstration
- Replay a recorded agent run in the debugger, swap the prompt, and show the side-by-side comparison plus the eval that would catch the regression in CI.
Read sources and limitations →
Also covers this layer
German open-source AI engineering platform whose traces record LLM and non-LLM calls, retrieval, embeddings and API calls, grouped into sessions for multi-turn conversations and agent graphs. Paid tiers add audit logs, fine-grained RBAC, SSO/SCIM, retention management, client-side masking and EU, US or JP data regions.
hybrid · Research snapshot 2026-09-06
- Ask for a demonstration
- Show me a session trace for a multi-turn agent conversation, then show the audit log of who accessed or changed that project.
Read sources and limitations →
Also covers this layer
LangChain's hosted platform for tracing LLM and agent applications. Traces record what agents did in production and feed evaluation datasets, online evaluation rules and annotation queues. Enterprise plans add role-based access control with workspace roles and custom organization roles; audit logging is not documented.
commercial · Research snapshot 2026-09-06
- Ask for a demonstration
- Show me a production trace for a failed agent run and demonstrate which workspace roles can view or edit that trace.
Read sources and limitations →
Also covers this layer
Dutch open-source LLMOps platform that automatically tracks every LLM call, tool usage and user interaction as traces, spans and metadata, adds prompt versioning, dataset experiments and multi-turn agent testing against a simulated user. Self-hosted and enterprise editions document audit logs, RBAC and SSO/SCIM.
hybrid · Research snapshot 2026-09-06
- Ask for a demonstration
- Show me the full conversation trace for one agent session, then show the audit log entry for the user who changed the evaluation configuration.
Read sources and limitations →
Also covers this layer
Open-source guardrail framework from Meta that runs layered scanners around agent execution: PromptGuard 2 for jailbreak detection, Agent Alignment Checks that audit chain-of-thought for goal misalignment, and CodeShield static analysis of generated code. Meta describes the alignment auditor as still experimental.
open_source · Research snapshot 2026-09-06
- Ask for a demonstration
- Show me LlamaFirewall scanning an email agent's inputs with PromptGuard 2 and flagging an indirect injection through Agent Alignment Checks, including the experimental caveats.
Read sources and limitations →
Also covers this layer
AI governance platform aimed at regulated industries that replaces manual risk reviews with workflow-driven assessments, evaluates models across a large metric set, and adds monitoring plus guardrails. Public evidence is largely vendor-authored, including a 2024 UK government assurance-technique listing.
commercial · Research snapshot 2026-09-06
- Ask for a demonstration
- Show me which of the 200+ model metrics are computed automatically for a deployed model and how results feed the risk assessment workflow.
Read sources and limitations →
Primary category
India-founded platform combining agent simulation, evaluation and production logging. Teams simulate multi-turn interactions across scenarios and personas, score runs with AI, programmatic or statistical evaluators plus human review pipelines, curate datasets from real interactions, and analyse production logs with distributed tracing.
commercial · Research snapshot 2026-09-06
- Ask for a demonstration
- Show me a simulation run across several user personas with evaluator scores per version, and the production logs for the same agent.
Read sources and limitations →
Primary category
UK vendor running automated red-team tests against AI models, applications and agents. It profiles the target, enumerates attack surface, executes techniques from a maintained attack library via CLI or SDK, and reports exploitable findings with remediation guidance. Documentation covers testing mechanics rather than audit, retention or access controls.
commercial · Research snapshot 2026-09-06
- Ask for a demonstration
- Show me an automated red-team run against my production agent, including which attack techniques were executed and the remediation guidance produced.
Read sources and limitations →
Also covers this layer
Governance platform organised around projects representing one AI system each, with frameworks, requirements, controls, evidence and reviews, plus a risk module that quantifies AI risk in monetary terms and a runtime inspection module for scheduled tests. Multi-framework mapping is vendor-maintained content, not a compliance guarantee.
commercial · Research snapshot 2026-09-06
- Ask for a demonstration
- Demonstrate how one control satisfies EU AI Act and ISO 42001 requirements simultaneously and how the linked evidence and review trail is exported.
Read sources and limitations →
Also covers this layer
Platform that inventories agents, MCP servers, skills and models across endpoints, SaaS agent builders and homegrown AI stacks, maps each agent's permissions and data access, red teams them before production, and evaluates runtime actions to alert, block, mask data or route to a human. Claims rest on vendor pages.
commercial · Research snapshot 2026-09-06
- Ask for a demonstration
- Show me Noma discovering an unapproved MCP server on a developer laptop, mapping its blast radius, and then routing a risky agent action to a human for approval.
Read sources and limitations →
Also covers this layer
Apache-2.0 platform from Comet that records every LLM call, tool invocation and agent step as traces, monitors cost, latency, error rates and feedback scores, scores incoming traces with online rules, and converts problem traces into test cases. The full backend can be self-hosted; RBAC and audit logs are not documented.
hybrid · Research snapshot 2026-09-06
- Ask for a demonstration
- Show me a self-hosted deployment where every tool invocation in an agent run is logged and automatically scored by an online evaluation rule.
Read sources and limitations →
Also covers this layer
ORCAA sells algorithmic audits that assess risks of a specific algorithmic use case using its Ethical Matrix framework, plus quantitative bias testing for regulatory compliance such as New York City Local Law 144 bias audits, AI governance consultation, and metric 'cockpit' design. Deliverables are reports, not certifications.
service · Research snapshot 2026-09-06
- Ask for a demonstration
- Show me how the Ethical Matrix is populated for one hiring algorithm and what a Local Law 144 bias audit report you issued contains.
Read sources and limitations →
Primary category
Vendor offering managed evaluators plus simulation environments for agent testing: hosted judges score hallucination and unsafe output, red-teaming algorithms probe for weaknesses, and simulated digital workflows exercise long-horizon agent tasks. Public pages document scoring and simulation but not audit records, retention or access control.
commercial · Research snapshot 2026-09-06
- Ask for a demonstration
- Demonstrate a simulated multi-step workflow run where my agent is scored for hallucination and unsafe output, and show what evaluation evidence I can export.
Read sources and limitations →
Also covers this layer
Israeli platform covering the AI agent lifecycle: cataloguing agents, models, prompts, MCP servers and coding agents through agentless integrations, then applying runtime guardrails that monitor prompts, tool calls and commands for prompt injection, tool poisoning and data exfiltration. Product claims come from vendor pages, not reference docs.
commercial · Research snapshot 2026-09-06
- Ask for a demonstration
- Show me Pillar validating that an agent's tool call matches its declared schema, flagging a permission-scope deviation, and blocking a poisoned instruction in an agent-to-agent handoff.
Read sources and limitations →
Primary category
Apache-2.0 LLM evaluation toolkit from Singapore's AI Verify Foundation that combines benchmark testing across safety and performance metrics with manual and automated red-teaming, offers guided workflows for IMDA's starter kit for LLM app testing, and produces shareable scoring reports usable in CI pipelines.
open_source · Research snapshot 2026-09-06
- Ask for a demonstration
- Show me running IMDA's starter-kit benchmarks plus an automated red-team attack module against our chatbot and the scoring report it produces.
Read sources and limitations →
Primary category
Open-source evaluation and red-teaming tool that generates adversarial inputs from configurable plugins, runs them against an LLM application, and grades outputs with deterministic and model-graded metrics in CI. The paid enterprise editions add RBAC and team scoping; audit logging and retention are not documented.
hybrid · Research snapshot 2026-09-06
- Ask for a demonstration
- Demonstrate an end-to-end red-team scan of my deployed agent, including which plugins ran and how findings are scoped to a team.
Read sources and limitations →
Primary category
MIT-licensed Python framework from Microsoft for probing generative AI systems for risk. It is aimed at security engineers running automated adversarial testing campaigns rather than at governance teams, and the repository provides no multi-user controls, evidence retention or reporting workflow of its own.
open_source · Research snapshot 2026-09-06
- Ask for a demonstration
- Demonstrate an automated PyRIT attack run against my deployed model endpoint and show what artefacts the run leaves behind.
Read sources and limitations →
Primary category
Open-source Python evaluation library for LLM and RAG applications. It runs LLM-based and deterministic metrics over datasets, generates synthetic test sets, and tracks results across experiments so changes can be compared. It is a local library with no hosted control plane, access control or audit trail.
open_source · Research snapshot 2026-09-06
- Ask for a demonstration
- Show me a Ragas experiment comparing two retrieval configurations on a generated test set, with per-metric scores and reasons.
Read sources and limitations →
Also covers this layer
Independent AI assurance firm running testing, evaluation, validation and verification workflows in the client's own environment via its Approved Intelligence Platform, producing structured deployment evidence and continuous post-deployment evaluation for civil and defence uses. Marketed as evidence for deployment decisions rather than as certification against a standard.
service · Research snapshot 2026-09-06
- Ask for a demonstration
- Demonstrate a TEVV run in our environment for one mission-critical model and show the evidence pack a deployment authority receives.
Read sources and limitations →
Also covers this layer
Runtime security product for AI agents that inspects prompts, reasoning steps and tool calls across coding assistants, productivity copilots and custom agents, blocking direct and indirect injection, destructive actions such as file deletion, and data exfiltration. Vendor pages also describe shutting down rogue agents and connections.
commercial · Research snapshot 2026-09-06
- Ask for a demonstration
- Show me Defend AI blocking an indirect injection delivered in an email to a productivity copilot and stopping a coding agent from deleting files, then show the rogue-agent shutdown action.
Read sources and limitations →
Also covers this layer
Model risk management and AI governance platform pairing a Python library that runs tests and generates model documentation with a review platform for validators, covering statistical, ML, LLM and agentic records with inventory, versioning and approval workflows aimed at regulated financial institutions.
commercial · Research snapshot 2026-09-06
- Ask for a demonstration
- Demonstrate the developer-to-validator handoff: library-generated documentation and test results entering a validation workflow with approvals and version control.
Read sources and limitations →
Also covers this layer
Weights & Biases product for tracking LLM calls and application logic with automatic tracing and cost tracking, scorer-based evaluation and comparison tools, plus pre- and post-response safeguards. Platform controls include role-based access at team or project level, SSO via OIDC, SCIM provisioning and scoped service accounts.
commercial · Research snapshot 2026-09-06
- Ask for a demonstration
- Show me a traced LLM application with cost tracking, and demonstrate restricting project access to a named team using SSO-provisioned users.
Read sources and limitations →
