Model lifecycle & governance
NannyML
Open-source Python library for post-deployment monitoring that estimates a model's performance when ground-truth labels are delayed or missing, using confidence-based estimation for classification and direct loss estimation for regression, and links univariate and multivariate drift alerts to performance impact.
open_source · generally available · Research snapshot 2026-09-06
Visit the official product source ↗Where it fits
Model lifecycle & governance · Observability & traceability
Useful conversation with: Data scientist, ML engineer, Model validator.
Ask for a demonstration
Demonstrate estimated versus realised ROC AUC on a tabular classifier with delayed labels, and show which drift alerts were linked to the performance change.
Capabilities and evidence
Support labels reflect the supplied research. Documentation and vendor claims are not independent product tests. “Not established” means the researcher did not find support; it does not prove a capability is absent.
Documented by provider
The NannyML repository states the library estimates post-deployment model performance without access to targets, detects data drift, links drift alerts to changes in model performance, supports tabular classification and regression, and implements confidence-based performance estimation (CBPE) and direct loss estimation (DLE), including metrics such as ROC AUC and RMSE.
Limit: Documented scope is tabular ML; the repository documents no registry, approvals or lifecycle governance features.
Source s1
Documented by provider
NannyML's stable documentation (version 0.13.1) documents performance estimation and calculation, business-value estimation, confusion-matrix elements, custom binary classification metrics, and univariate and multivariate data drift monitoring.
Limit: Docs describe methods and metrics but not enterprise features, and no vendor company details are given.
Source s2
Documented by provider
Multiple repository paths exist for the project (NannyML/NannyML plus forks such as nnansters/nannyml), so the canonical source should be confirmed before adoption.
Limit: Only establishes that several repository names surface publicly; ownership history was not investigated.
Source s1
Limitations to discuss
- Licence identifier not confirmed on the pages fetched.
- Commercial offering or company status was not evidenced; only the library was verified.
Sources
- NannyML/nannyml — post-deployment data science in python · GitHub / NannyML · official repository
Access date reported by researcher: 2026-09-06 - Welcome to NannyML's documentation! · NannyML · official docs
Access date reported by researcher: 2026-09-06
Listing does not imply partnership, supplier status, a working DutyGraph integration, or a compliance certification.
Suggest a correction