Data-contract standards for ML constraints¶
Research date: 2026-09-21
Question: Can an existing open YAML standard or tool represent and enforce this project's data, model-capability, deployment, diagnostic, and serving constraints?
Executive answer¶
No candidate fits as-is. The Open Data Contract Standard (ODCS) is the closest reusable document shape for dataset schema and quality rules. It has stable identity/version fields, numeric operators, units, severities, standard row/null/duplicate metrics, and SQL/custom extension points. It does not define immutable model-version capability bounds, tightening-only deployment policy, a portable typed diagnostic result, or stateful rate/concurrency enforcement.
If cross-tool data-contract interchange is valuable, use ODCS v3.2 as the dataset-facing envelope and add a small, namespaced project extension for model capabilities and policy references. Treat the extension as this project's specification: an opaque custom block does not acquire ODCS semantics merely by living in an ODCS document. Otherwise, a smaller project-owned YAML/JSON Schema that borrows ODCS's IDs, versions, typed operators, and units is the lower-cost choice.
Use Data Contract CLI, Soda Core, Great Expectations, dbt, or TensorFlow Data Validation (TFDV) as optional measurement adapters when their native data sources already fit. Do not make any of their configuration languages the universal contract. Keep rate buckets, concurrency leases, permits, renewal, and settlement in Model Serving Admission Control; static policy engines cannot supply that state machine.
Candidate assessment¶
| Candidate | What primary sources establish | Fit here | License and maturity signal |
|---|---|---|---|
| ODCS v3.2 | A platform-neutral YAML data contract with identity, version/status, schema, quality, SLA, servers, and extension fields. Quality rules support library metrics, SQL, vendor-specific custom implementations, numeric comparison operators, units, severity, and stable rule IDs. Built-ins include rowCount and null/missing/invalid/duplicate counts, including percent units. (standard, quality rules) |
Best extension base, not as-is. Strong for catalog cardinality and completeness. A custom metric can represent embedding success. Model capability immutability, deployment-policy tightening, effects, diagnostic results, and online admission are absent. | LF AI & Data project, Apache-2.0, current v3.2.0 specification and JSON Schema with multiple published releases. (repository, releases) |
| Data Contract Specification (DCS) and Data Contract CLI | DCS defines a YAML dataset contract, but its maintainers deprecate it in favor of ODCS and plan CLI support only through the end of 2026. The CLI now natively uses ODCS, lints and compares contracts, runs schema/quality/SLA checks, and exposes a Python Run result with passed/failed/warning/error state. (DCS repository, ODCS support, Python API) |
Adopt the CLI as an evaluator/linter only. Do not start on deprecated DCS. The CLI can execute ODCS row counts and SQL-derived ratios, but it does not supply this project's model or deployment semantics. | CLI is MIT-licensed and actively released; its changelog records ODCS support and current releases. (CLI repository, changelog) |
| SodaCL / Soda Core | YAML checks combine dataset metrics and thresholds. Built-ins include row_count, missing_percent, invalid_percent, and custom SQL/user-defined checks; scan outcomes include pass, fail, error, and configurable warn. Current Soda Core describes a YAML contract engine and multiple database adapters. (checks, contract language, repository) |
Evaluator only. Very good for SQL-backed cardinality, coverage, and an embedding-success ratio stored in rows. It is dataset-centric and supplies no immutable model contract, layered tightening rule, or stateful online limiter. | Actively developed, but current Soda Core is under Elastic License 2.0, whose hosted/managed-service restriction deserves review before embedding it in a product service. (license) |
| Great Expectations (GX Core) | Expectations are declarative assertions grouped into an Expectation Suite. They support thresholds such as mostly, severity, runtime parameters, structured validation results, and Checkpoint actions. A Validation Definition fixes a batch-to-suite association. (Expectations, Suites, Validation Definitions) |
Evaluator only. Built-in or custom Expectations can measure cardinality, completeness, and embedding success. Its suite/result lifecycle is useful precedent, but its Python/data-source model is not a portable ML capability contract or deployment policy. | Mature, actively released GX Core; Apache-2.0. (repository, changelog) |
| dbt contracts and data tests | An enforced model contract checks SQL model column names/types and supported warehouse constraints before/during build. Data tests are post-build SQL assertions; four generic tests ship built in and arbitrary failing-row queries are extensible. Platform enforcement varies, and dbt explicitly distinguishes model shape contracts from data-quality tests. (model contracts, data tests) | Evaluator only, when dbt already owns the data. Custom SQL can cover all three data metrics, but adopting dbt solely for application-produced ML evidence adds the wrong runtime and lifecycle. It has no model-serving or tightening semantics. | dbt Core is mature and Apache-2.0; platform-dependent constraint behavior is documented explicitly. (license) |
| TFDV schema | TFDV schemas express feature type/domain/presence expectations plus drift and skew thresholds. Named environments can include or exclude features for training versus serving; validation emits an Anomalies protobuf. (TFDV guide, API) |
ML-aware evaluator only. Strong for record/feature completeness and typed anomaly evidence. Environments mean applicability, not a deployment policy that narrows a model's bounds. Catalog-wide business measures and embedding success need custom statistics or application measurements. | Longstanding TensorFlow component, Apache-2.0. (repository) |
| OPA/Rego or CEL | OPA evaluates declarative Rego over structured input and may return arbitrary structured JSON; bundles distribute versioned policy/data and decision logs include decision and bundle revision IDs. CEL is an embeddable expression language; Kubernetes demonstrates separate CEL policy logic, parameter resources, scope bindings, and cumulative admission. (OPA, bundles, decision logs, Kubernetes CEL admission) | Generic evaluator alternative, not a contract standard. Either can express comparison and tightening logic, but this project must still define measurements, capability immutability, effects, and diagnostic schema. Neither provides atomic counters, leases, or settlement. Their flexibility is unnecessary until non-developers need policies that fixed operators cannot express. | OPA is an established Apache-2.0 general-purpose engine. Kubernetes CEL admission is stable, but that validates Kubernetes objects rather than ML products. (OPA repository) |
No primary-source specification reviewed defines a closer cross-platform ML capability contract. TFDV supplies the closest ML-specific data schema and typed anomaly format, while TFMA supplies model-metric thresholds and a blessing gate; both are TensorFlow pipeline artifacts rather than a portable model/data/deployment contract. (TFMA model validation)
Requirement fit¶
| Requirement | Existing support | Remaining project semantics |
|---|---|---|
| Item/catalog cardinality | Direct rowCount/row_count or table-row Expectations/tests in ODCS, Soda, GX, dbt; TFDV statistics can supply example counts. |
Define scope as (Data Source, Tracking ID, Catalog ID), freshness, provenance, and the capability-bound effect. |
| Completeness/coverage ratio | ODCS percent units, Soda missing_percent, GX percentage/mostly, dbt SQL, and TFDV presence fractions. |
Define the numerator/denominator precisely and distinguish missing source data from evaluator failure. |
| Embedding success rate | Custom SQL or custom metric/Expectation can compute it when results are tabular. | Define eligible-item denominator, model/version identity, attempt outcome taxonomy, and freshness. This is a first-class application measurement, not generic nullness. |
| Immutable model-version bounds | No candidate provides the required meaning. ODCS has document versions; that is not immutability or binding to a model artifact. | Publish a content-addressed/versioned Model Capability Contract with the model version and never edit it in place. |
| Deployment/team tightening | Generic OPA/CEL can evaluate it; TFDV environments only vary schema applicability. | Keep a separately versioned Deployment Limit Policy; effective checks are capability checks AND all matching policy checks. Reject relaxation or an empty interval at publication/load time. |
| Typed diagnostics | TFDV Anomalies, GX validation results, Soda states, and CLI Run results are useful adapter evidence. ODCS standardizes rule metadata but not portable result occurrences. |
Preserve a stable code, contract/policy versions, typed observed/expected values, provenance, severity, effect, retry semantics, and correlation in Diagnostic Occurrence. |
| Stateful rate/concurrency | None. OPA/CEL can decide against supplied state but does not own atomic counters, permits, expiry, renewal, or settlement. | Keep it in the independent serving-admission service and reference policy versions from its decisions. |
Recommendation¶
Build the narrow semantic layer; adopt existing pieces around it:
- Define a small project schema for
Model Capability Contract,Deployment Limit Policy, typed measurements, and diagnostic results. Use fixed operators and units; do not add Rego/CEL yet. - If data-contract interoperability is a real requirement, profile ODCS v3.2 for the dataset-facing portion and add one namespaced, JSON-Schema-validated extension. Do not encode core semantics in unvalidated free-form strings. If interoperability is not required, borrowing the ODCS concepts is cheaper than carrying its unrelated server, ownership, pricing, and SLA surface.
- Put data acquisition behind adapters. Data Contract CLI is the natural executor for ODCS; use
Soda/GX/dbt/TFDV only where their runtime already matches the source. Normalize their outputs
into
Constraint MeasurementandDiagnostic Occurrencerather than exposing vendor results. - Keep static qualification gates side-effect-free and separate from serving permits.
Adopting a data-quality DSL as the universal model policy, introducing OPA/CEL before fixed comparisons fail, or placing stateful serving admission inside YAML contract evaluation would all be overengineering. The smallest durable boundary is a project-owned typed contract with optional ODCS-compatible data sections and pluggable evaluators.