ML admission implementation evidence

Research date: 2026-09-21

Status: Primary-source inspection; design input, not production qualification. Baseline: Serving admission-plane landscape.

Decision-changing findings

Limitador remains a useful implementation reference, but its reservation API is insufficient evidence for a thin-wrapper adoption decision. Require a lifecycle conformance spike before selecting an engine. The inspected source is pinned to 0984355f8408430f75619d42515888af48347df9, resolved from upstream HEAD during this review; this does not establish a stable release guarantee.

Limitador: atomic reserve does not mean atomic lifecycle

The Redis reservation script checks every matched counter before adding holds. Its cleanup and counter initialization can still write on rejection, so the accurate guarantee is no partial new hold, rather than literally no writes. It scans HGETALL for each counter and caps each hold's expiration at that counter's rate-window boundary. Thus reservation evaluation scales with outstanding entries, and these holds are rate-window reservations, not independent concurrency leases. These are direct code observations, not measured latency results. (Pinned reserve script)

Commit re-resolves the currently matching policy, updates each counter sequentially, then releases holds. Changed policies can leave old holds until expiry. Retried commits charge again, including after a successful earlier release; an upstream test explicitly verifies this behavior. Inference: partial failures can leave partially applied charges, and an external idempotency table cannot make those separate mutations atomic without participating in the same transaction. (Pinned commit implementation and replay test)

The asynchronous Redis adapter calculates reservation time from its process clock and removes holds one counter at a time. Inference: clock skew between instances and failures partway through release belong in the qualification tests. (Pinned Redis adapter)

The reservation model explicitly uses one scalar across counters. Its optional hold fraction can reserve less than the requested amount, increasing potential overshoot at settlement. This is an intentional throughput/fairness tradeoff, not a hard resource-cap guarantee. (Pinned reservation model)

Redis: serialization, partitioning, and durability are separate contracts

Redis executes Lua atomically and blocks other server activities while a script runs. Long scans therefore affect the entire shard. Redis Cluster restricts multi-key operations to the same hash slot and uses asynchronous replication; acknowledged writes can be lost at failover. An atomic Lua script alone does not establish durable admission or prevent permit duplication after lost state. (Lua execution, Cluster specification)

Design implication: explicitly name the authority and failure guarantee. A strict capacity ledger needs durable admission state or a failover protocol that fences old execution and reconstructs outstanding work before reissuing capacity. Cell-local quotas simplify atomicity, but a global cap still needs safe quota allocation across cells; copying the global cap into each cell is incorrect.

Envoy: useful enforcement seam and local feedback controller

External processing supports request/response inspection, modifications, and immediate responses. Its configuration exposes failure behavior and message timeouts; observability mode does not wait for processor responses and therefore cannot enforce admission. The HTTP-service variant supports headers only. Choose the gRPC variant if body/stream processing is required, and make enforcement failure policy explicit. (Filter documentation, Configuration API)

Adaptive concurrency adjusts outstanding-request limits from latency measurements. Envoy documents that the controller must see and control all traffic to its relevant local cluster; distributed traffic that bypasses the filter invalidates that assumption. Its minimum-latency sampling can also cause temporary rejection. Use this mechanism to protect execution locally, not as proof of a fleet-wide tenant allocation. (Adaptive concurrency)

Kueue: separate eligibility, quota reservation, and execution readiness

Kueue marks a workload admitted when quota is reserved and all admission checks are ready. Checks have Pending, Ready, Retry, and Rejected states; Retry can release quota and evict an admitted workload. Adding a check does not automatically evict already-admitted workloads. This is a concrete example of distinct lifecycle facts and explicit policy-update behavior. (Admission checks)

ResourceFlavors describe hardware or pricing variations, with labels and scheduling constraints binding the chosen flavor to execution. Borrow this binding for approved ML execution profiles: choosing a cheaper profile must actually constrain the backend route and workload parameters. (Resource flavors)

Kueue also documents an optional Pods-ready timeout and requeue behavior. Admission is therefore distinct from actual readiness; a request-plane permit should make the same distinction without copying a Kubernetes reconciliation loop into synchronous serving. (All-or-nothing readiness)

Proposed innovations, not claims about existing implementations

1. Bind a permit to an executable profile. Select among a bounded ordered list of approved profiles, each with eligibility predicates, resource quantities, route, and output contract. Evaluate feasibility and reserve resources together; record the selected profile in the permit.
2. One authority for the whole lifecycle. Admission identity, policy version, original charged keys, resource vector, renewal sequence, and terminal settlement belong to one transactional state machine. An identical retry returns the previous result; a conflicting retry fails.
3. Separate accounting expiry from execution release. A lost heartbeat is uncertainty, not evidence that GPU work stopped. Reclaim hard concurrency after termination acknowledgement or an execution mechanism that enforces a bounded deadline. An epoch token only prevents stale work if the resource owner checks it; it cannot retroactively stop already-running computation.
4. Attach the enforcement guarantee to the compiled contract. A rule requiring hard capacity must be rejected at deployment if its adapter cannot enforce the corresponding execution bound. Advisory cost estimates and sampled quality signals should carry weaker, explicit guarantees.
5. Bound the control path itself. Limit matched rules, profiles, outstanding holds, cleanup work, operation history, and diagnostic metadata. Benchmark the worst configured bounds, including a hot tenant or pool, not only average traffic.

The architectural contribution is a small, inspectable chain from eligibility evidence to chosen execution profile to capacity ownership to terminal evidence. None of the reviewed projects proves that entire chain for synchronous ML requests. This is a proposed composition and differentiation; the review does not establish that it is globally unprecedented.

Verification and open work

Source review only; no upstream code or performance tests were executed. Commands used: read the research skill and existing landscape; git ls-remote https://github.com/Kuadrant/limitador.git HEAD (succeeded, SHA above); git status --short -- docs/research/ml-admission-implementation-evidence.md (empty before creation). Official documentation and source were inspected through web reads. Qualification still requires concurrent admission, duplicate/ambiguous outcomes, partial settlement, policy replacement, window rollover, clock skew, backend overrun, and datastore failover tests.