Structure training evidence and provider assembly by partition¶

Status: Implemented in the local working tree and verified with the complete local gate. Test interfaces follow the existing contracts, with direct application/group tests as proposed to the user. This engineering specification preserves the approved commerce contracts.

Problem Statement¶

Training evidence derivation is long even without direct conditionals. It mixes stream reduction, parameter selection, scoring, truth construction, geographic sets, and result assembly. Published and evaluation providers are assembled through repeated mappings and overrides.

Solution¶

Compose cohesive evidence-building objects and explicit partition-specific evidence values. Build providers through one factory per partition so evaluation isolation is visible in the data model.

User Stories¶

As a developer, I want explicit reduction, selection, scoring, and assembly stages, so that each stage can be understood independently.
As a developer, I want cohesive parameter objects, so that related options no longer travel as long repeated argument lists.
As a evaluator, I want explicit training/holdout/full-data partitions, so that evaluation cannot accidentally inherit publication evidence.
As a developer, I want one provider assembly path, so that new evidence tiers do not require matching edits in duplicate mappings.
As a consumer, I want unchanged ranking and provenance, so that the refactor preserves Recommendation Sets.
As a Commerce Property, I want one consistent scoped source read, so that evidence remains isolated and internally consistent.
As a operator, I want work-store lifetime stays explicit, so that temporary state is always cleaned up.
As a evaluator, I want unchanged parameter grids and metrics, so that baseline comparisons remain meaningful.
As a consumer, I want geographic/category constraints stay before bounded selection, so that local and constrained lanes remain correct.
As a tester, I want existing end-to-end and evaluation contracts retained, so that new objects cannot hide changed behavior.
As a operator, I want failed training preserves the serving head, so that atomic publication remains intact.

Implementation Decisions¶

Separate orchestration from coherent evidence builders for behavioral, popularity, and evaluation evidence; keep shared derived work-store ownership explicit.
Introduce typed partition-specific values and cohesive immutable parameter values rather than generic dictionaries or positional data clumps.
Use a candidate-provider factory driven by an explicit partition's evidence. Shared metadata similarity remains deliberately shared; behavioral/popularity evidence must be supplied for that partition.
Preserve the generate_recommendations public call contract through an adapter where necessary; avoid breaking worker and test callers.
Preserve parameter selection, temporal cutoffs, metrics, resource limits, native strategy selection, geographic support rules, consistency checks, and cleanup.

Testing Decisions¶

Use existing complete training, category/geographic strategy, evaluation, compatibility, and fallback tests as the behavioral seams.
Add evidence that publication and evaluation provider assembly receive distinct partition inputs and produce independently expected results; avoid self-comparison as the oracle.
Run focused training/evaluation tests, static checks, the complete suite, and installed-wheel verification.
Successful checks alone are insufficient: review the final implementation against every story, verify ownership moved into cohesive objects, and confirm duplicated decisions were removed.
Keep explicit validation guards when they communicate invariants; do not target an arbitrary count of conditional statements or introduce classes that merely forward calls.

Out of Scope¶

New models, changed grids, changed metrics or evidence schema, migrations, publication redesign, performance claims, and new source reads.

Further Notes¶

Preserve Commerce Scope isolation, snapshot-only serving, atomic publication, bounded resources, and exclusion of credentials and raw interaction identities from retained diagnostics. The refactor is complete only after implementation, focused verification, repository gates, and code-aligned documentation updates. PostgreSQL checks require an explicitly disposable database; record their omission if none is available. No deployment or hosted-setting change is authorized.