Compose strict deployment configuration sections

Status: Implemented in the local working tree and verified with the complete local gate. Test interfaces follow the existing contracts, with direct application/group tests as proposed to the user. This engineering specification preserves the approved commerce contracts.

Problem Statement

A single large deployment loader owns file and environment access, defaults, type checks, unknown-field validation, and assembly of unrelated configuration sections. Developers must understand the entire loader to change one setting.

Solution

Give each configuration section a cohesive strict parsing contract. Keep a small deployment loader for input acquisition and assembly, preserving accepted configuration and validation semantics.

User Stories

1. As a developer, I want section-owned schemas, so that one setting has one place for parsing and validation.
2. As a operator, I want unchanged defaults, so that existing deployment behavior remains stable.
3. As a operator, I want unknown fields rejected, so that misspelled controls cannot silently disappear.
4. As a operator, I want strict numeric and boolean validation, so that invalid coercions cannot alter safety limits.
5. As a developer, I want environment and file acquisition separate from parsing, so that section tests can use ordinary mappings.
6. As a operator, I want existing environment precedence preserved, so that overrides remain predictable.
7. As a security maintainer, I want secret references remain unresolved during configuration parsing, so that secrets stay out of representations and fingerprints.
8. As a developer, I want unchanged configuration fingerprints, so that refactoring alone does not invalidate comparable runs.
9. As a tester, I want existing full-loader tests retained, so that input acquisition and section composition remain covered.
10. As a maintainer, I want bounded, scope-aware personalization and Swimlane validation, so that the refactor preserves operational contracts.

Implementation Decisions

• Compose section parsers or strict schema objects for Data Sources, pipeline, freshness, retention, quality, personalization, Swimlanes, and observability; retain existing domain configuration objects.
• Keep environment/file acquisition at the outer loader and use pure section parsing. Reuse existing strict validators where their behavior differs from generic schema coercion.
• Preserve required fields, defaults, unknown-key rejection, numeric/boolean distinctions, semantic constraints, secret references, and configuration fingerprint inputs.
• Use existing Pydantic only where it preserves the contract; no dependency addition or blanket coercive model replacement.
• Keep public loader and configuration constructors compatible.

Testing Decisions

• Retain existing configuration tests for unknown keys, group limits, optional purchases, backend selection, personalization safety, observability precedence, and forbidden environment inputs.
• Add focused section parsing tests through stable parser interfaces and compare representative defaults/fingerprints with the baseline.
• Run configuration tests, static checks, the complete local suite, and installed-wheel verification.
• Successful checks alone are insufficient: review the final implementation against every story, verify ownership moved into cohesive objects, and confirm duplicated decisions were removed.
• Keep explicit validation guards when they communicate invariants; do not target an arbitrary count of conditional statements or introduce classes that merely forward calls.

Out of Scope

New settings, changed defaults, secret rotation, relaxed validation, dependency changes, and deployment updates.

Further Notes

Preserve Commerce Scope isolation, snapshot-only serving, atomic publication, bounded resources, and exclusion of credentials and raw interaction identities from retained diagnostics. The refactor is complete only after implementation, focused verification, repository gates, and code-aligned documentation updates. PostgreSQL checks require an explicitly disposable database; record their omission if none is available. No deployment or hosted-setting change is authorized.