Commerce Simulation Harness

The Commerce Simulation Harness is a local/test-only system for exercising the recommendation service through a browsable Simulation Storefront and bounded Synthetic Visitor Runner. It creates controlled Browsing Sessions and Online Orders in a separate mutable Simulation Data Source, then uses the existing Training API, worker, Data Source Adapter, and Serving API without privileged ingestion or snapshot mutation.

It is evidence for recommendation behavior under a synthetic scenario. It is not evidence of real conversion, revenue, retention, attribution, causal uplift, distributed capacity, or production scalability.

Architecture and ownership

• control-db owns recommendation Training Runs and immutable Recommendation Snapshots.
• simulation-db owns the mutable Catalog, views, Online Purchases, compatibility rules, Browsing Session metadata, Simulation Run lifecycle, and seed lineage.
• api and worker use one static simulation-source registration at process startup.
• storefront owns human sessions, ephemeral carts, run controls, checkpoint orchestration, and the human-readable projection of the authoritative evidence JSON.
• The Synthetic Visitor Runner calls the same /v1/storefront/* HTTP actions as a human browser.
• Generated relational content seeds Catalog and compatibility rows. Once mutated, the Simulation Data Source is not a verified Generated Relational Data Source or qualification artifact.

Only one Commerce Scope and one non-terminal Simulation Run are supported per stack: simulation-source / simulation-property / simulation-catalog.

Start the complete environment

Prerequisites are Docker with Compose support and the sibling telemetry checkout required by the base image build.

docker compose up --build

The recommendation API is available at http://127.0.0.1:8000 and the Simulation Storefront at http://127.0.0.1:8080. The default configuration uses separate PostgreSQL services and named volumes for the control database, Simulation Data Source, and evidence reports. It does not require an external image service, frontend toolchain, or network-dependent fixture.

The storefront provides non-mutating /health and /ready probes. Readiness requires the simulation schema and configured dependencies; liveness does not mutate data.

Interactive journey

1. Open the storefront and create a Simulation Run from the default Independent-Behavior Scenario.
2. Start the run, browse categories and Items, and add eligible Items to the cart.
3. Complete a simulated checkout. No payment, inventory, fulfillment, account, or shopper identity exists.
4. Trigger a Training Run checkpoint. New activity is closed, in-flight writes drain, source counts are captured, and the existing Training API is called with a deterministic idempotency key.
5. After the worker publishes a snapshot, inspect global lanes on home/category pages and anchored lanes on Item pages. Expand developer details to inspect score, confidence, support, provenance, fallback reason, snapshot, cutoff, and stale status.
6. Complete the final checkpoint and download the authoritative JSON report from the run result.

Short-lived opaque Browsing Session cookies isolate carts. They are not accounts and retain no shopper identity.

Automated journey

The one-shot runner uses the same storefront HTTP routes:

docker compose --profile runner run --rm simulation-runner

For a local editable environment, the equivalent entry point is:

recommendations-simulation run \
  --scenario config/simulation-scenarios/default-independent.json

Visitor mission assignment, Item selection, action plans, logical timestamps, session/order IDs, idempotency keys, and checkpoint layout derive from the resolved scenario identity. Wall-clock timestamps, scheduling order, request latency, and process-generated UUIDs are recorded as operational facts but do not affect logical reproducibility.

Scenario authoring

Scenarios are strict, versioned JSON. Unknown fields and unsupported versions fail closed. Resolution applies authored values, versioned defaults, an allowlist of web overrides, and finally non-overridable deployment caps. Allowed web overrides are visitor count, request rate, concurrency, duration, and root seed.

Resolver version 1.1 derives scenario identities and deterministic visitor/action streams with SHA3-512. It deliberately does not reproduce 1.0 digests or IDs; restart an in-progress simulation when moving between these versions. Recommendation-feedback random draws use reciprocal-rank-v2 for the same hash transition, so evidence from the two versions must not be compared as if the decision stream were unchanged.

Use default-independent.json as the baseline. default-feedback.json is visibly labelled recommendation_feedback; its recommendation-dependent behavior is not independent quality evidence and ranking metrics that require independence are reported as unavailable.

Checkpoints and evidence

A checkpoint stops both human and automated admission, performs a bounded drain, records source counts, submits the fixed Commerce Scope through POST /v1/training-runs, and polls GET /v1/training-runs/{run_id}. It never chooses a cutoff, cancels a worker, reads the control database, or substitutes a new Training Run after a timeout. Traffic resumes after an intermediate checkpoint only when the expected snapshot is visible through the Serving API.

CheckpointOrchestrator resolves its lifecycle dependency at construction: standalone execution uses NoopCheckpointLifecycle, and integrated execution uses SimulationLifecycleCheckpoint. Training/polling, source-and-snapshot verification, and terminal lifecycle reporting are explicit operations. Durable cancellation still precedes lifecycle completion; failure classification and the conditions for reopening admission are unchanged.

The versioned JSON report is authoritative. Its HTML page renders those stored values and does not recompute metrics. It includes scenario/seed/scope identity; reconciled visitor/action counts; source counts and checkpoint deltas; bounded HTTP latency/error aggregates; Training Run and snapshot attribution; coverage, fallback, provenance, and concentration; valid planted-mission ranking metrics; feedback distributions; explicit unavailable reasons; and capped safe diagnostics.

Safety model

• Environment must be exactly local or test; production-like labels fail startup.
• Outbound hosts, schemes, ports, resolved addresses, and redirects are allowlisted and revalidated.
• URLs containing credentials are rejected.
• Visitor, duration, rate, concurrency, retry, response-size, feedback, diagnostic, and artifact limits cannot be relaxed by a scenario.
• Active-run reset is rejected. Cancellation closes admission, signals runner work, and drains for a bounded period.
• Credentials are environment values and never enter scenario identities, reports, filenames, logs, or errors.

Reset, shutdown, and troubleshooting

Reset is available only after a run is terminal and preserves the terminal run record and evidence report while clearing mutable source and ephemeral session state. Stop the stack with docker compose down; add --volumes only when intentionally deleting local databases and reports.

If readiness fails, check in order: simulation-db health, simulation schema initialization, static source configuration, recommendation API health, fixed-scope values, and target-policy errors. A checkpoint timeout retains its known Training Run ID; retry inspection rather than submitting a new idempotency key. Missing or mismatched snapshot identity is a terminal checkpoint failure, not a condition the harness guesses around.