Skip to content

Embedded ANN retrieval for Similar Items and For You

Status: Default-off implementation present; representation relevance and ANN quality/resource qualification remain open. Not approved for broad rollout.

Problem Statement

Similar Items currently uses exact sparse metadata matching, while For You reranks a bounded union of precomputed global candidates. The developer wants embedded ANN to support both strategies and broaden personalized discovery without writing an ANN algorithm. A later request adds an opt-in PyTorch two-tower model.

Solution

Use a maintained embedded vector-search library for scope-local retrieval. Similar Items uses Item Metadata-derived vectors; For You derives a query from the authorized bounded Shopper Profile and searches eligible Items in the same published Catalog. Preserve serving isolation, authorization and fallback behavior while explicitly extending candidate discovery.

User Stories

  1. As a developer, I want a maintained embedded ANN implementation, so that search algorithms are not built from scratch.
  2. As a consumer, I want Similar Items to retrieve metadata-related Items, so that substitutable or closely related results remain meaningful.
  3. As a Shopper, I want For You to discover Items beyond existing precomputed candidate lists, so that personalization can explore the published Catalog.
  4. As a Commerce Property, I want all retrieval restricted by Data Source, Tracking ID and Catalog ID, so that evidence and results cannot cross Commerce Scopes.
  5. As a Shopper, I want only an authorized bounded Shopper Profile to inform my query, so that personalization remains purpose-specific.
  6. As a Shopper, I want acknowledged interactions to affect subsequent queries without rebuilding the index, so that recommendations can react between Training Runs.
  7. As a consumer, I want causal-token guarantees preserved, so that required acknowledged interactions are observed or the existing failure contract applies.
  8. As a consumer, I want eligibility and request exclusions enforced on every returned Item, so that approximation cannot bypass merchandising constraints.
  9. As a consumer, I want self-results and duplicate Items excluded where applicable, so that recommendation slots remain useful.
  10. As a consumer, I want bounded fallback behavior for missing profiles or unusable vectors, so that sparse evidence does not produce unsafe or fabricated results.
  11. As a operator, I want no source reads or model training during serving, so that request cost and source isolation remain controlled.
  12. As a consumer, I want snapshot identity and truthful provenance on results, so that the source of retrieval is inspectable.
  13. As a developer, I want one small retrieval interface with library details hidden, so that variant dispatch does not spread through handlers.
  14. As a tester, I want separate measures of representation quality and ANN recall, so that a faster index cannot conceal worse metadata modeling.
  15. As a maintainer, I want the current non-ANN paths to remain available until qualification succeeds, so that introducing ANN does not force an unverified replacement.

Implementation Decisions

  • Confirmed: support both Similar Items and For You in the existing deployment. The later PyTorch two-tower request supersedes the original model exclusion; an external vector service remains excluded.
  • Confirmed: An opted-in For You path may search eligible Items across the published Catalog rather than only rerank the current snapshot candidate union. The default path remains unchanged.
  • Implemented candidate: deterministic truncated SVD over the current weighted metadata feature matrix (configured maximum 512 dimensions by default), L2-normalized float32 vectors, cosine-equivalent inner product, and Faiss CPU HNSW over eligible Items. This candidate is not a claim of commerce relevance or ANN recall qualification. Existing sparse Similar Items and bounded For You remain the default and fallbacks.
  • The existing Shopper Profile item affinities, whose projection weights views 1, cart additions 3 and purchases 5, form the For You query. Recent and negative Items are excluded; absent usable vectors or profile signal uses the ordinary fallback. An acknowledged interaction changes the query without mutating the index.
  • A scope opt-in publishes a checksummed artifact with its Recommendation Snapshot. The control database stores the immutable vector/ID mapping and native index together with the serving-head transaction; readers validate scope, version, shape and checksum. Retrieval uses exact eligible search for small indexes or ANN underfill.
  • Preserve source/serving separation. Serving uses a published snapshot generation and personalization projections; inference here means bounded query construction and retrieval, not synchronous training.
  • Use composition and cohesive object ownership for representation, retrieval and ranking. Keep native-library types and index-specific settings inside the retrieval module. The loaded-index module includes an exact eligible-vector search path; qualification still compares it with an independent exact reference.
  • Preserve existing authorization and causal semantics, response limits and privacy rules. ANN results use low confidence and metadata-similarity provenance; their cosine scores are not calibrated recommendation confidence.
  • The index contains only snapshot-eligible Items. Retrieval excludes Similar Items anchors and recent/negative Shopper Items, deduplicates, and uses bounded exact search when ANN underfills. Exact search itself returns fewer than K only when fewer eligible nonexcluded Items exist or positive-score filtering removes the rest.
  • Similar Items continues to publish its existing precomputed Recommendation Sets as fallback; its opted-in ANN lookup is request-time. For You also retrieves at request time against published artifacts.

Testing Decisions

  • Test seams: existing Training Run to publication to Serving API flow, plus the loaded-index retrieval interface for exact-reference and resource tests.
  • Retain prior Serving API, Similar Items, personalization authorization, causal-token, fallback and scope-isolation tests. Assert observable IDs, eligibility, provenance and snapshot identity, not private dispatch calls.
  • Add a catalog-wide discovery scenario whose expected For You result is deliberately absent from the old global candidate union; verify an acknowledged profile change can change the query while keeping the same snapshot.
  • Test empty and undersized catalogs, missing metadata/profile Items, zero and nonfinite vectors, duplicate vectors, restrictive filters, quota underfill and invalid authorization. Numeric behavior and fallback choices must be specified before their test expectations are finalized.
  • Evaluate exact search on the new representation against the current sparse baseline separately from ANN versus exact same-vector search. Keep temporal holdout and cold/sparse Item cohorts distinct.

Out of Scope

Custom ANN algorithms, a separate vector-search service, cross-Commerce-Scope discovery, incremental live index mutation, merchant-source reads during serving, deployment and hosted-setting changes. Production rollout of the experimental two-tower objective remains gated by relevance and resource evidence.

Further Notes

Broad rollout is gated by representation and acceptance evidence, not authorized by the presence of this default-off implementation. Related decisions: Choose metadata and Shopper query representations; Define ANN quality and resource acceptance.

The Embedded ANN decision map retains rollout and qualification gates. The ready-for-agent label routes specification work; it does not waive blockers or assert broad deployment approval.