Embedded ANN for Similar Items and For You: first-release evidence¶
Status: research, 2026-09-25. This is an evidence review and proposed decision direction, not an approved product specification or production qualification. It uses original papers, first-party library documentation, first-party engineering blogs, and the repository's local experiment. Blog deployment reports are contextual examples, not independent validation for this service.
Conclusion¶
The first decision is the Item representation, not the ANN library. Preserve the existing sparse Item Metadata cosine implementation as a control and safe fallback. Trial at least two dense challengers: a versioned projection of the current categorical/text features (for example truncated SVD) and a pretrained product-text/sentence embedding. Select one only after commerce-specific Similar Items judgments and time-correct For You evaluation show it is useful. A 512-dimensional SVD projection retained 85.1% of the existing sparse top ten in one generated Catalog, but overlap measures change from the old model, not whether either model is relevant. Local experiment, randomized SVD foundation, Sentence-BERT paper, metadata-embedding recommendation paper.
Once a representation passes relevance evaluation, compare exact and approximate nearest-neighbor search on the same normalized vectors, eligible population, queries and exclusions. Faiss CPU Flat is the exact oracle; Faiss CPU HNSW is a plausible initial approximate candidate, not an automatic choice. Original HNSW and ANN-Benchmarks describe a recall-versus-latency engineering tradeoff, not an improvement in recommendation relevance. HNSW paper, ANN-Benchmarks paper, Faiss index guide.
Representation options and what evidence does not prove¶
| Option | Why test it | Limitation / release condition |
|---|---|---|
| Keep current sparse categorical, word and character TF-IDF cosine | Established current behavior, exact control, explainable metadata signals | It is not a Faiss dense-vector input; serving currently materializes top candidates at Training Run time. Current design. |
| Project the current sparse features to dense with truncated SVD | Reuses the known feature contract and can give every described Item a vector; lightweight offline build | Rank drift is real even before ANN (local top-ten overlap 0.797 at 256 dimensions, 0.851 at 512 on one smoke Catalog). Do not infer relevance or choose a dimension solely from overlap. Local experiment, randomized SVD paper. |
| Embed title/description, with structured metadata handled explicitly | Purpose-trained sentence embeddings make cosine semantic retrieval practical; may capture synonyms absent from lexical features | General sentence similarity is not a guarantee of substitutability, price/category discipline, multilingual coverage, or merchant relevance. It also introduces model versioning, inference, artifact and license obligations. Test on actual product pairs and profile tasks. Sentence-BERT paper, semantic product-search paper. |
| Learn metadata-aware recommendation embeddings from interaction data | Hybrid content/interaction embeddings have documented cold-start potential | A new learned objective, labels and validation protocol materially expand v1; keep this as a later challenger rather than silently folding it into an ANN infrastructure change. LightFM paper. |
For You's first query can use the approved bounded, authorized Shopper Profile to aggregate vectors of acknowledged Items, with purchases weighted above views and empty/unusable profiles following the existing non-personalized path. That is a proposed simple profile-query policy, not a two-tower model. The weights, recency window, treatment of repeated actions, exclusion of already-interacted Items, and cross-Catalog rejection must be versioned and evaluated rather than guessed from the papers. Existing personalization contract, repository context.
Evaluation should have three separate gates¶
- Model relevance: compare the current sparse baseline and dense challengers using full eligible-Catalog ranking, merchant- or expert-judged substitute pairs for Similar Items, and chronologically held-out acknowledged interactions for For You. Slice by strict-cold versus warm Items, category, title length, missing metadata, language, popularity, Catalog size and profile depth. Report Recall/NDCG at requested K, catalog coverage and disqualifying pairs. Avoid random leave-one-out temporal leakage and evaluating only sampled negatives; both can change apparent model rankings. Data-leakage study, sampled-metric study.
- Index fidelity: on a selected dense representation, measure recall@K against exact Flat on identical vectors, selectors and exclusions; report fill, duplicate/ineligible/anchor violations and results per eligibility-selectivity bucket. Sweep HNSW build/search parameters and at least Flat/HNSW alternatives. A proposed numerical recall threshold is an engineering decision, not a threshold established by these papers. ANN-Benchmarks paper, Faiss index guide.
- Operational fit: measure end-to-end serving p95/p99, build/serialize/load, warm/cold startup, high-water RSS across workers/scopes/overlapping generations, and failure/rollback. The current local index-only benchmark cannot certify these. Local experiment, Faiss index guide.
The representation and index gates must not be collapsed: 100% HNSW recall reproduces the dense model exactly, even if that model is worse than the sparse baseline.
Filtered retrieval and Apple Silicon¶
Faiss officially supports ID-based selectors, but its FAQ says subset filtering is limited and an index containing only the relevant elements is generally more efficient. HNSW can return missing (-1) IDs when too few nodes are visited. At 0.1% eligibility in the local 200k-Item stress, HNSW with efSearch=256 filled only 32.7% of top ten; the exact eligible scan filled them. Thus route very small eligible sets to exact scan and use an exact eligible-set fallback when approximate search underfills. A single globally increased efSearch is not a substitute for selectivity-aware evaluation. Faiss selectors, Faiss FAQ, local stress, filtered-ANN comparative study. The exact routing threshold remains to be measured on the selected representation and workload.
The pinned faiss-cpu==1.15.1 package publishes a macOS 14+ ARM64 wheel, so the user-approved macOS 26+ floor does not block it. Faiss documents CPU Flat and HNSW and ARM SIMD support. Apple's Accelerate provides optimized CPU BLAS, but its existence does not prove Faiss HNSW calls Accelerate or uses the Neural Engine. The local M2 Pro wheel reported ARM_NEON and a Metal bridge; only CPU Flat/HNSW were validated for selector-correct retrieval. A tiny Metal test returned an excluded ID, and an unfiltered Metal Flat p95 was slower than CPU Flat in one local run. Treat Metal/ANE acceleration as a separate future experiment, not a v1 dependency. PyPI wheel, Faiss code structure, Apple Accelerate, local experiment.
An alternate HNSW implementation is not automatically better. For example, hnswlib's official Python documentation warns that its Python callback filter is slow under multithreaded search because of the GIL. Compare any proposed alternative on identical filtered queries, recall, packaging, memory, concurrency and lifecycle tests before switching. hnswlib documentation.
First-party Medium engineering accounts illustrate the same distinctions at much larger scale: Pinterest describes the shortcomings of fixed overfetch when query filters have different selectivity, and Airbnb describes learned listing embeddings evaluated for both similar listings and real-time personalization. Their proprietary traffic and model choices do not establish that the same representation or filtered-search design wins for this Catalog. Pinterest Engineering, Airbnb Engineering.
Suggested decision for the implementation plan¶
Authorize a quality-gated, opt-in v1: existing sparse Similar Items and existing For You paths remain fallbacks; prototype dense SVD and product-text vectors; select a representation only with relevance evidence; then qualify exact Flat and CPU HNSW over that representation, with selectivity-aware exact search and hard eligibility validation. Publish the chosen vector version, Product ID mapping and index as one scoped, integrity-checked snapshot generation; preserve the previous serving head on any build or load failure. Keep two-tower training out of scope. The representation and ANN numerical launch gates, memory cap and exact-routing threshold should be recorded as explicit decisions after measurement, not attributed to literature.