E-commerce recommendation models and system architecture: a critical literature review¶
Literature cutoff: 5 September 2026. Prepared for: an applied ML scientist rebuilding a multi-property commerce recommendation service. Repository examined: recommendations, commit 63a3af00322f5cc38b9497e84b8c06350f09ea92.
Status: research and architectural recommendations, not an approved implementation specification. The production code was inspected to contextualize the research; no implementation, training, production experiment, or performance qualification was performed.
Executive assessment¶
The appropriate ambition is a recommendation decision system whose retrieval, ranking, constraints, page composition, feedback, and experimentation improve together. Selecting a large neural architecture alone cannot establish enterprise relevance. My recommended starting direction is a hybrid stack: retain reliable aggregate strategies as baselines and degradation paths, add intent-specific learned retrieval, introduce a supervised contextual ranker when exposure data supports it, and optimize the final page under explicit commerce constraints. Keep advanced sequence and generative models as challengers with measurable entry criteria. This is a synthesis of the evidence below, not a claim that this architecture has already beaten alternatives on your traffic.
Three observations govern the recommendation:
- Model feasibility depends on what the events mean. Orders support co-purchase learning without customer identities. Ordered sessions support short-term sequence learning without persistent identities. Cross-session personalization and online-to-store attribution require additional identity and observation contracts. The current repository exposes the first two types of grouping, not a longitudinal shopper key. Repository source contract.
- A recommendation strategy is a user intent, not a model family. Similar products, compatible accessories, replenishment, trending products, and an inspirational homepage have different labels and constraints. A shared encoder can support several tasks, but a single undifferentiated similarity score is not sufficient. A recent industrial two-tower case study supports this distinction; the detailed evidence appears in Part III. Osowska-Kurczab et al., 2025.
- The frontier is real but conditional. The review covers semantic-ID generation, scalable sequential transducers, generative slates, and 2026 e-commerce deployments. Reported commercial results are evidence about the authors' particular interventions and populations. They are not portable uplift estimates. The source notes explicitly identify preprints, proprietary evaluation, metric mismatches, and contradictory reporting.
How to read this review¶
- Executive assessment
- Part I — Problem formulation and data foundations
- Part II — Recommendation modeling: foundations to the 2026 frontier
- Part III — Retrieval systems and contextual ranking
- Part IV — Evaluation, causal learning, reranking and whole-page optimization
- Part V — A complete recommendation stack for this enterprise
- Part VI — Assessment of the repository and research-to-production roadmap
- Complete references
The modeling section develops mathematical foundations before discussing modern architectures. The evaluation section separates prediction, causal value, and experimentation. The architecture section translates the literature into a complete conditional stack, followed by an assessment of the existing repository and a staged experimental program.
Scope, search method, and evidence quality¶
This is a broad, critical narrative literature review, not a preregistered systematic review or an exhaustive ranking of every paper. Searches covered implicit collaborative filtering, item-to-item and basket models, sequential/session recommendation, multi-behavior and multimodal learning, retrieval and ANN, CTR/CVR and multi-task ranking, generative recommendation, slate/page optimization, counterfactual evaluation, exploration, and production systems. Searches combined canonical paper titles with subject queries such as “e-commerce generative retrieval 2025/2026,” “whole-page recommendation,” and “delayed conversion recommendation.” New-paper search results were checked against source publication/version dates; the cutoff refers to content available by the date above, not a crawler's apparent publication date.
Sources are original papers from authors' arXiv submissions, conference/proceedings hosts and institutional research pages, plus first-party software documentation and repository source code. Secondary search results were discovery leads only. Abstract-only evidence supports high-level descriptions, not reconstructed experimental protocols. Full-text examination is identified where used for equations or numerical findings. Complete references are consolidated at the end, grouped by topic; their prefixes distinguish modeling (M), evaluation (E), and architecture/ranking (A). Citations remain beside the relevant claims. Some sources appear in more than one part; these are the same study, not independent confirmations.
Evidence should be interpreted on two independent axes: scientific access (full text and artifacts versus abstract or first-party report) and ecological validity (public benchmark, offline proprietary study, or reported online experiment). Peer review does not make a private A/B result reproducible; open code does not prove business impact; an industry engineering article is not a controlled comparison. Negative results, protocol sensitivity, and reporting ambiguities are retained because they affect which experiments are worth running.
Confirmed requirements and remaining uncertainties¶
Your answers materially narrow the recommendation: homepage personalization, long-term retention, purchase conversion, and margin per session are the priority outcomes. The stack is Python/FastAPI on AWS, operated by three people, with catalogs of hundreds of thousands of products, potentially extreme Black Friday traffic, isolated training per client, and under 500 ms p99 end-to-end recommendation API latency.
You confirmed upstream availability of recommendation impressions and displayed positions; stable identities connecting online and store activity; attributes, images and text; inventory and prices; returns; and consent information. These data exist upstream, but the current recommendation source contracts do not expose all of them. This distinction is central to the implementation roadmap.
| Still unresolved | Treatment in this review |
|---|---|
| Exact peak QPS, request burst shape, concurrent properties, homepage lanes and items per page | No replica count, capacity guarantee, or cloud cost estimate is invented; provide sizing equations and a qualification plan |
| Relative priority and acceptable tradeoffs among margin, conversion and retention | Recommend a constrained objective and measured Pareto frontier; no unapproved numerical weights |
| Retention horizon, accounting definition of margin, session definition and attribution window | Explicitly define before experimentation; example horizons are illustrative |
| Logging-policy probabilities and randomized exposure coverage | Impressions and positions are available, but randomized propensities were not confirmed |
| Data freshness targets, availability SLO and regional failover requirements | Explain choices and keep the current snapshot path; no invented freshness guarantee |
| Whether the existing ban on service-side raw-event staging remains binding for new functionality | Present a design that can keep raw data upstream and note which alternatives need a revised data-retention contract |
The existing V1 specification has stricter latency targets for its simpler snapshot endpoint (p95/p99 90/150 ms), a 12-hour batch objective, and a 200k-item/100M-view/100M-purchase qualification profile. These are requirements, not measured capacity, and are not automatically the capacity or latency of the new personalized homepage path. Preserve the old endpoint's contract unless explicitly revised. Named customers are not a substitute for a traffic forecast.
Part I — Problem formulation and data foundations¶
I.1 Recommendation is conditional decision-making¶
Let $p$ denote a Commerce Property, $t$ the decision time, $u$ an optional shopper identity, $h_{u,t}$ the history observable by $t$, $c_t$ request context, $a$ an optional anchor item, $b_t$ an optional current basket, and $\ell$ a recommendation intent or placement. Define the request state
$$x_t=(p,u,h_{u,t},c_t,a,b_t,\ell).$$
Unavailable components are missing, not inferred identities. Let $\mathcal I_p(t)$ be the property-local catalog and $\mathcal E(x_t)\subseteq\mathcal I_p(t)$ the eligible items. A policy returns an ordered slate $S=(i_1,\ldots,i_K)$, or an ordered page of slates $P=(S_1,\ldots,S_L)$. At the business level the target is
$$\pi^* = \arg\max_\pi \mathbb E[Y(P)\mid\operatorname{do}(P\sim\pi)]\quad\text{subject to commerce, latency and resource constraints}.$$
Here $Y$ must be chosen with the business. Most recommendation training instead minimizes a predictive surrogate over logged interactions. The two objectives coincide only under additional assumptions. An observed purchase is evidence of demand under the historical experience, not proof that recommending the item caused that purchase. The evaluation literature in Part IV formalizes this gap.
Terminology: retrieval/candidate generation finds a manageable pool; ranking estimates contextual utility of its members; reranking constructs a list with interactions and constraints; page or swimlane orchestration selects and orders multiple lists. “Filtering” below means business eligibility and result filtering. Collaborative filtering is a separate term for learning from shared behavior.
I.2 What the supplied data identifies¶
| Data unit | What can be learned directly | What it does not establish |
|---|---|---|
| Product view | Observed interest or navigation conditional on how the shopper arrived | Whether a recommendation was displayed, whether other items were rejected, or whether the view was positive satisfaction |
| Ordered browsing session | Short-term transitions, multi-interest shopping context, session-to-item scoring | Stable preferences across visits unless linkage is supplied |
| Online order and line items | Basket association, quantity, purchase popularity, possibly value if monetary fields exist | Exposure attribution, which line item caused another purchase, or universally compatible relationships |
| Offline order and line items | Store-channel basket relationships and demand patterns | Online impression attribution or customer linkage from matching SKU/time alone |
| Catalog metadata | Textual/attribute similarity and constraints available in that snapshot | Historical availability, authenticated compatibility, or buyer preference |
| Recommendation impression, once exposed through the service contract | What was displayed at which position under a specified policy | Causal uplift without a suitable design or sufficient assumptions |
A useful event schema is not merely a bigger table. It needs distinct definitions for event identity, decision/request identity, property/catalog identity, product/variant identity, event time, arrival time, and observation source. In a proposed future data contract, separate rendered impressions, viewable impressions, product-page views, add-to-cart events, purchases, cancellations, and returns. Choose an attribution definition and do not switch it between training and reporting. These are recommended contracts, not assertions about data already collected.
I.3 Implicit feedback, exposure and negative examples¶
Observed interaction matrices are sparse and missing-not-at-random. A zero can mean “never exposed,” “unavailable,” “unnoticed,” or “not wanted.” Confidence-weighted implicit factorization and pairwise ranking make useful modeling assumptions about those zeros; they do not turn them into explicit dislikes. The mathematical treatments appear in Part II.
For retrieval, sampled unobserved items are contrastive competitors. Their sampling distribution is part of the objective. For an impression-response ranker, a matured, actually displayed nonresponse is a different label. For product compatibility, an expert-confirmed incompatible pair is a different label again. Keep these negative types distinct, and prevent known positives, duplicate variants, and delayed conversions from being accidentally treated as equivalent negatives.
Illustrative failure: a laptop and its charger may co-occur frequently, but a high-volume unrelated accessory can also co-occur because of a promotion. A similar laptop may receive many co-views because shoppers compare alternatives. Training one relation from their pooled counts can confuse substitutes, complements, and merchandising exposure. Different relation types need different labels, eligibility logic, and evaluation.
I.4 Combining website, online and offline evidence¶
The least demanding option is channel-aware aggregate learning: compute order associations and popularity by channel, then measure whether pooling improves online-task validation and eventual online outcomes. The repository currently pools scoring semantics while preserving channel-scoped order identity. This prevents accidental merging of online order 123 and store order 123; it does not establish that those orders belong to one shopper. Source contract, pipeline work store.
With valid identity linkage, a richer event representation can combine item, action type, channel, timestamp/time gap and contextual features. Use separate targets or task-conditioned heads for browsing and purchase behaviors. With no identity linkage, do not manufacture a cross-channel user sequence. Offline basket evidence can still help complement retrieval and cold online items whose store demand is known.
A proposed mixture is $s(a,i)=\alpha(x)s_{online}(a,i)+(1-\alpha(x))s_{store}(a,i)$ after appropriate score calibration, or a shared encoder with channel-specific heads. The choice and $\alpha$ are empirical, not a fixed 50/50 rule. Account for differing assortments, store geography, stock availability, promotions, fulfillment mode, and coverage of identified shoppers. Evaluate online-only, store-only, pooled, and channel-conditioned variants with identical online target windows. A higher pooled offline metric is insufficient evidence of an online improvement.
A second distinction is behavioral freshness versus label maturity. Recent views can immediately update session state, while a store purchase may arrive days later. Train and replay with both event time and the time the fact became available. An old event discovered today must not appear in a historical prediction made yesterday.
Part II — Recommendation modeling: foundations to the 2026 frontier¶
II.1. Define the prediction problem before choosing a network¶
The request notation from Part I applies. Your confirmed linked identities and rich upstream signals permit personalized modeling, while the current service requires new ingestion contracts. The same models also need deliberate anonymous and missing-history paths.
At least six distinct targets need separate definitions:
| Target | Conditioning information | Suitable positive evidence | Why it differs |
|---|---|---|---|
| Similar alternatives | Current product and shopping constraints | Comparison, substitution, or supervised similarity judgments | Should preserve the purpose of the current product |
| Complementary products | Product or partial basket | Compatible additional products; basket additions | Recommends a different role in the shopping mission |
| Next item in the session | Ordered recent interactions | The next observed interaction of a specified type | Captures immediate intent, including anonymous sessions |
| Personalized discovery | Shopper history and current context | Future qualified engagement or purchase | Requires stable identity to use cross-session behavior |
| Replenishment | Prior purchases, quantities, elapsed time | Repeat purchase within a defined horizon | Previously purchased products are often desirable |
| Next basket or page | Basket history or current page context | A set of items or response to an entire page | Items and categories compete or complement one another |
The distinction between substitutes and complements has an explicit early formulation in McAuley, Pandey, and Leskovec: supervised prediction of different product-relation graphs from product-associated information. Their Amazon study uses reviews and product features, not an assertion that every co-view is a substitute or every co-purchase is a complement. M07
Assessment. A single relevance label cannot faithfully express all six tasks. A laptop-to-laptop comparison and a laptop-to-compatible-dock suggestion may both be useful, but in different lanes and stages of the journey. Compatibility can be a hard relation supplied by catalog knowledge; behavioral co-occurrence alone is insufficient to establish it. For baskets, line-item order in a receipt is not necessarily a behavioral sequence. For replenishment, purchased-item exclusion would remove the target we are trying to predict. These are task-definition consequences, not reasons to abandon shared representations.
Implicit feedback is the main statistical difficulty. A view or purchase is an observation generated by exposure, availability, intent, presentation, and opportunity; a missing interaction is not direct evidence of dislike. Hu, Koren, and Volinsky formalize different confidence levels for implicit observations. BPR instead learns pairwise preferences between observed and unobserved items. Neither makes observational feedback into randomized preference measurements. M01, M02
Assessment. Start with typed events rather than one scalar “engagement.” Preserve event time, ingestion time, tenant, product variant, session, event source, and order/basket membership. Preserve a shopper identifier only when its linkage is established. The event target must be defined independently of the model. “Purchase in seven days” and “next view” have different label delays, sample sizes, and operational meanings. A purchase-focused retriever should not silently become a view predictor because views are easier to collect.
II.2. Classical models remain essential scientific controls¶
II.2.1. Confidence-weighted implicit matrix factorization¶
With interaction count $r_{ui}$, a canonical formulation is
$$ p_{ui}=\mathbf{1}[r_{ui}>0],\quad c_{ui}=1+\alpha r_{ui}, $$ $$ \min_{P,Q}\sum_{u,i}c_{ui}(p_{ui}-p_u^\top q_i)^2+ \lambda(\lVert P\rVert_F^2+\lVert Q\rVert_F^2). $$
Alternating least squares exploits the low-rank structure while accounting for unobserved entries at low confidence. The confidence is not a calibrated probability. This is a useful baseline for stable preference and dense recurring-customer histories, with compact item vectors that support efficient retrieval. Pure ID factors cannot infer a new item's identity embedding from its description. M01
Assessment. For session-only data, one can factorize session–item or basket–item matrices, but that changes the interpretation: latent session factors are not persistent customer profiles. Inference for a new session requires an aggregation or fold-in strategy. It would be misleading to label this shopper personalization without that distinction.
II.2.2. Bayesian Personalized Ranking¶
For triples $D={(u,i,j):i\text{ observed},j\text{ unobserved}}$, BPR minimizes
$$ \mathcal L_{BPR}=-\sum_{(u,i,j)\in D}\log\sigma(f(u,i)-f(u,j))+ \lambda\lVert\theta\rVert^2. $$
It is a ranking objective that can train MF or other differentiable scorers; it is not a particular neural architecture. The induced task depends on how triples are sampled. Its 2009 UAI publication predates its 2012 arXiv upload. M02
Assessment. A sampled $j$ is usually “not observed under these conditions,” not a verified rejection. Choose a sampler over the correct tenant and time-specific catalog. Exclude future information from that decision. For purchase prediction, a viewed-but-unpurchased item can be an informative negative after the target window matures; it is not equivalent to an item that was never presented.
II.2.3. Learned item-to-item models: SLIM and EASE¶
SLIM learns a sparse item coefficient matrix from historical interaction profiles with L1/L2 regularization and a zero diagonal; the sparse matrix supports fast scoring by aggregation. It is an interpretable, strong item-based baseline with an author-maintained implementation. M03
EASE solves
$$ \min_B\lVert X-XB\rVert_F^2+\lambda\lVert B\rVert_F^2, \quad\operatorname{diag}(B)=0. $$
Writing $P=(X^\top X+\lambda I)^{-1}$, its off-diagonal coefficients are $B_{ij}=-P_{ij}/P_{jj}$, with $B_{jj}=0$. Scores are $X_uB$. Negative coefficients are permitted; these are learned conditional associations, not necessarily a symmetric similarity. The zero diagonal prevents identity copying. EASE's dense item–item matrix and inversion constrain catalog scalability despite very simple training code. M04
Assessment. Test both raw co-occurrence and learned item coefficients on the same temporal split. A moderate per-tenant catalog may make EASE practical, whereas a huge catalog may require sparse neighborhoods, decomposition, or another retriever. Do not eliminate an inexpensive baseline because it predates transformers: it can expose whether new results come from better training or merely weaker comparators.
II.2.4. Item2Vec and metadata factorization¶
Item2Vec applies skip-gram with negative sampling to item co-occurrence and explicitly works without user information. Basket/session grouping supplies contextual relations; the output is item embeddings. It is directly relevant when aggregate item relationships are available but persistent customer identity is not. M05
LightFM represents users and items through sums of latent metadata-feature vectors, providing a bridge from pure collaborative factors to content-aware cold-start prediction. A new item can inherit factors from known attributes. The evidence supports hybridization in sparse and cold-start settings on the paper's datasets, not a universal guarantee that metadata wins. M06
Assessment. Item2Vec is a natural first learned item-relation experiment that fits the current service contracts; your richer upstream data also supports the personalized alternatives below. For metadata models, audit quality, historical coverage and consistency of the confirmed metadata and media, including technical attributes and variant relations. Different feature availability should produce explicit experiments, rather than letting “metadata” mean an undefined collection of fields.
II.3. What neural collaborative filtering and graphs contribute¶
II.3.1. Neural interaction functions are not automatically better retrieval models¶
NCF replaces or supplements dot-product interactions with a learned nonlinear user–item function. It established an influential way of learning interaction functions from implicit feedback. However, Rendle and colleagues' re-examination finds that adequately tuned dot products outperform the proposed MLP similarities in the revisited experiments, and highlights the difficulty of learning multiplication and efficiently retrieving with a generic MLP. M08, M09
Assessment. This distinction remains operationally important. A dot product decomposes into a precomputable item vector and a request vector. A generic cross-network over a request and each product normally requires scoring many pairs. The latter can be valuable after retrieval, where rich interactions matter, but is a different cost regime. Do not compare them solely by parameter count.
Mult-VAE models an implicit-feedback vector with a multinomial decoder and an annealed KL regularizer:
$$ \mathcal L=-\mathbb E_{q_\phi(z\mid x)}\sum_i x_i\log p_\theta(i\mid z) +\beta\operatorname{KL}(q_\phi(z\mid x)\Vert p(z)). $$
Its contribution includes evidence for the importance of the likelihood and regularization, not simply nonlinear capacity. It is a valuable unordered-history neural baseline. A full-catalog output layer can become expensive as the item universe grows. M10
SimpleX explicitly compares interaction encoders, losses, and negative-sampling ratios across 11 datasets and 29 comparator models. It reports that a simple encoder with a suitable contrastive objective and many negatives can beat substantially more complicated models. M11
II.3.2. Graph propagation is a collaborative inductive bias¶
LightGCN removes feature transformations and nonlinear activations from a graph CF backbone, retaining normalized neighborhood propagation and a weighted combination of layers:
$$ e_u^{(k+1)}=\sum_{i\in N(u)} \frac{e_i^{(k)}}{\sqrt{|N(u)||N(i)|}},\qquad e_u=\sum_{k=0}^{K}\alpha_k e_u^{(k)}. $$
The item update is analogous and final scores are dot products. Its result is evidence that the useful graph operation can be simpler than a generic GNN. Ordinary ID-based LightGCN does not create content representations for genuinely new disconnected items. M12
PinSage demonstrates a different, industrial graph route: efficient random-walk neighborhoods, graph convolutions with item features, increasingly difficult training examples, and distributed embedding inference. Its Pinterest evidence includes production experiments at graph scale, with 3 billion nodes and 18 billion edges in the reported setting. This is evidence that graph models can be engineered for production; it is not evidence that such a graph or operational budget exists for this enterprise. M13
SGL adds an auxiliary contrastive task to graph recommendation, making representations of a node agree across graph perturbations. Its paper evaluates long-tail accuracy and noise robustness. SimGCL questions the need for expensive graph-structure augmentations and instead perturbs embeddings, relating performance to representation uniformity. Both illustrate that representation regularization can matter as much as adding propagation depth. M14, M15
Assessment. Separate three hypotheses: graph connectivity improves collaborative structure; item content improves cold start; auxiliary contrastive learning regularizes representations. Test each independently. A content-aware two-tower model may capture much of the benefit without a graph pipeline. Graph gains are most plausible when useful higher-order relationships survive tenant and time boundaries. They are less compelling when most entities have only one edge, new inventory dominates, or graph refresh cannot match business changes.
For anonymous traffic, a session–item or basket–item graph is feasible without a shopper graph. That changes the semantics of higher-order neighbors and should be reflected in the name and evaluation of the model. The graph must be built only from data available at the training cutoff. A graph assembled from all historical and future interactions can leak labels even if the final loss uses a chronological train/test split.
II.4. Session and sequential recommendation¶
II.4.1. Session models and the ordering of evidence¶
GRU4Rec adapts recurrent networks to session-based recommendation, including session-parallel training. A hidden state summarizes preceding events, producing next-item scores. It is foundational because session recommendation can work without long-lived customer IDs. A recurrent model also offers a compact state update, though its representation and training choices must be compared with simpler session baselines. M16
Assessment. A timestamped session stream is more useful than a set of session products if the task is next-item prediction. Sessionization rules influence labels: a 30-minute inactivity boundary is not automatically appropriate for considered purchases or cross-device journeys. Preserve the ability to reconstruct session definitions in the upstream authorized data environment; this does not require a new service-owned raw-event copy. For training, a target product must not already occur in an input event representing the same transaction through another channel.
II.4.2. SASRec: the reference causal transformer¶
SASRec embeds prior item IDs and positions, applies causally masked self-attention and feed-forward blocks, then scores the next item using the resulting state. In general notation,
$$ h_t=\operatorname{Transformer}{causal}(e{i_1}+p_1,\ldots,e_{i_t}+p_t), \quad f_t(i)=h_t^\top e_i. $$
Its original formulation uses sampled binary cross-entropy and studies both sparse and dense sequential datasets. Order alone is not elapsed time: a learned position at $t-1$ does not reveal whether the previous action occurred one minute or three months earlier. M17
Assessment. A well-tuned causal sequential retriever is a more informative modernization baseline than jumping directly from co-occurrence to a billion-parameter model. Include event type and elapsed-time representations as explicit experimental extensions. Compare a full sequence with only the last event and with shuffled histories; these ablations test whether the additional sequential machinery uses information beyond recent-item popularity.
II.4.3. BERT4Rec and the architecture–loss confound¶
BERT4Rec uses bidirectional attention with randomly masked items and a Cloze-style training objective. At recommendation time, a final mask requests the next item. Bidirectional training can exploit context around an internal training mask; serving still uses only the available history. Future test interactions must never enter that history. M18
Petrov and Macdonald's Turning Dross Into Gold Loss shows that switching SASRec to cross-entropy with appropriate negatives can outperform BERT4Rec in their comparisons. Their conclusion challenges attributing historical gaps exclusively to bidirectionality. M19
Assessment. A fair experiment fixes catalog, event filtering, training horizon, validation budget, embeddings, negative policy, and evaluation candidate set before comparing encoders. Report both wall-clock training budget and number of sampled negatives. “SASRec” is otherwise an ambiguous baseline: an older implementation with one negative is not equivalent to a tuned modern training configuration.
II.4.4. Multiple interests and repeat consumption¶
MIND uses dynamic routing to form multiple user-interest vectors for Tmall retrieval. This avoids requiring a single vector to summarize every interest. ComiRec likewise extracts multiple interests and adds a controlled aggregation stage that trades accuracy against diversity; its reported Alibaba deployment is on an offline distributed platform, which should not be described as a proven online A/B win. M20, M21
RepeatNet explicitly mixes repeat and explore modes for session recommendation. Its repeated-consumption mechanism chooses products from the observed history while an exploration component scores other products. This makes repeat behavior an explicit modeling choice rather than a filtering afterthought. M22
TIFU-KNN models personalized item frequency for next-basket recommendation and compares it with RNN-based alternatives on four datasets. The important result is that recurrence and frequency need not be automatically learned well by a generic sequence architecture. The task predicts a basket, not an arbitrary ordering of its lines. M23
Assessment. Multi-interest retrieval is useful to test when a single sequence may mix several shopping missions. It is not the same as explicit lane diversity: several interest heads can all return the same popular category. Replenishment should receive separate evaluation slices, horizons, and exclusion policies. For groceries or consumables, frequency and elapsed-time baselines are mandatory. For expensive durable goods, the replenishment hypothesis may be weak; the accessory or next-mission hypothesis may be stronger. Which case applies must come from tenant data, not the prestige of a model family.
II.5. Combining views, online purchases, and in-store purchases¶
II.5.1. Multi-behavior models provide useful mechanisms, with important assumptions¶
MBGCN represents several feedback types in a unified graph, learning behavior influence through user–item propagation and behavior semantics through item–item propagation. The paper directly addresses the failure of treating all interactions as equivalent. M24
MB-GMN uses graph meta-learning to adapt representations to heterogeneous behavior types and personalized dependencies. MB-CGCN instead encodes a presumed behavior chain, carrying representations from earlier behaviors into later ones. Their benchmark improvements support studying behavior structure; a universal click→cart→purchase funnel does not follow from those experiments. M25, M26
Assessment. An in-store purchase is not necessarily downstream of a recorded online view. A customer may discover in the store, browse online after buying, or purchase through an unlinked household identity. Consequently, placing offline orders at the final step of an online funnel without evidence would inject false supervision. Missing online precursors are frequently unobserved data, not skipped steps or negative events.
A practical proposed event representation is
$$ z_t=e_{item(i_t)}+e_{behavior(b_t)}+e_{channel(c_t)}+ g(\Delta t_t)+e_{context(x_t)}, $$
with separate heads for view, online purchase, and store purchase where label coverage supports them. This is a design sketch, not a claim that additive embeddings are optimal. Concatenation with projections or cross-attention between behavior streams are alternatives. The model should retain the channel distinction until controlled experiments demonstrate that pooling is safe and useful.
The corresponding multitask objective can be written as
$$ \mathcal L=\sum_{b\in\mathcal B}\lambda_b \mathbb E_{(H,i,y_b)\in D_b}\ell_b(f_b(H,i),y_b) +\lambda_{reg}\Omega(\theta). $$
The meaningful decisions are how to define $D_b$, mask unknown labels, set observation horizons, and avoid letting abundant views overwhelm rare purchases. The coefficients are optimization choices, not objective business valuations. Learned weighting also cannot decide what the organization means by relevance.
II.5.2. Two distinct routes for store orders¶
Assessment — if identity is unlinked. In-store baskets can still train item-to-item complements, product popularity by store/region/time, basket embeddings, and product representations. Evaluate whether those candidates improve online recommendations after accounting for catalog and availability differences. Do not join a store order to a web session through unsupported guesses. No persistent shopper ID is required for basket co-occurrence, but reliable basket boundaries and product reconciliation are required.
Assessment — if identity is linked. Join events into a shopper history with time and channel provenance, while retaining whether linkage is deterministic or uncertain. A longer-term purchase representation can be combined with current-session intent. Evaluate store-only, online-only, and mixed-channel histories separately. A useful model could improve recommendations for frequently identified loyalty shoppers while doing nothing for anonymous visitors; aggregate averages would conceal that limitation.
Assessment — if only some customers are linked. The linked population may differ from the full population. Use the extra history when present, provide an explicitly trained missing-history path, and evaluate both populations. Treat unsupported channel outcomes as missing labels rather than zero purchases. Order cancellation, returns, quantities, bundles, and variant reconciliation require explicit semantics before they become targets or exclusions.
II.5.3. Remaining measurement questions¶
Stable identity, impressions and displayed positions, media/attributes, inventory/prices, returns and consent are confirmed upstream. Behavioral training must remain isolated by client. Remaining measurements include linkage coverage/error rates, reliable cross-system ordering and ingestion delay, product-versus-variant semantics, historical catalog snapshots, and the outcome horizons and tradeoffs. Their values determine useful history length, channel-loss masking and validation cohorts; do not assume that availability implies complete coverage.
II.6. Content, multimodality, and cold start¶
VBPR augments latent preference factors with learned projections of pretrained image features. It gives an early explicit demonstration of visual preference modeling and alleviation of item cold start in implicit recommendation. Its relevance is greatest when appearance matters; it does not establish that generic image similarity captures technical compatibility. M27
UniSRec learns transferable item and sequence representations from item text, with lightweight adaptation and contrastive pretraining across domains. It provides a route beyond domain-specific atomic IDs and evaluates transfer across recommendation domains/platforms. M28
HLLM divides the problem into an Item LLM that encodes product text and a User LLM that models sequences of those item representations. The paper studies pretrained weights, fine-tuning, and scaling, using Amazon Reviews and PixelRec plus an online experiment. This is an LLM representation approach; it is not simply prompting a chat model to invent a shopping list. M29
UniRec (TMLR 2026) explicitly treats text, images, categorical features, and numeric attributes as distinct modalities. It encodes attribute name/type/value and applies a hierarchical Q-Former to nested item histories. This is an important schema-aware direction: price, time, and rating are all numeric but have different meanings. Benchmark evidence is reported; the inspected abstract does not establish a production deployment. M30
Assessment. Cold start has several cases that should not be collapsed: new SKU with an established parent product, entirely new product, new tenant with metadata but no behavioral labels, new shopper in an established catalog, and anonymous visitor with one current-session action. A content encoder addresses some item cases. It cannot reconstruct a missing shopper preference from no evidence. A default or context-based policy remains necessary.
For an enterprise benchmark, compare at least three representations under matched serving conditions: ID-only collaborative, content-only, and hybrid. Evaluate new products defined by their first eligible timestamp, not merely products whose training interactions were randomly removed. Include missing-modality tests and product families with near-duplicate images or titles. Metadata may be incomplete, generated, or inconsistent; a high-quality encoder cannot correct a false product specification by itself.
Cross-client behavioral parameter sharing is prohibited by the confirmed requirement. Transfer-learning papers therefore inform methods that can start from permitted public pretrained representations and adapt independently per client. They do not authorize pooled customer histories, shared private behavior encoders or shared private reward models. Compare client-local learning curves and fallback quality, including smaller clients that cannot sustain the largest models.
II.7. Generative recommendation: several different scientific programs¶
“Generative” describes materially different outputs and learning problems. The following categorization is this review's synthesis:
| Family | Output space | Main question |
|---|---|---|
| Probabilistic reconstruction | Item-interaction vector | Can the model recover useful preferences from a noisy sparse vector? |
| Semantic-ID retrieval | Discrete codes that map to catalog items | Can decoding replace or complement vector nearest-neighbor retrieval? |
| Sequential transduction | Future items or actions conditioned on item/history tokens | Can shared sequence computation improve scaling and target-aware ranking? |
| Slate/session generation | Several jointly generated recommendations | Can the model optimize the response to a set or session? |
| Language-assisted recommendation | Textual representations, queries, categories, explanations, or item IDs | Which part benefits from language pretraining and controlled generation? |
II.7.1. Diffusion and language unification¶
DiffRec models interactions by denoising, deliberately limiting corruption so that personal information is retained. Its extensions compress item dimensions into latent clusters and incorporate timestamp weighting. Experiments support this modeling direction on three datasets; they do not establish superiority for an enterprise with strict request latency and a dynamic catalog. M31
P5 unifies recommendation tasks in a text-to-text pretrain/prompt/predict framework, representing interactions, metadata, and reviews as language sequences. It is a foundational investigation of task unification and transfer, rather than proof that free-form text should be the final production recommendation format. M32
Assessment. For any iterative denoising or language-generation model, evaluate total inference work, batchability, output validity, freshness, and catalog-constrained recall. A model that produces a semantically plausible phrase is not yet a retriever that can identify the exact available variant. An explanation's fluency must not be used as a relevance metric or evidence that its stated causal reason is true.
II.7.2. TIGER and semantic identifiers¶
TIGER quantizes item content into a tuple of semantic codes and autoregressively predicts the next item's code:
$$ P(c_1,\ldots,c_L\mid H)=\prod_{\ell=1}^L P(c_\ell\mid H,c_{<\ell}). $$
It uses a residual-quantization autoencoder and sequence-to-sequence transformer. The original benchmarks are three filtered Amazon review categories with roughly 12–18 thousand items and mean histories near nine, rather than enterprise-scale impression streams. The paper explicitly acknowledges beam-search inference cost; it also reports invalid generated IDs. Semantic sharing helps cold-item generalization in its experiments. M33
Assessment. A semantic code is a model/index contract. Monitor collisions, code utilization, valid-item yield, new-item insertion, and codebook/version compatibility. Changing the tokenizer changes the model's vocabulary, not merely one input column. Evaluate both generated-code recall and actual eligible-item recall. At serving time, a valid code can still map to an unavailable product; eligibility is a separate condition.
II.7.3. HSTU: actions as a scalable modeling modality¶
HSTU reformulates retrieval and ranking as sequential transduction. Ranking predicts an action after an interleaved target item; retrieval predicts future positive items. Its blocks combine pointwise projections, non-softmax attention with position/time bias, normalization, and multiplicative gating. Training shares computation across targets; M-FALCON amortizes inference. Table 7 reports online +12.4% E-Task and +4.4% C-Task, separately from offline normalized entropy. The named tasks are engagement/consumption proxies on a large platform, not ecommerce revenue. The celebrated trillion-scale count includes the overall recommendation parameter regime; it should not be read as the cost of a dense language transformer. M34
Dong and colleagues' 2025 context-parallelism paper addresses jagged recommendation tensors and reports a 5.3× increase in supported history length. This is systems evidence for scaling long-sequence HSTU, not direct evidence that longer histories improve this enterprise's users. M35
Assessment. The architecture is worth studying because it makes the feature representation, learning objective, GPU kernels, batching, and inference amortization one coordinated design. Copying the block while retaining incompatible training examples may miss the main advantage. Conversely, retaining some engineered summary features can be sensible when histories are missing or deliberately truncated. The next experiment should be sized to available behavior density and measured serving cost rather than the largest published parameter count.
II.7.4. OneRec and preference alignment¶
OneRec combines encoder–decoder generation, sparse experts, session-wise outputs, and iterative preference alignment using a learned reward model. It reports +1.6% watch time in Kuaishou deployment. The reward model scores alternatives that cannot all be simultaneously shown to the same user; these synthetic preferences are not observed counterfactual outcomes. M36
OneRec-V2 replaces the architecture with a lazy decoder and adds alignment from actual user feedback. Its full report describes a 1B model, context 3000, beam 512, a 5% traffic group over one week, and 36 ms reported inference on L20 GPUs. App Stay Time improved +0.467%/+0.741% relative to OneRec-V1 on Kuaishou/Kuaishou Lite. A separate 1% experiment with caching disabled reports cold-start video-view declines of 44.7%/36.7%, illustrating a coverage tradeoff despite engagement gains. These are configuration-specific author results; the reported inference latency is not a p99 API guarantee. M37
OneRec-Think adds text-grounded reasoning to generative recommendation and reports a +0.159% App Stay Time result through its industrial “Think-Ahead” design. This is useful evidence that reasoning can be operationalized, but the small platform-specific uplift should not be conflated with general reasoning accuracy or causal explanations. M38
Assessment. Alignment is attractive because the observed next item is a noisy proxy for the desired outcome. However, optimizing a learned reward more aggressively can exploit its errors. The comparison should include a conventional ranker consuming the same new features, a generative model without alignment, and the aligned model. Otherwise, the gain may come from added information or supervision rather than joint generation. A retained strong ranker can serve as a teacher during a staged transition; the final deployment need not be decided in advance.
II.7.5. The ecommerce-specific 2025–2026 frontier¶
TBGRecall (2025) proposes next-session prediction using session tokens and sets of item tokens, addressing the mismatch between autoregressively ordered next-item generation and retrieving several items without a required within-request order. Its training combines limited historical pretraining with partial incremental updates, and the paper reports public-benchmark and Taobao industrial experiments. Its emphasis on data recency is a useful challenge to a purely “more historical tokens” scaling strategy. M39
OneMall (2026) adapts semantic tokenization, compressed multi-behavior histories, sparse experts, and ranker-derived reinforcement-learning rewards to Kuaishou commerce. Source inconsistency: its abstract labels product-card +13.01% as GMV, but §4.3/Table 3 label it clicks and report GMV +14.71%; the abstract's short-video +15.32% “orders” is Table 3 exposure, with orders +11.65%; live-streaming +2.78% is exposure, with orders +4.47%. Treat precise headline uplift as provisional until clarified. Its model architecture is relevant first-party evidence; its public numerical reporting has a material caveat. M40
GrocLM (2026) targets grocery categories with two-stage LoRA tuning for rebuy cycles and sequential relationships, plus trie-constrained decoding. The full paper reports +7.5% relative cart-adds per impression for production restocking. This is category recommendation, not exact-SKU ranking or revenue uplift. Its source metadata displays a June submission date with a July-form arXiv identifier; both are before the review cutoff, and the mismatch is retained rather than silently “corrected.” M41
HPGR (2026) adds session hierarchy and preference-guided sparse attention to generative recommendation, with structure-aware pretraining and preference-aware fine-tuning. Its abstract reports APPGallery industrial/offline and online evidence against models including HSTU. This supports investigating structured selection of history, but does not establish ecommerce transfer or a universally optimal history compressor. M42
Assessment. These sources change the 2026 answer: production generative recommendation is no longer only a short-video extrapolation. There is direct commerce research. Nevertheless, heterogeneous tasks, private datasets, evolving preprints, reporting inconsistencies, and compute differences prevent a universal model ranking. The scientific target is a repeatable enterprise comparison across well-defined tasks, with a measured Pareto frontier of utility, latency, cost, reliability, and coverage.
II.8. Proposed model portfolio and experiments¶
The following is a research program within the confirmed client-isolated, three-person, AWS setting. Session-only rows describe anonymous/degraded cohorts, not the full upstream data available to you. Frontier models are challengers, not a recommendation to train a platform-scale system from scratch.
| Data/need | Required control | First serious challenger | Later frontier experiment | Key decision evidence |
|---|---|---|---|---|
| Session/order groups without shopper identity | Co-occurrence, popularity, item-neighborhood methods | Item2Vec, SLIM/EASE where catalog size permits | Content-informed item graph or generative session retrieval | Temporal eligible-item recall and added value over the current product |
| Ordered anonymous session | Last-item transitions and session-neighbor baseline | GRU4Rec and tuned SASRec | Session-aware transduction or TBGRecall-style objective | Gains over last-event and shuffled-history ablations |
| Stable shopper identity and rich history | Implicit ALS/BPR and recency/frequency | Sequence/hybrid retriever; multiple-interest retrieval | HSTU, OneRec-style generation | Incremental gain beyond current session, per history-length cohort |
| New catalog items | Attribute/category defaults and content nearest neighbors | Hybrid metadata/text/image representations | UniSRec/HLLM/semantic-ID model | New-item exposure, conversion, eligibility, and cold-start recall |
| Consumable repeat buying | Personal frequency plus recency | Explicit repeat/explore or next-basket model | GrocLM-style category generation | Repeated versus new-item performance, category/SKU consistency |
| Complement lane | Co-basket plus catalog compatibility constraints | Relation-specific product embeddings | Supervised relation graph or conditional generation | Compatibility precision and incremental basket utility |
Assessment. Keep a portfolio of candidate sources until experiments establish that consolidation helps. A source with lower standalone Recall@K may contribute rare relevant items missed by every other source. Measure marginal recall after union and deduplication, not just each source's isolated leaderboard position. Conversely, multiple sophisticated generators that all return the same items add operational cost without coverage.
Use a model comparison ladder to identify the reason for any gain:
- Same labels and events, improved sampler/loss. This tests training quality before changing capacity.
- Same objective, improved metadata. This tests information rather than architecture.
- Same data, sequential encoder. This tests ordered context.
- Same request information, larger or alternative encoder. This tests capacity/inductive bias.
- Same model, multi-behavior or cross-channel auxiliary learning. This tests transfer rather than silent pooling.
- Same candidates, stronger downstream ranking. This separates retrieval failure from ordering failure.
- Same logging/constraints, joint generation or alignment. This tests consolidation after the previous factors are controlled.
Do not require every experiment to preserve an unrealistically identical optimizer if that disadvantages a model; instead publish the search space and tuning budget, and compare at matched resource envelopes. Multiple seeds are useful when their variability is material to the deployment decision. Report uncertainty and the absolute metric as well as relative uplift. A 20% improvement from 0.010 to 0.012 is numerically and commercially different from 0.50 to 0.60.
For modern generative candidates, record a model card with vocabulary construction, positive definitions, repeat-item policy, target horizon, metadata versions, total versus active parameters, sequence-length distribution, beam/sample size, invalid/duplicate output rates, catalog-update latency, hardware, and request batching. A claimed latency without percentile, load, cache state, and candidate volume is not a service-level guarantee.
For omnichannel models, start with a diagnostic study: amount of incremental store-history information after controlling for online history, overlap of offline and online catalogs, order delay distribution, proportion of unlinked customers, and relative frequency of repeat purchases. If the extra source has little coverage, a sophisticated multi-channel encoder may be lower priority than reliable ingestion and product reconciliation. If it contains many repeat purchases absent online, the potential value is different and more targeted.
II.9. How to interpret this literature scientifically¶
Dacrema, Cremonesi, and Jannach's reproducibility study found that many examined neural methods failed to improve on adequately optimized simpler alternatives, while replication itself was often difficult. This is a critique of empirical practice, not a proof that neural recommendation cannot work. M43
Assessment. Each reviewed claim belongs to an evidence tier: original formulation; offline benchmark; industrial offline replay; reported production experiment; or independently reproduced enterprise result. This note reaches the first four tiers depending on the paper and does not claim the fifth. Publisher acceptance supports provenance and review, but does not make a result transferable across catalog, objective, exposure policy, or business regime.
The strongest reading path is: implicit-feedback formulation and BPR; item-based baselines and EASE; NCF re-examination and SimpleX; SASRec/BERT4Rec with the loss re-examination; relation-specific and multi-behavior models; then TIGER, HSTU, OneRec-V2, and OneMall. This sequence develops the statistical and systems vocabulary needed to interpret current frontier work without confusing algorithm novelty with enterprise value.
The review is deliberately selective rather than a census of all named GNN, contrastive, diffusion, and LLM variants. Its model shortlist should be revised after data profiling and product-target clarification. A model that performs well on views can be retained as one candidate source even if it is not the purchase ranker. A language model can improve item representations without appearing in the request-critical path. An aggregate item-to-item model can remain useful after personalization arrives. These are compatible design choices.
Part III — Retrieval systems and contextual ranking¶
III.1 Retrieval and ranking solve different computational problems¶
Covington, Adams and Sargin's YouTube paper describes separate deep candidate generation and ranking, illustrating the practical separation between finding plausible items in a large corpus and evaluating a much smaller set with richer features. Its evidence is from video recommendation, so the relevance to this project is the decomposition, not the watch-time objective. A01.
For a two-tower retriever,
$$s_{ret}(x,i)=f_\theta(x)^\top g_\phi(i),\qquad C(x)=\operatorname{TopM}{i\in\mathcal E(x)}s{ret}(x,i).$$
The item representation can be computed ahead of time. A downstream ranker $s_{rank}(x,i)$ may use arbitrary feature interactions and candidate-conditioned attention, because it operates on $C(x)$. This factorization is an architectural restriction, not just an implementation trick: it makes cheap corpus search possible but limits how context and item interact during retrieval. A reranker cannot recover a relevant product missing from $C(x)$.
Recommended retrieval portfolio for your homepage: recent-session candidates, long-term personalized candidates, category/brand affinities, complements to relevant prior purchases, replenishment where suitable, semantic/content discovery, and property-local popularity/trending. Start with a small, interpretable portfolio and measure each source's unique relevant coverage and online contribution. Union and deduplicate before expensive scoring; preserve all contributing source features. Candidate counts and quotas are tuning variables, not universal constants.
III.2 Learning the retrieval distribution¶
A sampled softmax-style objective contrasts a positive $i^+$ against a candidate sample $\mathcal B$:
$$\mathcal L_{ret}=-\log\frac{\exp(\tilde s(x,i^+))}{\sum_{j\in\mathcal B}\exp(\tilde s(x,j))}.$$
Yi et al. study two-tower retrieval with in-batch negatives and correct the induced popularity sampling bias using estimated item frequencies. A typical corrected logit includes a $-\log q(i)$ term, with details depending on the exact sampling scheme. Their deployment is YouTube retrieval. This correction addresses training-sampler bias, not missing exposure or causal policy bias. A02.
For this system, explicitly record the sampler, replacement/deduplication behavior, treatment of positive collisions, and temporal item eligibility. Mix broad exploration of the catalog with difficult retrieved competitors only as a measured experiment. Hard negatives can be false negatives: two good substitutes may both be relevant even if only one was purchased. Do not take the raw retrieval score as a calibrated purchase probability or add raw scores from heterogeneous sources without validating the combination.
III.3 A directly relevant industrial bridge: Allegro¶
Osowska-Kurczab et al. (2025) describe content-based two-tower models using text and structured attributes for similarity, complements and inspiration. Complement retrieval changes the architecture with category conditioning; inspiration also changes serving/index traversal. The product-page similar and complementary models were evaluated as fallback generators to co-view/co-purchase models, an unusually close analogue to this repository. The paper reports production throughput of 20k requests/s and 40 ms p99 CPU latency; this is author-reported system evidence, not an AWS sizing result. It demonstrates the viability of learned content fallbacks and reuse across intents, without implying that all intents should share identical representations. Full text §§2–3 inspected. A03.
III.4 ANN indexing is part of recommendation quality¶
HNSW organizes vectors in a navigable hierarchical proximity graph; its construction and search controls trade search effort, recall and memory. The original work is general nearest-neighbor search, not a recommendation relevance model. A04. Johnson, Douze and Jégou study GPU similarity search including exhaustive and approximate/compressed approaches; Faiss provides implementations and maximum-inner-product search. A05, Faiss documentation.
The engineering implication is to measure two separate quantities:
$$\operatorname{ANNRecall@M}=\frac{|C_{ANN}(x)\cap C_{exact}(x)|}{M},\qquad \operatorname{TargetRecall@M}=\frac{|C(x)\cap G(x)|}{|G(x)|},$$
where $G(x)$ is task ground truth and the exact/approximate comparison uses the same eligible corpus and score. Perfect ANN recall says nothing about whether the learned embedding captures purchase intent. Report both before and after hard filtering, and stratify by filter selectivity and catalog age.
For an illustrative 200,000-item catalog with 256-dimensional float32 embeddings, raw vectors occupy $200000\times256\times4=204.8$ MB, before index, metadata, replicas or framework overhead. Exhaustive single-query scoring requires about 51.2 million multiply-accumulate terms. These arithmetic estimates show why exact search should be a benchmark and correctness oracle, while actual peak traffic may justify ANN. They are not latency predictions. Avoid an all-pairs dense 200k-by-200k matrix unless its memory cost is deliberate: one float32 matrix alone is 160 GB decimal.
Filtering matters as much as the index. Simple post-filtering may return too few results after stock, region and policy constraints. OpenSearch's official documentation distinguishes filtering within vector search from post-filtering and exact filtered scoring; support depends on engine/version. Verify the actual AWS-supported version rather than assuming every latest upstream feature exists there. A06, AWS k-NN documentation.
III.5 Ranker families: what to compare¶
| Family | Mechanism and evidence | Role in your experimental program |
|---|---|---|
| Logistic regression and boosted trees | Logistic regression is a useful calibrated-response reference; XGBoost supplies scalable tree boosting with sparse-data handling A07 | Start with auditable source, recency, affinity, price and inventory features; strong baseline under a three-person operating constraint |
| Wide & Deep | Joint linear/memorization and neural/generalization components; Google Play industrial evaluation A08 | Useful conceptual baseline when explicit feature crosses remain valuable |
| DeepFM | Shared feature embeddings feed factorization-machine and deep components to learn low- and higher-order interactions A09 | A compact embedding-interaction challenger, not an automatic upgrade over trees |
| DCN-V2 | Explicit cross layers plus a deep network; full-matrix or low-rank mixtures increase interaction expressivity A10 | Strong structured-feature neural challenger with an explicit cost/quality tradeoff |
| DIN | Candidate-specific activation over historical behavior gives a different interest representation for each candidate; Alibaba advertising evidence A11 | Test when your linked histories are informative and candidates are few enough for conditioned scoring |
| Sequence encoder plus prediction heads | Causal history encoder supplies shared request representation; optionally add candidate interaction downstream | Compare to simple recency-weighted pooling before making long-sequence serving mandatory |
| Multi-task gated networks | Shared experts with task-specific gates, as in MMoE A12 | Candidate for click, purchase and other genuinely distinct targets; inspect negative transfer |
DCN-V2's cross layer is
$$x_{l+1}=x_0\odot(W_lx_l+b_l)+x_l.$$
It makes explicit feature interactions available alongside implicit MLP interactions; low-rank factorization reduces cost. The paper reports both benchmark studies and Google production gains. Full text §3 inspected; this review does not infer a comparable uplift on your features. A10.
A generic candidate-conditioned history summary is $z(x,i)=\sum_{j\in h}\alpha(i,j,x)e_j$. It is informative when a shopper has several interests and the current candidate selects the relevant one. DIN uses a local activation mechanism; the equation here is an explanatory abstraction, not a claim that DIN requires normalized softmax attention. A11.
III.6 Supervision and economic utility¶
Your confirmed impressions make a response ranker feasible. Construct one training example per agreed exposure unit, joining only features available before the decision. Distinguish clicks, cart additions, completed purchases, cancellations/returns and eventual net value. A product-page view alone must not be retroactively relabeled as a displayed recommendation.
For binary response $y$ and predicted probability $\hat p$, binary cross entropy is
$$\mathcal L_{BCE}=-\sum_n[y_n\log\hat p_n+(1-y_n)\log(1-\hat p_n)].$$
Pairwise logistic losses optimize ordered preferences; listwise softmax losses compare candidates within a request. Their scores need not be calibrated probabilities. A listwise objective also changes meaning when candidates, negative sampling or request grouping change. Use the same candidate distribution at training and serving as far as possible, and retain held-out candidate-generation versions for diagnosis.
The proposed business policy should make tradeoffs explicit. For example, optimize expected net margin while imposing guardrails on conversion and a defined retention measure. A candidate-level approximation could use
$$U_i(x)=\Pr(\text{purchase}_i\mid x,\text{exposure})\,\mathbb E[\text{net contribution}_i\mid x,\text{purchase}].$$
This is a predictive heuristic, not incremental profit and not a complete page objective. Net contribution must have a consistent treatment of discounts, cost of goods, returns, fulfillment and other relevant costs. If returns are already incorporated into conditional net contribution, do not subtract them a second time. Margin per session uses all eligible sessions, including zero-purchase sessions; measuring margin only among purchasers changes the objective and can conceal lost conversion.
Learn response predictions separately from the policy's economic tradeoffs. A training loss $\sum_k\lambda_k\mathcal L_k$ expresses optimization balance; its $\lambda_k$ are not automatically the business value weights used for serving. Monitor calibration by property, surface, device, product age and value band. A high AUC can coexist with poor probability calibration and poor top-of-list recommendations.
III.7 Conversion funnels and multi-task learning¶
ESMM models impression-to-click-to-conversion dependence via
$$p(\text{click and conversion}\mid x)=p(\text{click}\mid x)\,p(\text{conversion}\mid\text{click},x).$$
Its shared representations and entire-impression-space supervision address the clicked-sample selection/sparsity problem under that funnel. The experiments use Taobao data. It does not generally eliminate all confounding or identify counterfactual conversion. Full-text model and losses inspected. A13.
For your stack, online and store purchases may occur without a recommendation click. Therefore the ESMM click-conditioned chain cannot be assumed to explain every transaction. Define a post-click conversion target where appropriate, and separate broader attributed/observed purchase or store outcomes. Include delayed-label treatment from Part IV.
MMoE shares expert functions $f_j(x)$ with task-specific gates: $z_k(x)=\sum_jg_{kj}(x)f_j(x)$ followed by task head $h_k$. Its purpose is learning task relationships while allowing different mixtures. A12. A later YouTube ranking paper reports practical multi-objective ranking and selection-bias mitigation; it supports treating competing targets explicitly, not optimizing clicks alone. A14.
Recommendation for three people: establish independently useful purchase and value targets first, with click as an auxiliary signal if it helps. Compare a shared encoder with small heads to a compact gated model only after learning curves and per-task metrics show a reason. Retention requires a user/time-window target and mature experiments; assigning the same eventual retention label to every displayed item creates correlated supervision and weak attribution.
III.8 Newer retrieval systems do not erase the engineering tradeoff¶
Meta's first-party Andromeda report describes learned hierarchical ads retrieval co-designed with specialized hardware and downstream ranking. Its relevance is that retrieval itself can model richer interactions than a simple factorized ANN pipeline. It is an ads engineering report with proprietary implementation, not a reproducible off-the-shelf commerce model. For this team's scale and isolation requirements, it motivates measuring retrieval expressivity and index alignment rather than reproducing Meta's hardware architecture. A15.
Monolith combines collisionless dynamic embeddings, expiry/frequency filtering and online training. Its lesson is to treat freshness, memory and synchronization as joint design variables. It does not establish that per-event weight updates are required for this commerce system. Fresh session features, stock updates and newly encoded products can often be introduced independently of model retraining; their respective gains should be measured. A16.
Part IV — Evaluation, causal learning, reranking and whole-page optimization¶
IV.1. Define what a successful recommendation means¶
The central scientific distinction is between predicting an observed action and estimating the value of changing the recommendation policy. A product view is evidence about behavior under the historical website, assortment, exposure mechanism, prices and user intent. A missing purchase is not a labeled dislike. Schnabel et al. formalize recommendation observations as missing not at random and connect recommendation risk estimation to treatment assignment. Their result is conditional on the observation/propensity assumptions, not a theorem that arbitrary behavioral logs can be debiased. E1
Proposed estimand, subject to business agreement: let a policy $\pi$ choose a page $A$, given request context $X$, and let $Y_H(A)$ be customer value over a specified horizon $H$. The business question is
$$ \Delta_H=\mathbb E[Y_H(A\sim\pi_{new})-Y_H(A\sim\pi_{old})]. $$
This differs from $P(\text{purchase}\mid X,i,\text{historically shown})$, and also from revenue attributed to a clicked recommendation by an accounting rule. High predicted purchase probability can merely identify customers who were already going to buy. Views, web purchases and store purchases do not by themselves reveal the missing counterfactual.
Design synthesis: settle a primary value metric before model selection. Examples include purchases, net revenue, contribution margin, successful product discovery or longer-term repeat purchasing per randomized eligible customer. GMV ignores some costs and returns; click-through rate can favor tempting products that do not satisfy demand. Keep CTR, add-to-cart and conversion as diagnostic intermediate outcomes, with cancellations/returns, empty results, latency and relevant inventory availability as guardrails. Do not silently choose profit over customer relevance: the tradeoff is a product decision. Microsoft's research documents repeated cases where apparently favorable metric movements were misinterpreted, supporting explicit metric semantics and denominator scrutiny. E2
For omnichannel commerce, three definitions must remain separate: the transaction channel; the identity link connecting a person/session to transactions; and the causal influence of recommendations. The confirmed stable identity linkage makes store purchases usable for features and customer outcomes. It does not establish that a preceding web recommendation caused that purchase. An observed click can support an attribution rule without establishing incremental lift. These are deductions from the estimand, not a claim that identity infrastructure is missing.
IV.2. Build a credible offline evaluation before scaling models¶
Ji et al. demonstrate that ignoring the global interaction timeline can leak information even with leave-last-one-out splits performed separately for each user. Their experiments show that future-item information changes model comparisons; a chronological-looking per-user split does not guarantee realistic evaluation. E3
Proposed evaluation protocol: use global training, validation and final test intervals, with rolling origins when seasonality and catalog churn matter. Fit preprocessing, vocabulary, popularity statistics, item graphs, negative samplers, calibration and hyperparameters using information available at each training cutoff. Construct each feature using both event time and the time it became available to serving. A store purchase that happened on Monday but entered the platform Wednesday cannot inform Tuesday's simulated recommendation. Preserve historical assortment, product-family membership, availability and content snapshots where permitted. If only today's catalog is available, explicitly describe the backtest as approximate and measure sensitivity to changing eligibility.
Use validation for choosing neighborhood sizes, support thresholds, shrinkage, model family and reranking weights; report final results once on the untouched test period. A holdout that determines hyperparameters is validation regardless of its variable name. Deduplicate events and define sessions, baskets and repeat purchases before splitting; avoid allocating individual events from the same transaction into both training and labels. Freeze an evaluation manifest containing time boundaries, entity definitions, inclusion rules, label horizon, candidate policy and metric formulas.
For implicit ranking, with a request's relevant held-out set $G_x$ and ranked list $R_K(x)$, conventional definitions are
$$ \operatorname{Recall@K}(x)=\frac{|R_K(x)\cap G_x|}{|G_x|},\quad \operatorname{DCG@K}(x)=\sum_{j=1}^K\frac{2^{g_{xj}}-1}{\log_2(j+1)}, $$
with NDCG normalized by the best ordering for those labels. Choose binary or graded gains deliberately. A purchase, a qualified view and a return are not interchangeable labels. Specify whether aggregation weights requests, sessions, users, orders or clients equally; the resulting metrics answer different questions. Explicitly define behavior when $|G_x|=0$, and disclose the population excluded by requiring test positives. These definitions are evaluation design, not a causal reward model.
Krichene and Rendle prove that metrics computed by ranking positives against a small random item sample can disagree with full-catalog comparisons even in expectation. Very small samples make nominally different metrics behave increasingly like AUC. Corrected estimators help, but the sampling protocol remains part of the experiment. This is a reason to reject comparisons of “NDCG@10” numbers whose negative pools differ. E4
Proposed stage diagnostics: report retrieval recall within the historically eligible catalog, ranking metrics conditional on the retrieved pool, and end-to-end metrics after filtering, reranking and fallback. A reranker cannot recover an absent candidate, and evaluating only the ranking stage hides upstream misses. Exact eligible-catalog evaluation is preferable where tractable; if candidate sampling is necessary, report distribution, count, seeds, any correction and a smaller exact benchmark. Separately report ANN approximation recall against exact nearest neighbors; this is a systems fidelity metric, not user relevance.
Evaluate new/returning users, anonymous sessions, new/head/tail products, product families versus SKUs, repeat versus discovery purchases, web/store-linked customers, devices, surfaces and clients. Use confidence intervals clustered at the unit generating dependence, such as customers, instead of treating all clicks as independent. Compare multiple training seeds where stochastic training variance is material. Train and evaluate each client's behavioral models independently, as required. A cross-client statistical summary of separately estimated effects may help planning if permitted, but is different from pooling training data. Uncertainty is particularly important for small clients: a positive aggregate summary can coexist with meaningful tenant regressions.
IV.3. Exposure bias and counterfactual estimators¶
Joachims et al. show how inverse examination propensities can correct position-biased feedback for learning to rank within an explicit click/examination model. A high-ranked item receives more opportunity for examination, so naive click labels mix relevance and presentation. This propensity is not automatically the probability with which the recommender selected an entire slate. Distinguishing examination probabilities, click probabilities and action-policy probabilities prevents a common implementation error. E5
Let logs contain $(x_t,a_t,r_t,p_t)$, where $p_t=\mu(a_t\mid x_t)$ is the probability the logging policy actually assigned to the observed action. For a target policy $\pi$, define $w_t=\pi(a_t\mid x_t)/p_t$. Standard policy-value estimators are
$$ \widehat V_{IPS}(\pi)=\frac1n\sum_t w_t r_t,\qquad \widehat V_{SNIPS}(\pi)=\frac{\sum_t w_t r_t}{\sum_t w_t}. $$
IPS reweights logged actions toward the target distribution. SNIPS normalizes the total weight; it generally has finite-sample bias but can reduce variance and degeneracies during policy learning. Swaminathan and Joachims analyze propensity overfitting in counterfactual risk minimization and motivate self-normalization. Neither estimator invents outcomes for unsupported actions. E6
With a learned reward model $\hat q(x,a)$, the doubly robust estimator is
$$ \widehat V_{DR}=\frac1n\sum_t\left[\sum_a\pi(a\mid x_t)\hat q(x_t,a)+w_t{r_t-\hat q(x_t,a_t)}\right]. $$
Dudík et al. combine reward prediction with propensity-weighted residual correction. Under identification assumptions, consistency is obtained when the relevant nuisance model is correct: the logging probabilities or the conditional reward model. “Doubly robust” does not mean robust to arbitrary hidden confounding, absent support, corrupted rewards or a changed environment. Use sample splitting/cross-fitting when flexible nuisance models and evaluation share data, and quantify sensitivity to their errors. E7
Assumptions and proposed implementation checks:
- Consistency and well-defined action: the logged action includes what could affect outcomes—item, position, surface, presentation and relevant page context—not only an item ID.
- Conditional exchangeability: given recorded context, action assignment cannot depend on an unrecorded variable that also affects the potential reward. Correctly recorded randomized assignment makes this substantially more defensible.
- Overlap: $\pi(a\mid x)>0\Rightarrow\mu(a\mid x)>0$. Deterministic historical ranking rarely supports arbitrary new rankings. Model extrapolation outside support is a modeling assumption, not unbiased empirical evidence.
- Correct probabilities: propensities reflect the policy after eligibility, deduplication and sampling without replacement. A rank score or softmax fitted afterward is not necessarily the original assignment probability.
- Stable target population and reward: delay/censoring and channel measurement must be addressed. Historical population value need not equal next month's population value.
Inspect weight tails, unsupported target mass and effective sample size $(\sum w_t)^2/\sum w_t^2$; this is a diagnostic, not proof of identification or a universal confidence interval recipe. Clipping weights reduces variance at the price of bias. Report clipping sensitivity and compare IPS/SNIPS/DR with uncertainty. When overlap is inadequate, narrow the target policy, gather randomized data or explicitly return “not identifiable”; avoid presenting a numerical estimate as a business lift.
Ai et al.'s Dual Learning Algorithm jointly estimates relevance and examination propensities from clicks. It is an influential alternative to separately fitting a click model, but automatic joint fitting still relies on structural assumptions that make the decomposition learnable. It should not be described as causal identification from clicks without assumptions. E8
For pages, naive importance sampling uses the probability of the complete ordered slate. This becomes statistically difficult as the action space grows combinatorially. Swaminathan et al. propose slate estimators exploiting reward structure to improve sample efficiency. Such estimators require the corresponding structural assumptions; multiplying marginal item propensities is generally invalid for dependent sampling, and itemwise additive rewards miss complementarity or cannibalization. E9
Chaudhari et al. extend slate OPE from expected reward to the reward distribution, with conditions for unbiasedness and consistency and experiments on synthetic data and a MovieLens-based simulator. This is relevant when expected value conceals downside risk or group differences, but simulator evidence does not establish production commerce effectiveness. E10
IV.4. Conversion funnels, delay and omnichannel labels¶
Part III introduced ESMM and the limits of its post-click funnel. The following literature sharpens those limits and extends the targets to delayed value. E11
ESCM² shows that the ESMM construction alone need not yield unbiased CVR estimates and introduces counterfactual risk regularization to address estimation and funnel-dependence problems under its formulation. This matters scientifically: an elegant probability identity and multi-task architecture do not by themselves establish that a learned intermediate head estimates the intended quantity. E12
Chapelle's delayed-feedback model jointly represents eventual conversion and conversion delay. Until sufficient time has passed, an apparent negative can be a future positive; waiting for all labels instead sacrifices freshness. A useful mathematical form is $P(\text{no observed conversion by }d\mid x)=1-p(x)+p(x)S(d\mid x)$, where $p$ is eventual conversion probability and $S$ is survival conditional on eventual conversion. This addresses censoring under the modeled process, not attribution causality. E13—KDD 2014 paper, original full-text mirror consulted.
Recent work moves beyond binary conversion. Li et al.'s TRACE benchmark and READER model address delayed post-click GMV where one click can precede multiple purchases. READER separates repurchase-related behavior using routed branches and adjusts incomplete regression labels. The January 2026 arXiv record identifies the paper as WWW 2026 camera-ready. It provides a particularly relevant research direction if the desired reward is cumulative order value, but the published setting is advertising, and its superiority is benchmark evidence rather than a forecast for web-plus-store attribution. E14
Proposed label design: retain transaction time, ingestion time, transaction/line identity, channel, quantity, currency, cancellations and returns, plus trustworthy customer linkage where permitted. Define web conversion, store conversion and total linked conversion as separate outcomes; preserve an unlinked population instead of treating missing linkage as zero spend. Use fixed, matured horizons for final evaluation, and train with either matured windows or explicit delay models. Preserve refunds that arrive after the purchase; deciding when a net-value label is final is a business choice. Validate customer matching accuracy, coverage and treatment-dependent changes before making omnichannel lift claims.
Views, purchases and stores can use separate heads or event-type tokens in a shared representation, but pooled event weighting must follow the task. Browsing a costly laptop repeatedly may reflect comparison; repeatedly purchasing supplies may reflect replenishment. A fixed “purchase equals ten views” number is not scientifically supplied by the data. Tune any proxy weighting on held-out business-relevant objectives and ablate channel contributions. This recommendation is problem-specific synthesis; none of the above papers establishes the enterprise's correct weighting.
IV.5. Exploration and long-term optimization¶
Li et al.'s contextual-bandit formulation uses user/item context while trading off exploitation and information gathering. Their LinUCB work includes evaluation from randomized Yahoo traffic. The important transferable contribution is the exploration/logging/evaluation loop; its click objective and news domain are not an e-commerce revenue guarantee. E15
Proposed progression: begin with a small, instrumented exploration allocation over already eligible candidates or over interpretable recommendation strategies. Log assignment probabilities and policy versions. Exploration may compare a complementary-products strategy with a replenishment strategy, or diversify new-item exposure; those are different action spaces with different reward credit. Keep high-risk eligibility constraints deterministic and explore inside the feasible set. A/B-test the exploration policy itself against the incumbent, including short-term cost and longer-term benefits; do not assume that exploring more always helps.
A contextual bandit optimizes immediate observed reward for a context and chosen action; it does not automatically optimize effects on future contexts. SlateQ makes RL over recommendation sets tractable by decomposing long-term slate value under user-choice and transition assumptions. It includes simulation and YouTube evidence. A useful form is $Q(s,A)=\sum_{i\in A\cup{\varnothing}}P(i\mid s,A)\bar Q(s,i)$. The no-choice option is essential, and a single-choice decomposition requires scrutiny for commerce baskets with several purchases. E16
DeepPage explicitly studies page-wise recommendation using deep RL, joint item selection and two-dimensional layout, evaluated with a real e-commerce dataset. It establishes page organization as an optimization problem; its experiments are not sufficient reason to replace a production stack with RL before exposure, delayed reward and evaluation are reliable. E17
Engineering inference: separate RL for generating a finite list of item tokens from RL optimizing a shopper's multi-session lifetime value. The former may maximize a learned preference reward at the end of one generated page, while never estimating a customer's future retention. A reward model trained from biased clicks can be optimized perfectly and still prefer the wrong business policy. Use simulations to debug assumptions and stress-test policies, not to certify incremental revenue. Advance to long-horizon RL only after a measurable bandit or supervised improvement baseline, dependable reward collection and a credible online evaluation budget.
IV.6. Reranking: individual scores are not a page objective¶
Pei et al.'s Personalized Re-ranking Model uses a Transformer over the candidate list to capture relationships that pointwise scores omit. It is designed as a modular follow-up to an existing ranker and reports offline and online e-commerce experiments. It is a useful practical stepping stone from independently scored items to context-dependent slate scores, but attention itself imposes no inventory, compatibility or diversity guarantee. E18
Carbonell and Goldstein's MMR provides a transparent greedy baseline balancing relevance against similarity to already selected items:
$$ i^*=\arg\max_{i\notin S}\left[\lambda r(i)-(1-\lambda)\max_{j\in S}\operatorname{sim}(i,j)\right]. $$
Originally developed for retrieval and summarization, it is a reranking principle rather than a learned commerce-specific utility. Its relevance/similarity scales need deliberate alignment; otherwise the numerical value of $\lambda$ is misleading. E19
Chen et al. accelerate greedy MAP inference for determinantal point processes and apply it to recommendation diversity. For positive-semidefinite $L=\operatorname{diag}(q)S\operatorname{diag}(q)$, selecting a set with large $\log\det L_A$ balances quality and representation volume. Their method also considers local repulsion in a sliding window. Exact MAP is generally hard, and generic greedy approximation guarantees should not be asserted without their monotonicity/constraint conditions. A DPP discourages redundant directions; it is not automatically a model of product compatibility or complementarity. E20
Steck's calibrated recommendation aligns the distribution of recommended categories with the user's historical interest distribution, preventing lesser interests from being crowded out. A typical penalty is smoothed $D_{KL}(p_u|q_{u,A})$ combined with utility. This is interest-distribution calibration, distinct from probability calibration such as predicted 10% conversion matching observed frequencies. E21—Calibrated Recommendations, RecSys 2018
Abdollahpouri et al. introduce personalized reranking to improve long-tail representation while retaining useful accuracy, and examine coverage metrics. It supports treating popularity bias explicitly rather than assuming a standard accuracy objective will resolve it. E22
Proposed constrained page objective:
$$ A^*=\arg\max_{A\in\mathcal F(X)}\left{\sum_j d_j\hat u(X,i_j)+\eta\,\operatorname{coverage}(A)-\lambda\,\operatorname{redundancy}(A)-\gamma D(p_X,q_A)\right}. $$
Here $\mathcal F$ encodes hard eligibility and product rules, and the remaining terms are tunable soft objectives. This is a design template, not an empirical result. Candidate availability, product compatibility, required assortment and repeated-SKU prohibitions belong in feasibility, not in small score penalties that can be outweighed. Diversity can be valuable for discovery, yet harmful if it replaces compatible accessories with unrelated categories. Historical calibration can perpetuate exposure bias or miss a new shopping mission. Use surface-specific settings, a Pareto frontier and online validation instead of one global diversity constant.
IV.7. Swimlanes are a coupled whole-page decision¶
Netflix's first-party homepage account describes personalized row selection and ordering, balancing relevance, diversity, freshness and stability. Although its consumption domain differs, it illustrates that organizing recommendation collections is a separate learning problem from scoring individual titles. E23
Proposed commerce composition: represent each candidate lane as (strategy, anchor, candidate set, display budget, presentation, eligibility predicate). A page policy selects lane identities, positions, sizes and item assignments jointly enough to avoid redundant results. “Similar laptops,” “Compatible accessories,” “Buy again” and “Recently viewed” are different user promises. Lane membership should satisfy that promise even when another item's generic propensity is higher. Lane titles are part of the action because they change interpretation and potentially examination.
Independent per-lane optimization fails when the same product occupies several scarce above-the-fold positions, adjacent lanes repeat the same intent, or a top lane changes whether the customer reaches lower lanes. Consequently, a lane's observed CTR is confounded with position and surrounding page composition. Use page-level deduplication with product-family semantics, cross-lane relevance/diversity accounting, visibility-aware logging and tests of lane order. A practical first optimizer can greedily pick the next lane by marginal page utility with a small beam over configurations; global end-to-end page generation is a later candidate, not a prerequisite for coherence.
Two recent primary sources make page generation a live frontier. PageLLM, originally submitted in 2025 and revised in May 2026, combines page-level and item-level learned rewards for PPO alignment. Its authors report GMV +0.44% and CTR +0.14% in an unnamed commerce search system with over ten million users. The full text describes user-ID bucketing and a full-policy treatment, alongside incremental use of its reward as a ranking feature. Offline splits are per user 80/10/10 rather than an explicitly documented global temporal protocol. Treat it as an interesting preprint with author-reported evidence and unresolved reproducibility/transfer questions, not a certified universal best stack. E24
GenRec, identified by its April 2026 arXiv record as SIGIR 2026 camera-ready, uses page-wise next-token prediction, prompt token merging and GRPO-SR preference optimization with relevance-gated rewards. The authors report a month-long JD App experiment with +9.5% click count and +8.7% transaction count over their incumbent. Its relevance here is joint page supervision and constrained reward alignment. Transaction-count lift is not the same metric as GMV or profit; comparison with PageLLM's percentage is not meaningful without common population, incumbent and experimental design. E25
IV.8. Online experiments decide deployment value¶
Proposed experimental contract: predefine hypothesis, eligible population, assignment unit, primary estimand, metric horizon, minimum practically important effect, guardrails, maximum duration and stopping procedure. Persistent customer assignment is usually appropriate for recommendations spanning sessions; session randomization answers a narrower question and can contaminate histories. Analyze all assigned eligible customers for the main intention-to-treat result. Conditioning only on people who clicked, saw a lower lane or purchased after assignment can select different populations in the two arms.
Applying the confirmed priorities: report margin per session as a ratio of total net margin to total eligible sessions in each arm, with user-clustered uncertainty and a documented returns horizon. Since recommendation policy can change the number and mix of sessions, also report margin per assigned customer, sessions per customer, conversion and the predeclared retention outcome. A rise in margin per session accompanied by fewer returning customers may fail the retention objective. Customer assignment can support these measures together; “per session” does not imply session-level randomization. The product team must choose the primary decision criterion or explicit tradeoff, and define retention and maturation windows before power analysis.
For the 500 ms p99 API budget, test latency on the complete serving path, including feature retrieval, inventory filtering, candidate union, reranking and fallbacks. Benchmark peak-load distributions and timeouts separately by client and cache state; mean latency is insufficient. With three engineers, a small interpretable lane optimizer and candidate-level reranker are a more credible first experiment than a bespoke online long-horizon RL platform. This is a resource-aware engineering judgment, not a general claim that complex models cannot meet the SLA.
For a simple balanced two-arm continuous-outcome experiment, an approximate sample size per arm is $2\sigma^2(z_{1-\alpha/2}+z_{1-\beta})^2/\delta^2$. This is an illustrative planning formula; customer dependence, rare outcomes, heavy-tailed revenue, clustering and sequential analysis change the required calculation. No defensible traffic percentage or experiment duration can be chosen from the prompt alone. Estimate variance and detectable lift from the intended population, allow label maturation, and reserve a procedure for multiple concurrent comparisons and repeated peeking.
CUPED uses pre-experiment covariates to reduce outcome variance. In its basic form, $Y^{adj}=Y-\theta(X-\mathbb EX)$, with $\theta=\operatorname{Cov}(Y,X)/\operatorname{Var}(X)$; the idealized variance multiplier is $1-\rho^2$. This improves precision without changing the target effect when applied under the appropriate assumptions. Deng et al.'s original WSDM 2013 paper describes the method and Bing experiments. E26
Lin and Crespo's CLeaR 2026 work extends variance reduction by carefully combining pre-experiment and in-experiment data, with theoretical analysis and Etsy experiments. Its motivation explicitly acknowledges that arbitrary post-treatment covariates can bias estimates. It is useful advanced reading once a basic trustworthy experiment pipeline exists; it does not justify naively controlling for treatment-induced clicks or session lengths. E27
Sample-ratio mismatch can reveal allocation, eligibility or telemetry problems and should be diagnosed before interpreting treatment effects. Microsoft's first-party work describes a taxonomy of such failures. For recommendations, missing exposures after a timeout or treatment-dependent event loss can change measured populations; count assignment, delivery and render separately. E28
Interference matters when experimental customers compete for inventory or recommendations redistribute demand among sellers. Airbnb research uses clusters of similar listings in field experiments to measure and reduce interference bias; this is evidence against blindly applying independent-unit assumptions in a constrained marketplace. User randomization does not automatically remove supply-side interference. Cluster, geographic, seller-side or switchback designs answer different estimands and must account for carryover and their lower effective sample sizes. E29
Proposed release sequence: test deterministic contracts and failure behavior, run historical end-to-end evaluation, shadow traffic to inspect latency/eligibility, run an A/A instrumentation check when the measurement pipeline changes, then a randomized small rollout with predefined analysis. Retain an incumbent and deterministic fallback path, and continue measuring model freshness, fallback frequency, eligible candidate counts and delayed labels. The service-level effect includes these paths: reporting gains only when the new model returns successfully hides part of its treatment effect.
IV.9. Questions that remain enterprise decisions¶
The literature cannot supply the remaining answers: How are retention and conversion defined, over which horizons, and how should retention trade off against margin per session? Does recorded impression data distinguish returned recommendations from actually visible items? Are known randomized policy probabilities available? Are event-time catalog/inventory/price snapshots and historical ingestion timestamps accessible? What Black Friday QPS and load shape must the p99 latency test reproduce? Which clients permit randomized experimentation and how much traffic is available? Is contribution margin fully observable, including returns and fulfillment costs? What distinguishes substitutes, complements, replenishment and promotions in each catalog? These are explicit open questions. Upstream data availability, stable identity, strict client isolation and the 500 ms p99 API target are already confirmed and should not be asked again.
The scientific priority is a reproducible path from an identified task to credible evidence: point-in-time features, explicit candidates and feasibility, well-defined outcomes, informative offline diagnostics, valid exploration logs and randomized business measurement. Model capacity, generative page policies and long-horizon RL are then testable hypotheses within that framework.
Part V — A complete recommendation stack for this enterprise¶
The following architecture is a proposed synthesis tailored to the confirmed requirements, not a description of one paper or of the current implementation. It favors components a three-person team can operate and independently evaluate. It preserves client-isolated training and treats the 500 ms p99 target as a whole-path constraint.
V.1 Logical architecture and ownership¶
flowchart TD
subgraph Data[Client-isolated data and learning]
SRC[Upstream catalog, exposures, views, orders, returns and consent]
PIT[Point-in-time features and matured labels]
TRAIN[Baselines, retrieval, ranking and calibration training]
EVAL[Temporal evaluation and release gates]
ART[Versioned models, indexes, features and fallback snapshots]
SRC --> PIT --> TRAIN --> EVAL --> ART
end
subgraph Serve[Online homepage decision]
REQ[Authenticated property, shopper, context and page request]
STATE[Consent, session state and live eligibility]
RET[Parallel candidate sources with deadlines]
UNION[Union, deduplicate and cheap filtering]
RANK[Batch contextual ranking]
PAGE[Lane selection and constrained item reranking]
VALID[Final live checks and bounded refill]
RES[Page response with request and version IDs]
REQ --> STATE --> RET --> UNION --> RANK --> PAGE --> VALID --> RES
ART --> RET
ART --> RANK
FALL[Last-good models and property-local fallbacks] --> RET
FALL --> VALID
end
RES --> LOG[Assignment, delivery, rendered and viewable exposure logs]
LOG --> SRC
SRC --> STATE
Rendering belongs to the application unless its ownership is deliberately moved. The recommender must still specify lane intent and return decision IDs so that the application can log what was rendered and viewed. Keep logical components modular, but do not create a separate network service for every box. In particular, a compact ranker, deduplication, filtering and lane optimizer can share one serving process until independent scaling warrants separation.
V.2 Data plane: make the available data usable¶
Your upstream data availability removes a major scientific blocker. The engineering work is to expose consistent semantics and reconstruct the information state of each decision.
| Contract | Essential information | Main consumer |
|---|---|---|
| Recommendation request/assignment | Property/catalog, decision ID, permitted shopper/session key, request time, surface, experiment and policy version, eligible population | Serving and experiment analysis |
| Exposure | Decision ID, actual item and lane, lane position/item position, presentation version, rendered/viewable status and time | Response labels and position analysis |
| Interaction | Deduplicable event ID, event type/time/arrival, scope, product/variant, session/shopper linkage, request linkage where valid | Session state and behavioral training |
| Transaction and return | Channel-scoped order/line IDs, quantity, timestamps, currency, economics, adjustments, linked identity | Purchase/value labels and replenishment |
| Historical item state | Effective/arrival time, family/variant, content, inventory, region, price, compatibility, eligibility | Training replay, retrieval and final filters |
| Experiment logging policy | Assignment unit, arm, probability when randomized, sampling mechanism, exclusions and rollout version | A/B and counterfactual evaluation |
| Consent/deletion state | Property-local permissions and revocation/deletion propagation status | Eligibility for personalization and retention lifecycle |
Do not place raw identities in model names, metric labels or public logs. Store only features justified by the task. The existing no-raw-staging requirement can be respected by keeping event-level history in the customer's upstream controlled system and producing training batches/derived features there or through authorized streams. Persisting new service-owned raw impression/history tables would be a changed data contract. Availability of the data does not by itself amend retention rules. Sequential features may themselves encode sensitive behavior; label them as such rather than treating embeddings as anonymous by definition.
Three time scales should be independently configurable: item/stock eligibility updates; shopper/session feature updates; model/index refresh. Test freshness ablations rather than assuming all three need continuous training. Record feature freshness separately from snapshot generation time and model training cutoff.
V.3 Strict client isolation extends beyond rows¶
Train behavioral encoders, task heads, calibration, popularity priors, semantic codebooks fitted on private data, negative samplers and reward models independently by client. Shared infrastructure and shared training code are compatible with isolation; pooled learned behavioral parameters are not. An apparently harmless shared negative queue, global popularity feature, teacher trained on another client's logs, or cross-client fine-tuned content model can violate the requirement.
A public pretrained text/image encoder can be a common starting artifact if its license and data-use terms are acceptable; all private adaptation stays client-local. For a small client, use pretrained semantic features plus simple local models and fallback evidence instead of borrowing another client's behavioral history. Track provenance for base models and adapters. If Catalog boundaries must also remain independent under the existing specification, enforce that stricter scope until explicitly revised.
For operational isolation, bind a caller's authenticated scope server-side, partition indexes and feature keys, test cache-key collisions and forbidden cross-scope retrieval, and constrain resource consumption per client. A metadata filter supplied by the caller is not sufficient authorization. Large clients may warrant separate capacity pools to keep a Black Friday surge from exhausting every other property.
V.4 Retrieval portfolio and score contracts¶
Each source should return item IDs plus source-specific scores, evidence timestamps, model/index version and a status. A source score is meaningful within its own semantics. Preserve multiple source memberships for the ranker instead of retaining only the first source that produced an item. The existing provenance mechanism is a useful base, but a learned union needs more than a single winning tier.
A suggested initial homepage portfolio is:
| Source | Model/logic | Purpose |
|---|---|---|
| Session interest | Recency-weighted item embeddings; later a sequence encoder | Current shopping mission |
| Long-term preference | Local implicit factorization or two-tower user encoder using linked history | Persistent category/brand/style interest |
| Relation candidates | Current co-view/co-purchase and learned content relation models | Related discovery and complements |
| Replenishment | Repeat-history and elapsed-time/category models | Appropriate repeat purchases |
| Cold and exploratory discovery | Content retrieval with a controlled exposure allocation | New items and broader demand discovery |
| Popularity/trending | Existing local decayed rankings with current eligibility | Robust broad coverage and degradation |
This is a set of candidate sources, not six mandatory visible homepage lanes. Candidate source and displayed lane can be decoupled as long as the lane's promise is satisfied. Use quotas initially to prevent one easy-to-score source from dominating, then test quota changes and source ablations. Measure unique contribution at the union and final page, not just per-source recall in isolation.
If retrieval is personalized by multiple interest vectors, limit parallel fan-out and deduplicate early. If an expensive retriever times out, continue with the bounded portfolio already returned. Do not let a low-yield source repeatedly extend the request deadline.
V.5 Hard eligibility, soft policy and live validation¶
Apply relatively static constraints inside retrieval when possible: property/catalog, supported region, product type, valid item lifecycle, intent-specific category/compatibility requirements. Apply dynamic stock, price, delivery, consent and exclusions using a serving-friendly authoritative state. Validate again before returning the final page, since retrieval indexes and cached candidates can lag.
Hard rules should not be represented by tiny score penalties. For an accessory lane, a verified incompatible connector or unsupported model is disqualifying even if the accessory is popular. For “buy again,” previously purchased items may be the point; a global “exclude everything purchased” rule would destroy that lane. For discovery, suppressing already owned durable goods may be appropriate. Product-family deduplication needs exceptions where comparing variants is intentional.
Separate types of failure: missing preference evidence, no eligible inventory, incompatible candidates, source outage, stale features, model timeout, and an overly narrow business rule. A low fill rate has different remedies for each. Enforce a bounded refill operation with the same constraints; do not repeatedly retrieve until the page fills. Returning a shorter list or hiding a lane is a valid result when its user promise cannot be met.
V.6 Whole-page composition and swimlane semantics¶
Define a lane by intent, valid item types/relations, anchor or personalization context, candidate provenance, minimum/maximum size, title semantics and exposure rules. A learned lane score should account for its marginal value given already selected lanes, not simply its standalone click rate.
A practical first page builder enumerates a small set of eligible lane templates, ranks their candidate items in batches, and chooses lanes with a greedy or small-beam search. Penalize duplicate families and repetitive intents; preserve required merchandising constraints as explicit feasibility rules. Keep stable ordering where experiments show that excessive movement disrupts the experience, and measure page-level effects. More complex list-aware models can replace components later using the same logging and eligibility contracts.
A 2026 hybrid example: Hasani et al.'s cascaded storefront system generates placement themes, then catalog-grounded retrieval keywords, filters quality/diversity, and feeds existing item/page rankers. Generated artifacts are cached. The reported online experiment used the teacher for generation; compact student comparisons were offline. Table 2 reports +2.7% cart additions/page view and +1.0%/visit, with respective reported p-values .0001 and .01. These are not completed-purchase or margin lifts. Proprietary data and taxonomy dependence limit reproducibility; narrowed retrieval can also collapse lanes. Full text §§4–6 inspected. For this team, the useful experiment is cached theme generation over approved templates with a deterministic fallback, after the non-generative page baseline works. A17.
V.7 Fallbacks are part of the policy¶
| Condition | Proposed response | Required validation |
|---|---|---|
| Anonymous or consent-restricted visitor | Permitted session/context recommendations or local nonpersonalized lanes | Do not read forbidden persistent history |
| No recent session activity | Long-term local profile, then contextual/popularity candidates | Keep content appropriate to surface |
| New product | Semantic/attribute candidates plus verified relation constraints | Do not invent behavioral support |
| Sparse client history | Local simple models and content; deterministic lane templates | No cross-client behavioral borrowing |
| Ranker timeout or invalid output | Last-good compatible ranker or pre-ranked source/fallback policy | Same hard eligibility and scope rules |
| Retriever outage | Remaining sources plus local snapshot candidates | Record source loss and monitor fill/quality |
| Old snapshot | Last-good candidates with live filtering and explicit freshness | Stale catalog eligibility alone is insufficient |
| No trustworthy compatibility | Hide accessory lane or use verified rule candidates | Never substitute unrelated best sellers under an accessory promise |
| Excess duplicates or exhausted inventory | Bounded refill, smaller lane, or omit lane | Do not relax hard constraints for cosmetic fill |
Log the fallback policy, reason, fraction of final items affected and overall page outcome. Include degraded requests in experiment and SLO analysis. A model's successful-request-only relevance is not the delivered system relevance. Set precedence between per-item evidence fallback and whole-request operational fallback so that an outage cannot accidentally change semantic promises.
V.8 AWS deployment path for a three-person team¶
Use the current Python/FastAPI investment. My preferred initial shape is a small number of stateless serving containers, managed data services, separately scheduled training, immutable versioned artifacts, and one coherent observability/experiment pipeline. The choices below are a shortlist, not a mandate to deploy every listed service.
| Capability | Practical AWS path | Decision boundary and evidence |
|---|---|---|
| API and compact CPU ranking | ECS-managed service behind the existing application ingress; co-locate light ranking/page logic initially | ECS supports target tracking and scheduled scaling; use measured request capacity and scheduled peak preparation A18 |
| Hot session features and popular candidate caches | ElastiCache; durable state may remain in the existing upstream system or a keyed store | Choose measured cache semantics and operational fit, not a claim of unlimited automatic scale A19 |
| Durable keyed online state | DynamoDB when its access patterns fit | On-demand capacity has previous-peak and ramp behavior; provision/warm for the forecast rather than treating it as an instantaneous infinite reserve A20 |
| Feature history and current features | Existing warehouse plus managed online store; SageMaker Feature Store if its lifecycle saves enough work | Feature Store offers latest online records and historical offline records; point-in-time and arrival-time joins still need correct contracts A21 |
| Vector retrieval | Benchmark an in-process Faiss index against managed OpenSearch vector search | In-process gives fewer hops but makes index replication/lifecycle your responsibility; managed search is attractive for filter/index operations A05, A06 |
| Larger neural inference | SageMaker real-time endpoint if isolation and batching/scaling justify it | Managed real-time endpoints support autoscaling; no latency guarantee follows without load measurement A22 |
| Managed model baseline | Evaluate Amazon Personalize on a client-isolated dataset as a build-versus-buy reference | Official recipes cover user personalization and personalized ranking; compare objective control, new-item behavior, filtering, cost and page integration A23 |
Training can run in isolated containers against upstream-approved training views, producing model/index artifacts into scoped object storage. A heavy feature platform or distributed GPU recommender framework is not a prerequisite for a useful local two-tower model. TorchRec provides large sharded embedding and parallelism primitives if model size actually requires them; TFRS supports retrieval/multi-task workflows if TensorFlow is already the team's strength. Neither library supplies the entire commerce policy, isolation or experiment stack. A24, A25.
Avoid making fresh, remote general-purpose LLM generation a required homepage request dependency. If a generative challenger eventually wins, choose caching, distillation and dedicated inference based on measured latency and reward quality. A managed model service can reduce infrastructure work while still leaving scientific validation, logging and business rules with the team.
V.9 The 500 ms p99 budget and Black Friday¶
Illustrative deadline allocation, not measured performance: reserve 35 ms for request/context setup, 45 ms for online feature access, 85 ms for the parallel retrieval stage, 70 ms for batched ranking, 30 ms for page assembly, 30 ms for final validation/refill, 25 ms for serialization and service overhead, and 180 ms of headroom for queuing, network variation and tail events. The total is 500 ms. This is a starting allocation to test, not a prediction or an accepted set of component SLOs. The p99 of a sum is not the sum of component p99s. Define the measurement boundary to include all recommendation API dependencies and report timeouts/errors separately rather than censoring them.
Batch candidate scoring per page. Fetch features in bulk. Cache long-lived item embeddings and relation candidates; cache user-derived state only with property, identity/consent and feature-version semantics. Do not share personalized response caches between shoppers. Use request deadlines and bounded queues so that overload triggers a timely, validated fallback instead of a backlog of expired work. Benchmark cache misses, cache rebuilds, model swaps, index changes and inventory shocks.
“Astronomical traffic” is a stress requirement, not a number from which capacity can be computed. Obtain gateway historical peaks and expected campaign amplification. Let $\lambda$ be peak API requests/s, $m$ retrieved candidates/request, $k$ ranked candidates/request, and $f$ feature reads/request after batching/caching. Then retrieval work scales with the chosen search procedure times $\lambda$, ranking volume with $\lambda k$, and uncached feature demand with approximately $\lambda f$. Mean in-flight work is approximately $\lambda\mathbb E[T]$ under steady-state conditions; using p99 in that equation does not turn it into a precise tail-concurrency estimate.
Load-test a scenario matrix: ordinary mixed-client traffic; the largest client's concentrated burst; sustained expected peak; sudden burst before scaling; node/AZ or dependency failure; cold cache; highly selective inventory filters; and synchronized model/index refresh. Record p50/p95/p99, throughput, timeouts, queue depth, fallback rate and valid-page fill together. Under load, preserving only latency by sending empty pages is not success.
Pre-scale before known events, confirm service quotas, warm required models/indexes and maintain spare capacity. AWS documents scheduled ECS scaling and finite DynamoDB burst/ramp behavior; those are reasons to prepare rather than rely solely on reactive scaling. This review intentionally supplies no instance count or dollar forecast without QPS, user state cardinality, payload, region and measured per-replica capacity. A18, A20.
V.10 Publication, reproducibility and monitoring¶
Version a compatible bundle: feature definitions and schemas; source/time/label manifests; model weights and preprocessing; embeddings; ANN index; semantic ID mapping if used; calibration; lane policy; eligibility policy; and fallback snapshot. A query encoder must not silently query an incompatible item index. Atomic bundle activation and rollback are natural extensions of the repository's atomic snapshot head.
Separate semantic and operational monitors. Semantic examples include eligible target recall, score calibration, product-family repetition, exposure concentration, new-item coverage, lane promise violations and economics. Operational examples include feature/stock lag, model/index version mismatches, cache hit rate, timeout/fallback distributions, per-client error rates and logging loss. Monitor source missingness and event volume alongside model metrics: unchanged model code can behave differently after an upstream schema or population change.
Sculley et al. identify data dependencies, hidden feedback loops and configuration coupling as major ML-system maintenance risks. Breck et al. formalize production-readiness tests spanning data, models, infrastructure and monitoring. Their contribution is the engineering discipline, not a ready-made release threshold for your recommender. A26, A27.
Part VI — Assessment of the repository and research-to-production roadmap¶
VI.1 What should be retained¶
The repository is a credible operational foundation for aggregate recommendations. It implements property/catalog scoping, bounded relational streaming, co-view/co-purchase reducers, metadata similarity, time-decayed popularity, a strategy registry, eligibility/deduplication, provenance-bearing fallback chains, durable asynchronous runs, leases, immutable snapshots and atomic publication. These are source-code observations, not claims that the system has met its enterprise traffic targets. README, as-built reference, domain, worker, storage.
Keep the six current strategies as reproducible controls and degraded-mode assets. Extend the candidate-provider abstraction and publication lifecycle rather than discarding them because their models are simple. The new research question is which learned sources and online policy layers add value on top of those foundations.
VI.2 Gaps that matter scientifically¶
| Current code observation | Consequence | Proposed next step |
|---|---|---|
ViewRow has session/time/product; PurchaseRow has channel-scoped order/time/quantity; no persistent shopper, impression/position, return or margin contract |
Rich upstream data confirmed by you cannot yet drive the service's personalized training tasks | Add explicit upstream-to-training contracts, retaining current adapters for existing strategies source.py |
| Final ranking walks an ordered evidence chain and sorts within each tier | It is deterministic evidence prioritization, not a learned cross-source contextual ranker | Preserve the policy as baseline/fallback; add union features and supervised scoring fallbacks.py |
| Serving retrieves one pre-ranked strategy set and slices entries | No request-time user scoring, whole-page lane policy or live inventory revalidation is shown in this path | Introduce a separate personalized page decision interface with a shared eligibility and logging contract api.py |
| Co-occurrence support/shrinkage are selected using the same 28-day holdout later represented in quality metrics | Those metrics are validation evidence, not untouched final-test evidence | Add a separate test interval and rolling evaluations before drawing model-superiority conclusions run.py |
| One catalog snapshot supplies eligibility and content candidates to both evaluation and publication | Earlier decision-time catalog state is not reconstructed; full temporal validity depends on upstream snapshot semantics | Use historical effective/arrival-time item state or label the resulting backtest approximate run.py |
| Similar Items is evaluated against held-out co-view truth | Co-view prediction is a useful proxy, but it does not directly measure substitutability or technical compatibility | Add relation judgments and intent-specific downstream outcomes evaluation targets |
| Global strategies use one global relevant-item set in evaluation | Item coverage can dominate interpretation; a large count of exposed requests is not represented as independent evaluation units | Add request/customer/time-window evaluation and economic metrics run.py |
| Snapshot eligibility and staleness metadata are batch concepts | A still-served old snapshot can contain items whose live stock or price changed | Preserve last-good serving while independently validating dynamic eligibility api.py |
| Current internal API intentionally leaves authentication/authorization outside the service | Client-specific model/feature access must be bound to trusted scope at deployment | Verify the existing gateway boundary before extending personalized access as-built security section |
These observations locate research dependencies. They are not an independent bug audit, a claim that the current approved V1 specification demanded personalization, or a change to its contracts. The code already takes meaningful precautions against interaction leakage with whole-group temporal partitioning and train-only behavioral providers. The remaining distinction is between those precautions and a fully untouched, point-in-time personalized-policy evaluation.
VI.3 Recommended experimental sequence¶
| Stage | Concrete experiment/deliverable | Evidence required to advance |
|---|---|---|
| 0 — Measurement and data semantics | Expose the confirmed upstream signals; define margin/session, conversion and retention; freeze temporal manifests; reconcile stock/returns/identity | Correct decision-to-exposure-to-outcome joins; independent test set; client-isolation checks; explicit missing-data coverage |
| 1 — Strong personalized baseline | Add local ALS/BPR or pooled-history two-tower retrieval with semantic item features; combine with current sources; compact tree/logistic purchase/value ranker; deterministic homepage templates | End-to-end eligible recall, calibrated outcomes, cohort results, measured sub-500 ms p99 under an agreed load profile |
| 2 — Sequence and multi-behavior value | Compare tuned SASRec-style history encoding, recency pooling, channel-specific signals, and new-item semantic features | Ablations show incremental value from sequence order, store history and content rather than unequal tuning |
| 3 — Ranking and page composition | Compare tree baseline with compact DCN-V2/DIN-style ranker as justified; test list-aware reranking and cross-lane selection | Randomized improvement on the chosen economic/retention decision criterion without unacceptable conversion or reliability regressions |
| 4 — Controlled exploration | Randomized eligible candidates or lane strategies with correct propensities; variance-aware analysis | Useful support and tolerable cost; valid policy-value analysis in the supported action space |
| 5 — Frontier challengers | Distilled content/sequence models, semantic-ID retrieval, HSTU-scale-appropriate variants, cached generative themes or page models | Outperform a strong hybrid baseline under equal data access and real serving/operating budgets; retain interpretable rollback |
No calendar duration is promised. Data contracts, label maturation, experiment power and available engineering time determine the schedule. Avoid running every neural family simultaneously. For three people, one retrieval challenger and one ranking/page challenger at a time are usually enough to sustain a coherent experiment loop; that is an operating recommendation, not a statistical theorem.
A practical division of responsibility is data/measurement ownership, modeling/evaluation ownership, and serving/reliability ownership, with shared review of release evidence. Do not equate three roles to three isolated specialists: the key interfaces—feature time semantics, score meaning, eligibility and attribution—need joint decisions.
VI.4 Public benchmarks and their limits¶
Use public data to validate implementations and compare modeling mechanics, then make release decisions with the clients' actual data and experiments. Amazon review-derived benchmarks are valuable for text and sequence research but are not impression logs and may have very different density after filtering. Hou et al.'s BLaIR work releases Amazon Reviews 2023 and studies text/item representations; it is useful for semantic pretraining research, subject to terms and private-data isolation. A28.
The TREC 2025 Product Search and Recommendation Track overview, released in August 2026, describes a related-product recommendation task with annotated complementary and related-product relationships. That is unusually relevant to evaluating relation intent beyond co-view proxies. It does not evaluate your personalized homepage economics. A29.
Keep a separate industrial benchmark manifest covering catalog churn, many variants, sparse products, offline purchases, delayed returns, promotional exposure and peak-load behavior. Synthetic data already in this repository can verify grouping, isolation, deterministic fallbacks and failure semantics. It cannot certify real shopper relevance or expected uplift. A synthetic mission that generates co-purchase pairs may favor models that reconstruct the generator's own assumptions.
VI.5 Decision recommendation¶
For the confirmed setting, prioritize client-local hybrid retrieval, a compact purchase/value ranker, and an explicit homepage composition policy. Use long-term and session histories together; represent online/store behavior distinctly; use pretrained content representations for new products; retain the aggregate snapshot stack as a baseline and fallback. Build an independent temporal test and a business experiment pipeline alongside those models. This is the most defensible first architecture to test given the team, the available signals, and the latency requirement.
The advanced research portfolio should be focused, not dismissed. Sequential transduction, semantic IDs, multi-behavior compression, reward-aligned generation and generative page planning are meaningful 2024–2026 directions. The review's evidence supports testing them against this hybrid stack, while tracking data access, compute, label definitions, valid-item yield, page quality and online outcomes. It does not identify one universal winner or a guaranteed percentage gain for this enterprise.
VI.6 Suggested reading order¶
- Statistical foundations: M01 (implicit MF), M02 (BPR), M04 (EASE), E1 (selection bias). Understand the assumptions behind positives and missing data.
- Practical stack: A01 (two-stage system), A02 (retrieval sampling), A03 (e-commerce intent-specific two towers), A10 (DCN-V2). Connect objectives to computation.
- Sequence and modality: M17–M19 (SASRec/BERT4Rec and loss comparisons), M24 (multi-behavior graph), M28–M30 (transfer and multimodal representation). Separate new information from model capacity.
- Evaluation: E3–E7 (timeline, metric sampling, propensities and DR), E13–E14 (delays/value), E26 (CUPED). Establish what can actually be concluded.
- Page and long-term objectives: E18–E25 (reranking, calibration, whole-page generation), E16 (SlateQ). Examine choice and reward assumptions.
- Frontier: M33–M40 (TIGER, HSTU, OneRec family and OneMall), A17 (hybrid generative storefronts). Read experiment tables and deployment details alongside architectural diagrams.
Complete references¶
References provide authors, full titles, year, venue/status where verified, and direct original-source links. Long author lists may use “et al.”; the linked primary record contains the complete list. ArXiv identifiers are stable reference identifiers, and cited version suffixes preserve the inspected version. All web sources were accessed for this review on 5 September 2026.
Modeling references and inspection record¶
Inspection labels: F = original full-text methods and/or evidence sections inspected; A = original abstract and publication metadata inspected; R = author-maintained repository inspected. Abstract-screened papers are used only for broad method/evidence descriptions; detailed numeric claims above come from inspected text or explicitly identified abstracts. No experiments from these papers were reproduced during this review. All linked versions are at or before the cutoff.
- M01 — Hu, Y., Koren, Y., and Volinsky, C. (2008). Collaborative Filtering for Implicit Feedback Datasets. ICDM. F. Author-hosted original PDF.
- M02 — Rendle, S., Freudenthaler, C., Gantner, Z., and Schmidt-Thieme, L. (2009). BPR: Bayesian Personalized Ranking from Implicit Feedback. UAI; arXiv uploaded 2012. F. Paper.
- M03 — Ning, X., and Karypis, G. (2011). SLIM: Sparse Linear Methods for Top-N Recommender Systems. ICDM, pp. 497–506. R. Original DOI; authors' implementation and description.
- M04 — Steck, H. (2019). Embarrassingly Shallow Autoencoders for Sparse Data. WWW. F. Full text.
- M05 — Barkan, O., and Koenigstein, N. (2016). Item2Vec: Neural Item Embedding for Collaborative Filtering. A. Paper.
- M06 — Kula, M. (2015). Metadata Embeddings for User and Item Cold-start Recommendations. A. Paper.
- M07 — McAuley, J., Pandey, R., and Leskovec, J. (2015). Inferring Networks of Substitutable and Complementary Products. KDD. F. Author-hosted paper.
- M08 — He, X., Liao, L., Zhang, H., Nie, L., Hu, X., and Chua, T.-S. (2017). Neural Collaborative Filtering. WWW. A. Paper.
- M09 — Rendle, S., Krichene, W., Zhang, L., and Anderson, J. (2020). Neural Collaborative Filtering vs. Matrix Factorization Revisited. A. Paper.
- M10 — Liang, D., Krishnan, R. G., Hoffman, M. D., and Jebara, T. (2018). Variational Autoencoders for Collaborative Filtering. WWW. F. Full text.
- M11 — Mao, K., et al. (2021). SimpleX: A Simple and Strong Baseline for Collaborative Filtering. CIKM. A. Paper, v3 updated 2023.
- M12 — He, X., Deng, K., Wang, X., Li, Y., Zhang, Y., and Wang, M. (2020). LightGCN: Simplifying and Powering Graph Convolution Network for Recommendation. SIGIR. F. Full text.
- M13 — Ying, R., He, R., Chen, K., Eksombatchai, P., Hamilton, W. L., and Leskovec, J. (2018). Graph Convolutional Neural Networks for Web-Scale Recommender Systems. KDD. A. PinSage paper.
- M14 — Wu, J., et al. (2021). Self-supervised Graph Learning for Recommendation. SIGIR; first preprint 2020. A. SGL paper.
- M15 — Yu, J., Yin, H., Xia, X., Chen, T., Cui, L., and Nguyen, Q. V. H. (2022). Are Graph Augmentations Necessary? Simple Graph Contrastive Learning for Recommendation. SIGIR; first preprint 2021. A. SimGCL paper.
- M16 — Hidasi, B., Karatzoglou, A., Baltrunas, L., and Tikk, D. (2016). Session-based Recommendations with Recurrent Neural Networks. ICLR; first preprint 2015. A. GRU4Rec paper.
- M17 — Kang, W.-C., and McAuley, J. (2018). Self-Attentive Sequential Recommendation. ICDM. F. SASRec paper.
- M18 — Sun, F., et al. (2019). BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer. CIKM. F. Paper.
- M19 — Petrov, A. V., and Macdonald, C. (2023). Turning Dross Into Gold Loss: is BERT4Rec really better than SASRec? RecSys. F. Paper.
- M20 — Li, C., et al. (2019). Multi-Interest Network with Dynamic Routing for Recommendation at Tmall. CIKM. A. MIND paper.
- M21 — Cen, Y., Zhang, J., Zou, X., Zhou, C., Yang, H., and Tang, J. (2020). Controllable Multi-Interest Framework for Recommendation. KDD. A. ComiRec paper.
- M22 — Ren, P., Chen, Z., Li, J., Ren, Z., Ma, J., and de Rijke, M. (2019). RepeatNet: A Repeat Aware Neural Recommendation Machine for Session-Based Recommendation. AAAI 33, pp. 4806–4813. F. Original proceedings.
- M23 — Hu, H., He, X., Gao, J., and Zhang, Z.-L. (2020). Modeling Personalized Item Frequency Information for Next-basket Recommendation. SIGIR. A. TIFU-KNN paper.
- M24 — Jin, B., Gao, C., He, X., Jin, D., and Li, Y. (2020). Multi-behavior Recommendation with Graph Convolutional Networks. SIGIR. F. Author-hosted MBGCN paper.
- M25 — Xia, L., Xu, Y., Huang, C., Dai, P., and Bo, L. (2021). Graph Meta Network for Multi-Behavior Recommendation. A. MB-GMN paper.
- M26 — Cheng, Z., Han, S., Liu, F., Zhu, L., Gao, Z., and Peng, Y. (2023). Multi-Behavior Recommendation with Cascading Graph Convolution Networks. A. MB-CGCN paper.
- M27 — He, R., and McAuley, J. (2016). VBPR: Visual Bayesian Personalized Ranking from Implicit Feedback. AAAI; first preprint 2015. A. Paper.
- M28 — Hou, Y., Mu, S., Zhao, W. X., Li, Y., Ding, B., and Wen, J.-R. (2022). Towards Universal Sequence Representation Learning for Recommender Systems. KDD. A. UniSRec paper.
- M29 — Chen, J., Chi, L., Peng, B., and Yuan, Z. (2024). HLLM: Enhancing Sequential Recommendations via Hierarchical Large Language Models for Item and User Modeling. arXiv preprint. A. Paper.
- M30 — Lei, Z., Feng, T., Hua, Z., Xie, Y., Lin, G., Yang, S., Liu, G., and You, J. (2026). UniRec: Unified Multimodal Encoding for LLM-Based Recommendations. TMLR. A. Paper, v3.
- M31 — Wang, W., Xu, Y., Feng, F., Lin, X., He, X., and Chua, T.-S. (2023). Diffusion Recommender Model. SIGIR; inspected revision 2025. A. DiffRec paper.
- M32 — Geng, S., Liu, S., Fu, Z., Ge, Y., and Zhang, Y. (2022). Recommendation as Language Processing (RLP): A Unified Pretrain, Personalized Prompt & Predict Paradigm (P5). RecSys. A. Paper.
- M33 — Rajput, S., et al. (2023). Recommender Systems with Generative Retrieval. NeurIPS. F. TIGER full text.
- M34 — Zhai, J., et al. (2024). Actions Speak Louder than Words: Trillion-Parameter Sequential Transducers for Generative Recommendations. ICML. F. HSTU full text.
- M35 — Dong, Y., Li, H., Li, S., Patel, N., Liu, X., Wang, X., and Zhuge, C. (2025). Scaling Generative Recommendations with Context Parallelism on Hierarchical Sequential Transducers. arXiv preprint. A. Paper.
- M36 — Deng, J., Wang, S., Cai, K., Ren, L., Hu, Q., Ding, W., Luo, Q., and Zhou, G. (2025). OneRec: Unifying Retrieve and Rank with Generative Recommender and Iterative Preference Alignment. arXiv preprint. F. Paper.
- M37 — Zhou, Guorui, et al. (2025). OneRec-V2 Technical Report. arXiv technical report, v4 dated 2025-10-28. F. Full text and author list.
- M38 — Liu, Z., et al. (2025). OneRec-Think: In-Text Reasoning for Generative Recommendation. arXiv preprint. A. Paper.
- M39 — Liang, Z., et al. (2025). TBGRecall: A Generative Retrieval Model for E-commerce Recommendation Scenarios. arXiv preprint, v2 dated 2025-11-23. A. Paper.
- M40 — Zhang, K., et al. (2026). OneMall: One Model, More Scenarios — End-to-End Generative Recommender Family at Kuaishou E-Commerce. arXiv preprint, 2026-01-29. F. Full text.
- M41 — Zhong, Y., Ruan, C., Hasani, M., Tenneti, T., Wang, H., and Ma, F. (2026). GrocLM: Grocery Category Recommendation in E-Commerce with Large Language Models. arXiv preprint. F. Full text.
- M42 — Chen, Z., Chang, H., Liu, T., Zhou, C., Cao, Y., Ding, J., Liu, M., and Qin, B. (2026). Beyond the Flat Sequence: Hierarchical and Preference-Aware Generative Recommendations. A. HPGR paper.
- M43 — Dacrema, M. F., Cremonesi, P., and Jannach, D. (2019). Are We Really Making Much Progress? A Worrying Analysis of Recent Neural Recommendation Approaches. RecSys. A. Paper.
Evaluation, experimentation, and page optimization references¶
- E1 — Schnabel, Tobias; Swaminathan, Adith; Singh, Ashudeep; Chandak, Navin; and Joachims, Thorsten (2016). Recommendations as Treatments: Debiasing Learning and Evaluation. ICML. arXiv:1602.05352. Paper.
- E2 — Dmitriev, Pavel; Gupta, Somit; Kim, Dong Woo; and Vaz, Garnet (2017). A Dirty Dozen: Twelve Common Metric Interpretation Pitfalls in Online Controlled Experiments. KDD. Institutional publication record.
- E3 — Ji, Yitong; Sun, Aixin; Zhang, Jie; and Li, Chenliang (2023; first preprint 2020). A Critical Study on Data Leakage in Recommender System Offline Evaluation. ACM Transactions on Information Systems 41(3), article 75, pp. 1–27. DOI:10.1145/3569930. Paper.
- E4 — Krichene, Walid, and Rendle, Steffen (2020). On Sampled Metrics for Item Recommendation. KDD. Institutional publication record.
- E5 — Joachims, Thorsten; Swaminathan, Adith; and Schnabel, Tobias (2017; preprint 2016). Unbiased Learning-to-Rank with Biased Feedback. WSDM. arXiv:1608.04468. Paper.
- E6 — Swaminathan, Adith, and Joachims, Thorsten (2015). The Self-Normalized Estimator for Counterfactual Learning. Advances in Neural Information Processing Systems 28. Proceedings.
- E7 — Dudík, Miroslav; Langford, John; and Li, Lihong (2011). Doubly Robust Policy Evaluation and Learning. ICML. arXiv:1103.4601. Paper.
- E8 — Ai, Qingyao; Bi, Keping; Luo, Cheng; Guo, Jiafeng; and Croft, W. Bruce (2018). Unbiased Learning to Rank with Unbiased Propensity Estimation. SIGIR. arXiv:1804.05938. Paper.
- E9 — Swaminathan, Adith; Krishnamurthy, Akshay; Agarwal, Alekh; Dudík, Miroslav; Langford, John; Jose, Damien; and Zitouni, Imed (2017). Off-policy Evaluation for Slate Recommendation. Advances in Neural Information Processing Systems 30. Proceedings.
- E10 — Chaudhari, Shreyas; Arbour, David; Theocharous, Georgios; and Vlassis, Nikos (2024; preprint 2023). Distributional Off-Policy Evaluation for Slate Recommendations. AAAI. arXiv:2308.14165. Paper.
- E11 / A13 — Ma, Xiao; Zhao, Liqin; Huang, Guan; Wang, Zhi; Hu, Zelin; Zhu, Xiaoqiang; and Gai, Kun (2018). Entire Space Multi-Task Model: An Effective Approach for Estimating Post-Click Conversion Rate. SIGIR. arXiv:1804.07931. Paper. One study cited in both modeling and evaluation contexts.
- E12 — Wang, Hao; Chang, Tai-Wei; Liu, Tianqiao; Huang, Jianmin; Chen, Zhichao; Yu, Chao; Li, Ruopeng; and Chu, Wei (2022). ESCM²: Entire Space Counterfactual Multi-Task Model for Post-Click Conversion Rate Estimation. SIGIR. arXiv:2204.05125. Paper.
- E13 — Chapelle, Olivier (2014). Modeling Delayed Feedback in Display Advertising. KDD. DOI:10.1145/2623330.2623634. Publisher record; original full-text mirror consulted.
- E14 — Li, Xinyu; Chen, Sishuo; Xv, Guipeng; Zhang, Li; Luo, Mingxuan; Chan, Zhangming; Sheng, Xiang-Rong; Zhu, Han; Xu, Jian; and Lin, Chen (2026). Delayed Feedback Modeling for Post-Click Gross Merchandise Volume Prediction: Benchmark, Insights and Approaches. WWW 2026 camera-ready according to the authors' arXiv record. arXiv:2601.20307. Paper.
- E15 — Li, Lihong; Chu, Wei; Langford, John; and Schapire, Robert E. (2010). A Contextual-Bandit Approach to Personalized News Article Recommendation. WWW. arXiv:1003.0146. Paper.
- E16 — Ie, Eugene; Jain, Vihan; Wang, Jing; Narvekar, Sanmit; Agarwal, Ritesh; Wu, Rui; Cheng, Heng-Tze; Chandra, Tushar; and Boutilier, Craig (2019). SlateQ: A Tractable Decomposition for Reinforcement Learning with Recommendation Sets. IJCAI. Institutional publication record.
- E17 — Zhao, Xiangyu; Xia, Long; Zhang, Liang; Ding, Zhuoye; Yin, Dawei; and Tang, Jiliang (2018). Deep Reinforcement Learning for Page-wise Recommendations. arXiv:1805.02343, v2. Paper.
- E18 — Pei, Changhua; Zhang, Yi; Zhang, Yongfeng; Sun, Fei; Lin, Xiao; Sun, Hanxiao; Wu, Jian; Jiang, Peng; and Ou, Wenwu (2019). Personalized Re-ranking for Recommendation. RecSys. arXiv:1904.06813. Paper.
- E19 — Goldstein, Jade, and Carbonell, Jaime (1998). Summarization: (1) Using MMR for Diversity-Based Reranking and (2) Evaluating Summaries. TIPSTER Text Program Phase III Workshop, pp. 181–195. DOI:10.3115/1119089.1119120. Original proceedings. This is the specific MMR source consulted, not a mislabeled citation to the separate SIGIR paper.
- E20 — Chen, Laming; Zhang, Guoxin; and Zhou, Hanning (2018; preprint 2017). Fast Greedy MAP Inference for Determinantal Point Process to Improve Recommendation Diversity. Advances in Neural Information Processing Systems 31. arXiv:1709.05135. Paper.
- E21 — Steck, Harald (2018). Calibrated Recommendations. RecSys, pp. 154–162. DOI:10.1145/3240323.3240372. Publisher record. Original paper was consulted through an author-listed ResearchGate reproduction; publisher access was restricted during checking.
- E22 — Abdollahpouri, Himan; Burke, Robin; and Mobasher, Bamshad (2019). Managing Popularity Bias in Recommender Systems with Personalized Re-ranking. arXiv:1901.07555, v4. Paper.
- E23 — Alvino, Chris, and Basilico, Justin (2015). Learning a Personalized Homepage. Netflix Technology Blog, 9 April. First-party engineering article. Article.
- E24 — Wang, Xinyuan; Wu, Liang; Wang, Dongjie; and Fu, Yanjie (2025; revised 2026). PageLLM: A Multi-Grained Reward Framework for Whole-Page Optimization with Large Language Models. arXiv:2506.09084, v2, 23 May 2026. Preprint. Full text.
- E25 — Zou, Yanyan; Qi, Junbo; Huang, Lunsong; Li, Yu; Xu, Kewei; Gao, Jiabao; Zhao, Binglei; Yang, Xuanhua; Xu, Sulong; and Li, Shengjie (2026). GenRec: A Preference-Oriented Generative Framework for Large-Scale Recommendation. SIGIR 2026 camera-ready according to the authors' arXiv record. arXiv:2604.14878. Paper.
- E26 — Deng, Alex; Xu, Ya; Kohavi, Ron; and Walker, Toby (2013). Improving the Sensitivity of Online Controlled Experiments by Utilizing Pre-Experiment Data. WSDM, pp. 123–132. DOI:10.1145/2433396.2433413. Publisher record; author/platform-hosted original paper. The host intermittently returned a browser-verification page during final checking.
- E27 — Lin, Zhexiao, and Crespo, Pablo (2026). Variance Reduction Combining Pre-experiment and In-experiment Data. Fifth Conference on Causal Learning and Reasoning, PMLR 323, pp. 699–717. Proceedings.
- E28 — Fabijan, Aleksander; Bajpai, Vaibhav Kumar; Blanarik, Trevor; Caughron, Max; Chen, Kewei; Zhang, Ruhan; Gustafson, Adam; Budumuri, Venkata Kavitha; and Hunt, Stephen (2020). Diagnosing Sample Ratio Mismatch in A/B Testing. Microsoft Experimentation Platform. First-party article linking the KDD 2019 research paper. Article.
- E29 — Holtz, David; Lobel, Ruben; Liskovich, Inessa; and Aral, Sinan (2020). Reducing Interference Bias in Online Marketplace Pricing Experiments. arXiv:2004.12489. Paper.
Retrieval, ranking, architecture, and additional dataset references¶
- A01 — Covington, Paul; Adams, Jay; and Sargin, Emre (2016). Deep Neural Networks for YouTube Recommendations. RecSys. DOI:10.1145/2959100.2959190. Institutional record; original PDF.
- A02 — Yi, Xinyang; Yang, Ji; Hong, Lichan; Cheng, Derek Z.; Heldt, Lukasz; Kumthekar, Aditee Ajit; Zhao, Zhe; Wei, Li; and Chi, Ed H. (2019). Sampling-Bias-Corrected Neural Modeling for Large Corpus Item Recommendations. RecSys. Institutional record.
- A03 — Osowska-Kurczab, Aleksandra; Nazarko, Klaudia; Marzec, Mateusz; Wojciechowska, Lidia; and Kremeňová, Eliška (2025). Suggest, Complement, Inspire: Story of Two Tower Recommendations at Allegro.com. arXiv:2508.03702. Paper.
- A04 — Malkov, Yu. A., and Yashunin, D. A. (2016 preprint; journal publication 2020). Efficient and Robust Approximate Nearest Neighbor Search Using Hierarchical Navigable Small World Graphs. IEEE Transactions on Pattern Analysis and Machine Intelligence 42(4), pp. 824–836. DOI:10.1109/TPAMI.2018.2889473. Paper.
- A05 — Johnson, Jeff; Douze, Matthijs; and Jégou, Hervé (2017 preprint; journal publication 2021). Billion-scale Similarity Search with GPUs. IEEE Transactions on Big Data 7(3), pp. 535–547. arXiv:1702.08734. Paper. Associated software: Meta/Faiss contributors, Faiss Documentation, official wiki, accessed 5 September 2026.
- A06 — OpenSearch Project (living documentation). Filtering Vector Search Results and Efficient k-NN Filtering. Filtering overview, efficient filtering. Amazon Web Services: k-Nearest Neighbor (k-NN) Search in Amazon OpenSearch Service, AWS documentation. Accessed 5 September 2026; upstream and managed-service capabilities must be checked separately.
- A07 — Chen, Tianqi, and Guestrin, Carlos (2016). XGBoost: A Scalable Tree Boosting System. KDD. DOI:10.1145/2939672.2939785. Paper.
- A08 — Cheng, Heng-Tze; Koc, Levent; Harmsen, Jeremiah; Shaked, Tal; Chandra, Tushar; Aradhye, Hrishi; Anderson, Glen; Corrado, Greg; Chai, Wei; Ispir, Mustafa; Anil, Rohan; Haque, Zakaria; Hong, Lichan; Jain, Vihan; Liu, Xiaobing; and Shah, Hemal (2016). Wide & Deep Learning for Recommender Systems. First Workshop on Deep Learning for Recommender Systems. arXiv:1606.07792. Paper.
- A09 — Guo, Huifeng; Tang, Ruiming; Ye, Yunming; Li, Zhenguo; and He, Xiuqiang (2017). DeepFM: A Factorization-Machine Based Neural Network for CTR Prediction. IJCAI. arXiv:1703.04247. Paper.
- A10 — Wang, Ruoxi; Shivanna, Rakesh; Cheng, Derek Z.; Jain, Sagar; Lin, Dong; Hong, Lichan; and Chi, Ed H. (2021; preprint 2020). DCN V2: Improved Deep & Cross Network and Practical Lessons for Web-scale Learning to Rank Systems. WWW. DOI:10.1145/3442381.3450078. Paper.
- A11 — Zhou, Guorui; Song, Chengru; Zhu, Xiaoqiang; Fan, Ying; Zhu, Han; Ma, Xiao; Yan, Yanghui; Jin, Junqi; Li, Han; and Gai, Kun (2018; preprint 2017). Deep Interest Network for Click-Through Rate Prediction. KDD. arXiv:1706.06978. Authors' preprint. Author order follows the linked arXiv record.
- A12 — Ma, Jiaqi; Zhao, Zhe; Yi, Xinyang; Chen, Jilin; Hong, Lichan; and Chi, Ed H. (2018). Modeling Task Relationships in Multi-task Learning with Multi-gate Mixture-of-Experts. KDD. Institutional record.
- A13 — ESMM: see the complete E11 / A13 entry above.
- A14 — Zhao, Zhe, et al. (2019). Recommending What Video to Watch Next: A Multitask Ranking System. RecSys. Institutional record and complete author list.
- A15 — Sun, Chonglin; Yu, Nancy; Lu, Haiyu; Wang, Liang; Pu, Yunchen; Liu, Gaoxiang; Bhatia, Neeraj; and Musumeci, GP (2024). Meta Andromeda: Supercharging Advantage+ Automation with the Next-gen Personalized Ads Retrieval Engine. Engineering at Meta, 2 December. First-party engineering report. Article.
- A16 — Liu, Zhuoran; Zou, Leqi; Zou, Xuan; Wang, Caihua; Zhang, Biao; Tang, Da; Zhu, Bolin; Zhu, Yijie; Wu, Peng; Wang, Ke; and Cheng, Youlong (2022). Monolith: Real Time Recommendation System With Collisionless Embedding Table. ORSUM at RecSys. arXiv:2209.07663. Paper.
- A17 — Hasani, Moein; Shahidi, Hamidreza; Levinson, Trace; Zhong, Yuan; Shu, Guanghua; Gudla, Vinesh; and Tenneti, Tejaswi (2026). A Cascaded Generative Approach for e-Commerce Recommendations. arXiv:2605.11118, v1, 11 May. Paper. Experimental details checked in the full text.
- A18 — Amazon Web Services (living documentation). Automatically Scale Your Amazon ECS Service. Documentation. Accessed 5 September 2026.
- A19 — Amazon Web Services (living documentation). Scaling ElastiCache Serverless Clusters. Documentation. Accessed 5 September 2026.
- A20 — Amazon Web Services (living documentation). DynamoDB On-demand Capacity Mode. Documentation. Accessed 5 September 2026.
- A21 — Amazon Web Services (living documentation). Create, Store, and Share Features with Feature Store. Amazon SageMaker AI Developer Guide. Documentation. Accessed 5 September 2026.
- A22 — Amazon Web Services (living documentation). Real-time Inference. Amazon SageMaker AI Developer Guide. Documentation. Accessed 5 September 2026.
- A23 — Amazon Web Services (living documentation). Choosing a Recipe. Amazon Personalize Developer Guide. Documentation. Accessed 5 September 2026.
- A24 — Meta/PyTorch TorchRec contributors (living software documentation). TorchRec: PyTorch Domain Library for Recommendation Systems. Official repository. Accessed 5 September 2026.
- A25 — TensorFlow contributors (living software documentation). TensorFlow Recommenders. Official documentation. Accessed 5 September 2026.
- A26 — Sculley, D.; Holt, Gary; Golovin, Daniel; Davydov, Eugene; Phillips, Todd; Ebner, Dietmar; Chaudhary, Vinay; Young, Michael; Crespo, Jean-François; and Dennison, Dan (2015). Hidden Technical Debt in Machine Learning Systems. Advances in Neural Information Processing Systems 28. Proceedings.
- A27 — Breck, Eric; Cai, Shanqing; Nielsen, Eric; Salib, Michael; and Sculley, D. (2017). The ML Test Score: A Rubric for ML Production Readiness and Technical Debt Reduction. IEEE Big Data. Institutional record.
- A28 — Hou, Yupeng; Li, Jiacheng; He, Zhankui; Yan, An; Chen, Xiusi; and McAuley, Julian (2024). Bridging Language and Items for Retrieval and Recommendation. arXiv:2403.03952. Paper.
- A29 — Alvarez, Dean E.; Kallumadi, Surya; Campos, Daniel; Zhai, ChengXiang; Magnani, Alessandro; Takehi, Rikiya; and Ekstrand, Michael D. (2026). Overview of the TREC 2025 Product Search and Recommendation Track. arXiv:2608.17138, 17 August. Track overview.
Repository primary sources¶
All local sources below were inspected at commit 63a3af00322f5cc38b9497e84b8c06350f09ea92; no production data was examined.
- Commerce Recommendations README; domain terminology; product vision.
- Commerce Recommendation Service approved specification; Technical Implementation as-built reference.
- Canonical source contracts; strategy definitions and domain; pipeline orchestration; derived work store; fallback ranking; evaluation.
- API; worker; storage and publication.