Commerce datasets for embedded ANN qualification¶
Status: research, 2026-09-25. This is a dataset-selection recommendation, not a license opinion, an approved evaluation protocol, or a production qualification result. No third-party dataset was downloaded or committed for this review.
Recommendation¶
Use Amazon Shopping Queries (ESCI) as the primary public product-metadata and hard-negative corpus for Similar Items experiments, and UCI Online Retail II as a small, permissively licensed chronological purchase-history corpus for For You experiments. Keep the committed ENERGY STAR models as a deterministic, commerce-shaped metadata smoke fixture. Do not substitute a movie dataset: it would omit the product attributes and shopping behavior that these two strategies must handle. None of these corpora alone can authorize turning on ANN by default. Amazon ESCI repository, UCI Online Retail II, EPA ENERGY STAR Model Index, current qualification distinction.
| Corpus | Useful evidence | Boundary / access |
|---|---|---|
Existing energy-star-models.csv.gz |
A real EPA-derived model catalog: local file is about 782 KiB and has model ID, brand, category, model name/number, UPC and availability/certification dates. Good for deterministic metadata ingestion, missing fields and cold-item smoke. EPA Model Index. | It has no browsing, order, shopper or relevance judgments. Certification is not merchant Recommendation Eligibility, and similar model names do not prove substitutability. It is not a For You or quality benchmark. EPA Model Index. |
| Amazon Shopping Queries (ESCI) | Real product titles, descriptions, bullet points, brands and colors joined to query-product Exact/Substitute/Complement/Irrelevant judgments. The large split has 130,652 queries and 2,621,288 judgments across US, ES and JP locales. Useful for product-text representation, cold items, disqualifying complements and a carefully defined proxy for substitute retrieval. Official README. | Judgments are query-to-product, not item-to-item: two products judged substitutes for a query are not automatically substitutes for each other. No shopper histories or orders. The official product Parquet is 1.03 GB and examples Parquet 48.9 MB in Git LFS; do not accidentally clone/download it in CI. The repository states Apache-2.0; preserve attribution and review its license before redistributing derived product records. Official README, product file, examples file, license. |
| UCI Online Retail II | About 1.07 million real transaction lines over two years, including InvoiceNo, StockCode, product Description, Quantity, InvoiceDate, UnitPrice and CustomerID; 43.5 MB download. Supports within-store, time-correct held-out purchase retrieval and purchase-only For You baselines. UCI explicitly lists CC BY 4.0 with attribution. UCI dataset card. | No item views, long descriptions, explicit category/brand, merchandising eligibility or consent states; modest Catalog size cannot qualify large-index memory/latency. CustomerID and InvoiceNo remain sensitive source identifiers even if numeric: stream/reduce, never log, fixture, publish or commit them; generate only aggregate metrics. Cancellation invoices need explicit exclusion policy. UCI dataset card, repository privacy contract. |
| OTTO RecSys sessions | An optional second, much larger behavioral stress set: official release states 12.9 million training sessions, 216.7 million training events, roughly 1.86 million items, clicks/carts/orders, temporal train/test split and CC BY 4.0. Useful for session-only next-item quality and scale. OTTO repository. | Item IDs are anonymized and the release documents no product title/description/category mapping. It cannot evaluate this service's metadata Similar Items representation, and sessions are not durable authorized Shopper Profiles. Download is via Kaggle. OTTO repository. |
| Coveo SIGIR eCom 2021 | Real browsing/purchase sessions and catalog category/price plus already-computed text/image vectors; potentially closer to both workflows. Coveo official README. | Dataset use is research/education only, requires a form and acceptance of terms, and prohibits de-anonymization. It supplies vectors rather than raw product text, so it is not a clean test of this repository's metadata representation. Do not use in this project without a separate terms/usage decision. Coveo official README. |
| Amazon Reviews 2023 | Rich product metadata, user-item review timelines and bought-together links at very large scale. McAuley Lab dataset page. | The dataset publisher says it is not in a position to assign a license and makes it available primarily for research. Reviews are not purchases, and raw reviewer IDs/text create additional privacy and rights obligations. Do not make this the default qualification source without an explicit legal/usage review. Publisher license response, dataset page. |
Evaluation protocol proposed for implementation¶
- Similar Items quality: keep ESCI US locale separate from other locales and use its published train/test query split. Build Item vectors from product fields only, never from ESCI labels or test queries. Treat same-query E/S versus C/I as a candidate-pair heuristic to select hard examples, not as item-pair ground truth. Have a merchandiser or blinded expert judge anchor/candidate pairs for substitution, compatibility, category, price and disqualifiers; compare dense full-catalog ranking with the current sparse baseline using Precision@10 and slices. Record inter-rater disagreements and do not infer an unjudged pair is irrelevant. ESCI schema and labels, current relevance gate.
- For You quality: from UCI Online Retail II, stream one Commerce Property/Catalog mapping, exclude cancellations/returns by an explicit rule, order each customer's invoices by event time and use a global cutoff plus later held-out purchases. Do not train on later item metadata/interactions or evaluate against sampled negatives; rank the entire eligible Catalog, compare current published-candidate For You and ANN discovery on NDCG@10, and report profile-depth/cold-item slices. This measures purchase prediction, not authorization or view-driven behavior. Keep CustomerID and InvoiceNo only in an isolated, ephemeral evaluation process; emitted evidence must be aggregate and carry attribution, not source rows. UCI fields and cancellation notation, current personalization contract, current relevance gate.
- Index fidelity and operations: after selecting one representation, benchmark exact Flat and approximate HNSW on the same eligible vectors and queries, then run the unmodified repository qualification-scale profile and disposable PostgreSQL lifecycle. The small UCI Catalog and ESCI's search-judged pool do not represent production traffic or a merchant's eligibility distribution; neither can by itself establish the recall, latency, RSS or atomic-publication gates. Current ANN evidence, OTTO scale and protocol.
The first useful next step is a bounded, opt-in evaluation harness that streams the two selected corpora from untracked local paths and emits only versioned, aggregate metrics and dataset provenance. Do not commit raw third-party product records, CustomerIDs, InvoiceNos, sessions or reviews. Keep ANN off until the qualitative and operational gates pass on representative, scoped workloads. Repository context, current ANN evidence.