Skip to content

Qualify embedded ANN on Apple Silicon and supported deployments

Status: Specification draft from user-confirmed scope; unresolved design and qualification decisions remain explicit. Not implemented.

Problem Statement

Native ARM package availability alone does not demonstrate that ANN is optimized for Apple Silicon or meets commerce latency, memory and relevance needs. Library marketing and unrelated benchmarks cannot establish fitness for this application's filtered, multi-scope workload.

Solution

Qualify a maintained embedded library with repeatable exact-reference, representation-quality, resource and deployment evidence. Evaluate Faiss first because of the user's preference and documented ARM/Accelerate CPU support, retaining a credible comparison when useful.

User Stories

  1. As a developer, I want Meta Faiss evaluated first, so that the preferred maintained implementation receives concrete scrutiny.
  2. As a operator, I want native Apple Silicon execution verified, so that Rosetta results cannot be mistaken for ARM performance.
  3. As a operator, I want actual numeric and threading backends recorded, so that optimization claims are reproducible.
  4. As a developer, I want CPU acceleration distinguished from GPU and ANE support, so that unsupported Apple hardware claims cannot influence design.
  5. As a maintainer, I want supported Python and NumPy combinations verified, so that the dependency fits repository packaging.
  6. As a operator, I want Linux deployment compatibility checked too, so that local optimization cannot break deployment.
  7. As a tester, I want an exact same-vector search reference, so that ANN recall has a valid oracle.
  8. As a tester, I want metadata representation quality evaluated separately, so that ANN tuning is not blamed for vector-model regression.
  9. As a operator, I want tail latency measured under concurrent requests, so that single-query speed cannot hide overload.
  10. As a operator, I want memory measured across scopes, processes and snapshot overlap, so that an index fitting once is not mistaken for production fit.
  11. As a operator, I want build, serialization and cold-load times measured, so that index maintenance stays within operational budgets.
  12. As a consumer, I want filter selectivity and quota fill evaluated, so that high unfiltered recall cannot conceal unusable recommendations.
  13. As a developer, I want a reproducible evidence manifest with versions and hardware, so that results can be compared honestly.
  14. As a maintainer, I want licenses and package provenance reviewed, so that native dependency adoption is auditable.
  15. As a product owner, I want explicit acceptance thresholds before selection, so that a library is not accepted based on subjective speed.
  16. As a operator, I want a documented rollback and staged enablement gate, so that unqualified ANN is not enabled broadly.

Implementation Decisions

  • Apple Silicon optimization is an explicit preference; Faiss is the lead candidate, not yet an approved pinned dependency or index configuration.
  • Primary-source research reports Faiss native ARM NEON and Apple Accelerate CPU build support. The locally installed Faiss 1.15.1 macOS ARM64 wheel reports MAC_METAL, exposes one device and searched a small Flat index through the Metal bridge. A native eligibility selector was not honored in that check, and the one-off unfiltered Flat comparison did not show a Metal speed advantage. No Neural Engine claim or production-safe Metal filtering claim is established. The research ticket contains source citations and release-specific packaging evidence; installed binaries still require qualification.
  • Use exact Flat search as one candidate oracle and evaluate an ANN family such as HNSW only after representation and budgets are chosen. USearch is a credible comparison, not a mandated second production adapter.
  • Respect repository-supported Python 3.12 through 3.14, NumPy 2, native Apple Silicon development/CI and actual Linux deployment architecture. Resolve minimum macOS version and any support-matrix change explicitly rather than silently dropping support.
  • Measure against the existing 200,000-Item qualification context and end-to-end Serving API objectives of p95 at most 90 ms and p99 at most 150 ms; these are existing objectives, not measured ANN performance or an ANN-only budget.
  • Recall target, K values, acceptable representation drift, peak memory, concurrent scopes/requests, build/load budget and target Apple chip/RAM remain user/design decisions. Do not fabricate numeric acceptance thresholds.
  • Benchmark query-time For You and Similar Items workloads where retrieval is used, restrictive eligibility/exclusions, warm/cold states, multiple workers/scopes and background builds. Bound native threads explicitly and report oversubscription effects.
  • Separate functional checks, development-scale evidence and formal Qualification Claims. Use synthetic or permitted derived inputs; never export raw interactions or sensitive identities into benchmark artifacts.
  • Select the library/index from measured tradeoffs after thresholds are approved. No performance ranking, GPU requirement, source-build requirement or dependency installation is authorized by this specification alone.

Testing Decisions

  • Use one direct retrieval interface to compare ANN to exact same-vector results; define tie handling, eligible reference population and underfilled denominator before measuring recall.
  • Reuse existing deterministic scenario, temporal holdout, cold/sparse cohort and installed-wheel verification conventions. Check ranking quality and coverage, not just neighbor recall.
  • Record package versions/hashes, architecture, OS, CPU/RAM, dimension, dtype, index/search parameters, thread settings, random seeds and query/filter distribution.
  • Report p50/p95/p99, throughput, recall, quota fill, build time, load time, index size and peak RSS. Distinguish ANN kernel time from complete Serving API time.
  • Run clean-wheel smoke and compatibility checks on supported targets, native-load/restart tests, invalid input handling and bounded concurrency tests. No benchmark result is claimed in this specification.

Out of Scope

Custom ANN algorithms, a separate vector-search service, cross-Commerce-Scope discovery, incremental live index mutation, merchant-source reads during serving, deployment and hosted-setting changes. The later user request adds PyTorch MPS model-training qualification to this gate.

Further Notes

Evidence: Evaluate embedded ANN libraries on Apple Silicon, research branch research/apple-silicon-ann at commit 4b43852. Research completed; bounded synthetic application-oriented experiments and pinned-wheel smokes are recorded in the ANN experiment report. They do not qualify a production representation, index, memory budget, full Serving API path or rollout.

The Embedded ANN decision map retains unresolved decisions. The ready-for-agent label routes specification work; it does not waive blockers or assert design approval. No source code, dependency or runtime contract is changed by writing this document.