Short answer: treat metadata eligibility as a versioned contract, and test it before semantic retrieval. In an e-commerce catalog, tenant, market, lifecycle, and stock rules should be resolved deterministically before a vector index is asked for similar text. This makes retrieval behavior explainable when a product disappears, a catalog is backfilled, or a buyer changes region.
The practical flow is small: normalize each product into text plus typed fields, record the policy version that admitted it, and pass only eligible IDs into lexical or vector search. The reliability gain comes from making rejection observable. A result that is semantically close but forbidden is a policy failure, not a ranking failure.
Start with a retrieval contract, not a query template
Write the contract in terms a catalog owner can review. tenant_id and market are exact boundaries; status and sellable are explicit lifecycle decisions; a stock window has a timezone and a null policy. Keep descriptive facets separate from authorization and availability fields. When a field changes meaning, increment the policy version and re-evaluate affected records instead of silently reusing old embeddings.
I once expected a category predicate to be harmless metadata. An eval replay showed a discontinued child-size helmet winning a safety-gear query because the filter ran after retrieval. Three traces later, the actual defect was clear: the reranker had already consumed the invalid item. I followed the request from the catalog snapshot to the embedding queue, then through the candidate log and the final prompt; every component reported success because none owned the business predicate. The confusing part was that the answer looked useful to a casual reviewer, while a buyer in the wrong market could still see the product. We added adversarial rows for cross-tenant, expired, and missing values, recorded the policy version beside each expected ID, and made the rejection reason part of every test result. It failed loudly, which made the repair straightforward. That loop changed our review question from “did the model rank this well?” to “was this item allowed to enter the race?”
Start with IDs.
How can metadata filters make product search retrieval predictable?
Use two explicit stages. First, evaluate cheap predicates against an authoritative catalog store or a filterable index. Then retrieve and rerank only among the surviving IDs. The interface between stages should carry the candidate IDs, policy version, and a reason when the set is empty. That envelope is useful in a notebook and in production because the same decision can be replayed.
from dataclasses import dataclass from datetime import date @dataclass(frozen=True) class Product: tenant_id: str market: str status: str sellable: bool stock_until: date | None def eligibility(product: Product, tenant: str, market: str, today: date) -> tuple[bool, str]: if product.tenant_id != tenant: return False, "tenant" if product.market != market: return False, "market" if product.status != "active": return False, "lifecycle" if not product.sellable: return False, "sellable" if product.stock_until is not None and product.stock_until < today: return False, "stock_window" return True, "ok" The vector service can receive the resulting IDs as a server-side filter, or the catalog query can remain authoritative while retrieval operates on its output. Fetching broad neighbors and filtering in application code is less predictable: a small top-k can omit the valid item, while a large top-k increases latency and exposes tenant-shaped signals in logs. Deterministic eligibility belongs at the boundary.
Build evals around policy transitions
A useful harness is a replay corpus, not a handful of happy-path queries. Each case stores query text, tenant, market, stock date, expected policy label, and expected product IDs. Check the policy and ranking separately: forbidden IDs must never enter the candidate set; an eligible expected ID should be present before reranking; the returned evidence should carry the current product version.
Include transition cases: a product changes from active to discontinued, a market is added, a stock deadline crosses midnight, or a tenant field is missing during a backfill. Run the same corpus in a notebook and in CI. The output should include candidate count, rejection counts by reason, p95 retrieval latency, and reranker token count. Prompt cost matters because twenty near-duplicate variants can swamp an otherwise correct answer.
Your mileage may vary with catalog shape. A ten-variant product may deserve one shared description chunk and ten volatile price records in metadata; measuring both recall and evidence volume tells you which representation is stable.
Operate the boundary and know when to switch
The contract is not suitable when eligibility depends on a computed value unavailable at query time, such as a personalized contract price assembled from several systems. Resolve that value first, then use semantic retrieval to explain the selected products. For a tiny catalog that fits comfortably in a transactional database, full-text search may be easier to operate than a vector pipeline.
The catch is ownership. Every filter field needs an owner, allowed values, null behavior, and a backfill plan. Trace the policy version and candidate counts, and alert on sudden zero-result spikes by tenant or market. A 99.9% rejection rate can be a legitimate promotion rule or a timezone mistake; only the reason distribution distinguishes them.
Roll out one category at a time behind a flag. Compare the new contract with a baseline on forbidden-result rate, eligible recall, p95 latency, index bytes per active product, and reranker tokens. After launch, sample zero-result requests and inspect rejection reasons rather than judging only the generated answer. Re-run the corpus whenever taxonomy, market coverage, or stock semantics change.
For a backfill, write the policy version into the same record as the embedding metadata and emit an event containing the product ID, old decision, new decision, and source timestamp. Consumers can then rebuild only records whose decision changed. Keep the old version readable during the rollout window so a replay can explain why yesterday's result differs from today's. This is where a schema registry or a checked-in JSON Schema earns its keep: the accepted values, required fields, and date formats become reviewable artifacts instead of tribal knowledge. A failed validation should quarantine the record with a reason code, not coerce an empty string into a valid market. During a migration, compare rejection histograms for the old and new versions on the same snapshot. Large differences are a prompt to inspect the data, not evidence that the new policy is better. Once the new version is stable, expire old vectors deliberately and record that retention decision. These small records make incident response a reconstruction exercise instead of guesswork.
The durable rule is simple: eligibility is deterministic, similarity is probabilistic, and reliability comes from recording the boundary between them.
