For recruiting candidate search, use staged retrieval with explicit collections, bounded queries, and source context that survives every fallback. Start with the tenant's private index, broaden only when its evidence is thin, and return document identifiers or URLs beside each claim. A slower, less complete answer is preferable to a fast match that a recruiter cannot audit.

This is a retrieval problem before it is a generation problem. The flow below keeps candidate records, access-control metadata, and citation fields together; it also gives a web-backed lookup a fixed budget so an unavailable source cannot hold the search screen hostage.

How should Node.js retrieval architecture handle recruiting candidate search fallbacks?

I use three stages. Stage one searches the tenant's curated candidate collection with hard filters. Stage two searches a shared, policy-approved talent pool only when the first stage returns too few eligible hits. Stage three performs a web lookup for public portfolio evidence, then marks those results as external and lower-trust. Every stage emits the same result contract, so the caller does not need to know which path answered.

The fallback condition is evidence-based, not an exception catch. A result set can be non-empty and still fail a minimum score, freshness, or citation threshold. Conversely, a transient 429 from a web source should consume one retry and then end the stage; it should not silently turn a private search into an unbounded scrape.

Here is a compact TypeScript skeleton. The adapters are deliberately boring: a vector store, a keyword store, and a web search service can be swapped independently. tenantId, ACL labels, and source locators remain mandatory at the boundary.

type Access = { tenantId: string; visibility: "private" | "shared" | "public"; roles: string[] }; type Evidence = { id: string; url?: string; title: string; excerpt: string; updatedAt?: string }; type CandidateHit = { candidateId: string; score: number; access: Access; evidence: Evidence[]; stage: 1 | 2 | 3 }; type Query = { tenantId: string; text: string; viewerRoles: string[]; limit: number }; interface RetrievalStage { search(query: Query, signal: AbortSignal): Promise<CandidateHit[]>; } function hasEnoughEvidence(hits: CandidateHit[]): boolean { const cited = hits.filter((hit) => hit.evidence.length > 0); return cited.length >= 5 && (cited[0]?.score ?? 0) >= 0.72; } async function withDeadline<T>(run: (signal: AbortSignal) => Promise<T>, ms: number): Promise<T> { const controller = new AbortController(); const timer = setTimeout(() => controller.abort(), ms); try { return await run(controller.signal); } finally { clearTimeout(timer); } } async function retry429<T>(run: () => Promise<T>): Promise<T> { for (let attempt = 0; attempt < 2; attempt++) { try { return await run(); } catch (error) { const status = (error as { status?: number }).status; if (status !== 429 || attempt === 1) throw error; await new Promise((resolve) => setTimeout(resolve, 250 * (attempt + 1))); } } throw new Error("retry limit reached"); } export async function retrieve( query: Query, privateIndex: RetrievalStage, sharedIndex: RetrievalStage, webIndex: RetrievalStage, ): Promise<CandidateHit[]> { const first = await withDeadline((signal) => privateIndex.search(query, signal), 700); if (hasEnoughEvidence(first)) return first; const shared = await withDeadline( (signal) => sharedIndex.search({ ...query, limit: Math.min(query.limit, 20) }, signal), 700, ); const merged = [...first, ...shared].filter((hit) => hit.access.tenantId === query.tenantId); if (hasEnoughEvidence(merged)) return merged.slice(0, query.limit); try { const external = await withDeadline( (signal) => retry429(() => webIndex.search({ ...query, limit: 10 }, signal)), 1_200, ); return [...merged, ...external.map((hit) => ({ ...hit, stage: 3 as const }))].slice(0, query.limit); } catch { return merged.slice(0, query.limit); } } 

The sample uses an intentionally small limit and timeout. Tune them with an evaluation set, not intuition. The important invariant is that webIndex cannot remove the tenant filter from earlier hits, and an empty external response is still an honest result rather than a fabricated candidate.

How do citations and access control survive recruiting search fallbacks?

Index one candidate profile as several traceable items: employment history, skills, certifications, and portfolio links. Each item keeps a stable evidence.id, its original URL or document key, an update timestamp, and the access labels needed by the viewer. Store the source text or excerpt alongside the embedding. Rewriting a profile into a summary before indexing makes later review harder because the summary no longer proves what the source said. I also keep the source system's content version and deletion marker, because a recruiter should be able to tell whether a result was current when the query ran. When a candidate changes employers, the old item is not silently overwritten: the new item receives a new version, the old one is excluded from normal retrieval, and an audit record points to the replacement. That extra bookkeeping feels slow during ingestion, but it prevents a familiar support question: “Why did this answer cite a role the candidate left last year?”

Apply authorization twice. Filter at query time, then filter again after merging stages. The second check protects against adapter mistakes and stale ACL metadata. A shared record that lacks a tenant mapping is not a public record; quarantine it from the result set and log the missing field.

For generation, pass evidence as numbered passages and require citations in the output schema. The UI should link [3] to the stored locator, not to a URL invented by the model. If no passage supports a claim, return “insufficient evidence” and let the recruiter refine the query.

Short answers help here.

Ship the contract first.

Failure budgets are part of the retrieval contract

Set a deadline per stage and a total deadline for the request. In the example, the total can reach roughly 2.6 seconds before serialization, so production code should reserve headroom for reranking and rendering. Record stage latency, hit count, minimum score, citation count, timeout, and retry status as structured fields. A single aggregate latency metric will hide a web fallback that is quietly consuming the budget.

Retry only conditions that are plausibly temporary. HTTP 429 deserves a bounded retry and respect for the source's Retry-After guidance when available. A malformed query, a denied tenant, or a 4xx policy response should not be retried. When a stage times out, return the evidence already collected and mark the response as partial so users understand why public results are absent.

Make ingestion idempotent. Derive an item key from tenant, candidate, source locator, and content version; upsert that key and explicitly delete records removed from the source. Otherwise a profile update can leave two stale embeddings, and a fallback may cite the older one. This is an ordinary data-lifecycle issue, but it directly changes ranking quality.

What trade-offs make a fallback suitable for candidate search?

Choice Strength Cost or boundary
Tenant vector index Good semantic recall for varied recruiter language Needs embedding refreshes, ACL filters, and a score threshold that is calibrated on real queries
Tenant keyword index Exact matching for certifications, locations, and IDs Misses synonyms and paraphrases; keep it as a companion path rather than a universal replacement
Shared talent pool Adds recall when a private corpus is sparse Requires explicit consent, tenancy rules, and a visible provenance label
Public web lookup Finds portfolio pages not present in internal records Freshness, robots policies, and citation quality vary; keep a short timeout and never treat it as verified employment data
Self-hosted retrieval service Full control over retention and regional placement Your team owns capacity, upgrades, monitoring, and incident response; it is not suitable when nobody can carry that operational load

The catch is that broadening search can change the meaning of “best candidate.” If a job requires verified work authorization or an internal referral, a web fallback is not suitable even when it improves recall. Stick with the private index and return fewer results. If regional residency rules prohibit a shared pool, remove that stage rather than hiding it behind a flag.

I am not sure one score threshold will transfer across job families: a data-engineering query and a design query have different vocabulary and profile density. Resolve that uncertainty with a labeled set of recruiter queries, measuring recall at a fixed citation requirement and checking false-positive rates by tenant. Keep threshold changes versioned so an evaluation can explain why a candidate moved.

The right architecture is therefore a policy pipeline, not a list of providers. Keep the internal interface stable, make each fallback observable, and let evidence quality decide when to broaden. Generation can summarize the returned passages later; retrieval must first prove where they came from.

References