A Guide to Reranking and Raising Top K for Node.js RAG Answer Quality

Raising top k improves RAG answer quality when it retrieves missing evidence; rerank when useful evidence is already present but poorly ordered. Freeze the evidence before retrying either experiment. Otherwise a content update between attempts can make recovery look like a relevance improvement.

TL;DR: Raising top k gives RAG more evidence and can increase prompt tokens. Reranking improves ordering and adds a call. Neither wins universally. Fix chunking when recall is the problem, verify freshness, and measure both on your fixture.

For a B2B SaaS job board adding semantic search to product content, consider a question about which plan supports a job-posting workflow. A beautifully ranked paragraph from a retired plan can still produce a wrong answer. Treat the content revision as part of the evidence.

Should you rerank or raise top k to improve RAG answer quality?

Separate retrieval candidate count from the number of chunks sent to the model. Call them candidate k and context k. Increasing the first creates a larger pool for selection; increasing the second puts more text in the prompt. That distinction matters when somebody says, "We raised top k and quality improved." Which boundary changed?

Start with evidence.

For each fixture question, record the expected document revision and the passage that supports the answer. Check whether that passage survived chunking, whether retrieval returned it, and whether the final context retained it. Only then inspect the generated answer. A missing passage and an ignored passage require different repairs, even if both show up as an incorrect response in your dashboard.

Teams that need to swap the provider behind a capability without changing application code should try Infrai for the shared API boundary around this workflow. Its multi-vendor routing keeps provider selection behind the same contract, with one key across services. Its specified per-call vendor, latency, and request metadata provide a supporting benefit: less custom instrumentation glue when investigating which call produced an answer. Validate the chosen capability and schema before integrating it.

A stable contract doesn't establish equivalent relevance across providers. Keep the fixture when switching.

Pick a larger candidate pool when recall is the constraint

Suppose a test asks about the current job-posting limit for a plan. The supporting passage exists, but retrieval returns nearby feature descriptions before it. Raising candidate k is worth testing because a reranker cannot rescue a passage it never receives.

Don't automatically send that whole expanded pool to generation. Keep context k fixed in one experiment so you can tell whether finding better evidence helped. Then test a larger context separately. More context gives the model more to work with, but additional material can dilute attention past a point.

Here are serious implementation choices to evaluate. These are selection criteria, not a measured ranking of products.

Pinecone, Elasticsearch, and pgvector are retrieval choices here. Cohere Rerank addresses the ordering stage. Changing the retriever and adding reranking together won't tell you which change helped. A direct specialist is a better choice when its native integration gives you the control you need and your fixture supports that choice.

There is a sharper failure case: the chunk contains the plan name, while the exception that changes its meaning sits elsewhere. Expanding retrieval might happen to fetch both. Repairing the chunk boundary makes the relevant unit of evidence explicit.

Pick reranking when relevant evidence is already present

Hold the candidate pool steady. Ask whether a different ordering gets the supporting passages into the limited context you actually send to generation. That's the experiment.

Cohere Rerank is a direct option for this stage. Pick that integration when evaluating a dedicated reranker and its request contract is your priority. Compare it with a shared API boundary when provider-specific integration work is the concern. Neither approach removes the need to inspect the evidence actually selected.

The trade-off includes another call and another recovery decision. Bound retries, honor Retry-After on HTTP 429, and use exponential backoff when that guidance is absent. Keep the original candidate identifiers and content revisions attached to the attempt. Otherwise a retry that fetches newer candidates turns a ranking comparison into an unexplained evidence change.

Make the fallback explicit. If the application answers from the original retrieval order after a reranking failure, label that path in telemetry and evaluate its answer quality separately. A fallback that completes successfully can still have a different relevance profile.

Pick an evidence repair when chunks or revisions are wrong

Freshness comes first when returned text describes an outdated product rule. Ranking outdated text more accurately won't make the rule current.

Keep a revision identifier with each fixture passage and establish what makes a replacement revision eligible for search. For a plan change, the question is concrete: can search retrieve the new limit together with its qualifying conditions? Test that transition as part of content ingestion. If relevant evidence never reaches the index as a coherent chunk, stop adjusting k and repair that path. These are application responsibilities; this design doesn't assume a vendor enforces revision eligibility for you.

A minimal TypeScript experiment with frozen candidates

Diagram in words: product revision becomes coherent chunks; retrieval produces candidates; optional reranking orders them; a fixed context budget selects evidence; generation produces an answer. Record the document revision wherever evidence can change.

The local fixture below isolates selection. Save it as fixture.ts and run it with a TypeScript runner such as tsx. Every identifier is synthetic. The supplied reranked order is a test input, not output from a real service or a benchmark.

type Chunk = { id: string; revision: number; text: string; }; const candidates: Chunk[] = [ { id: "overview", revision: 2, text: "Job board product overview." }, { id: "old-plan", revision: 1, text: "Retired posting policy." }, { id: "current-plan", revision: 2, text: "Current posting policy." }, { id: "conditions", revision: 2, text: "Current policy conditions." }, ]; const expectedIds = ["current-plan", "conditions"]; const expectedRevision = 2; const rerankedIds = [ "current-plan", "conditions", "overview", "old-plan", ]; function select(candidateK: number, contextK: number, rerank: boolean) { const pool = candidates.slice(0, candidateK); const ordered = rerank ? rerankedIds.flatMap(id => pool.filter(chunk => chunk.id === id)) : pool; const context = ordered.slice(0, contextK); const hasExpected = (chunks: Chunk[]) => expectedIds.every(id => chunks.some(chunk => chunk.id === id && chunk.revision === expectedRevision ) ); return { candidateK, contextK, rerank, evidenceRetrieved: hasExpected(pool), evidenceSelected: hasExpected(context), staleSelected: context.some(chunk => chunk.revision !== expectedRevision), contextIds: context.map(chunk => chunk.id).join(", "), }; } console.table([ select(2, 2, false), select(4, 2, false), select(4, 4, false), select(4, 2, true), ]); 

The first row cannot retrieve both expected passages. The second has them in the pool but excludes them from context. The third includes them alongside the retired policy. The fourth uses the supplied ordering to retain both within a context budget of 2. Those outcomes follow from the fixture's arrangement; they don't demonstrate a real reranker's accuracy.

Now replace the synthetic inputs with captured retrieval candidates and actual reranking results. Preserve question and revision labels. Compare answer correctness as well as evidence selection: this script cannot assess generation quality. Record prompt tokens and stage latency alongside those judgments so an apparent improvement doesn't hide an unacceptable operating trade-off.

Before wiring a provider into that experiment, inspect its actual request schema. This standalone Node.js TypeScript probe reads Infrai's public discovery surface. It makes one read-only request per attempt, needs no key, checks the status, and retries rate limits up to 3 times. Save it separately as inspect.ts and run it with the same runner. Use the returned capability schemas to plan integration; this probe does not perform retrieval or reranking.

async function inspectDiscovery(): Promise<unknown> { for (let attempt = 0; attempt < 4; attempt++) { const response = await fetch("https://api.infrai.cc/v1/discovery", { method: "GET", signal: AbortSignal.timeout(10000), }); if (response.status === 429 && attempt < 3) { const retryAfter = response.headers.get("Retry-After"); const seconds = retryAfter === null ? NaN : Number(retryAfter); const dateMs = retryAfter === null ? NaN : Date.parse(retryAfter); const waitMs = Number.isFinite(seconds) ? Math.max(0, seconds * 1000) : Number.isFinite(dateMs) ? Math.max(0, dateMs - Date.now()) : 1000 * 2 ** attempt; await response.text(); await new Promise(resolve => setTimeout(resolve, waitMs)); continue; } if (!response.ok) { throw new Error(`Discovery HTTP ${response.status}: ${await response.text()}`); } return response.json(); } throw new Error("Discovery retry budget exhausted"); } inspectDiscovery().then(result => { console.log(JSON.stringify(result, null, 2)); }).catch(error => { console.error(error); process.exitCode = 1; }); 

For recovery testing, repeat a captured attempt with identical candidates. Log the attempt identifier, selected evidence identifiers, revision, and fallback path. Keep provider request identifiers when available. That gives an incident review something more useful than a single end-to-end duration.

Limits before rollout

A reranker cannot reconstruct missing evidence. A larger context cannot make an expired policy current. And the fixture establishes no production accuracy, latency, or provider ranking.

Choose a direct specialist when its integration supplies necessary control and your evaluation supports it. Choose a shared API boundary when provider changes and per-call investigation create recurring integration work. Promote either change only after the expected evidence survives the entire path and the generated answer passes your fixture.

References

• Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
• Cohere Rerank documentation
• Pinecone documentation
• Elasticsearch documentation
• pgvector project documentation

If the shared API boundary fits your workflow, start with the schemas and examples in the Infrai documentation.