Two questions get run together. Did this sentence appear in my documents? And does this document support the claim? The first is decidable by a string slice, the second is not. Offline Evidence Finder, a static page I built, does only the first: lexical search over an editable English and Chinese collection, returning a source id, the exact quote, and the quote's UTF-16 start and end offsets inside source.text. It does not generate answers or judge whether a passage supports a factual claim.
Live at the Space and the standalone app, fixed commit d309763. The behavior below is from that commit.
A query that hits without answering
The sample has an unanswerable query, 今天火星咖啡价格. Of four unanswerable queries it is the only one that returns a hit (1/4), a single character 天 inside the unrelated excerpt router-zh.
Fixed development sample, not a new benchmark. The unanswerable query matches the single character 天 in an unrelated excerpt, score 1.597193.
Returning this hit is correct by design. The tool does lexical overlap, and lexical overlap promises nothing about answerability.
The hit's score differs between the two corpus revisions: 1.4588… and 1.5972…. Both revisions have 14 passages, df(天)=1 in both, and the same quote so the same dl. Only avgdl moved, from 116 to 134.64…. With idf = ln(1+(N-df+0.5)/(df+0.5)) = ln 10 constant and tf=1, the whole difference comes from avgdl in the length-normalization term. That is a recomputable instance of scores depending on the corpus.
The span is the re-checkable claim
The evidence bundle exports all current source texts plus one citation per hit, with source_id, start, end, the offset unit, and quote. What a reader can re-check for each citation is:
source.text.slice(start, end) === quote UTF-16 because JS strings index in UTF-16 units, so charCodeAt and slice share one coordinate system. The cost is that the unit is not the glyph a reader sees. Passage splitting backs off only on surrogate pairs, not on combining marks (e plus U+0301) or ZWJ sequences, so a quote can cut through a grapheme. A 502-unit string can take the grapheme boundary at 499 under a 500 limit and satisfy both constraints at once; this implementation does not do grapheme protection, as a scope choice.
Readers can run the following on a downloaded bundle as an external re-check:
import { readFileSync } from 'node:fs'; const b = JSON.parse(readFileSync(process.argv[2], 'utf8')); const byId = new Map(b.sources.map(s => [s.id, s.text])); const seg = new Intl.Segmenter(); let ok = 0, split = 0; for (const c of b.citations) { const t = byId.get(c.source_id); if (t.slice(c.start, c.end) !== c.quote) throw new Error('span mismatch: ' + c.id); ok++; const bounds = new Set([...seg.segment(t)].map(s => s.index)); bounds.add(t.length); if (!bounds.has(c.start) || !bounds.has(c.end)) split++; } console.log({ citations: b.citations.length, sliceOk: ok, graphemeSplit: split }); This catches a location that is correct but splits a glyph. It cannot catch a passage that fails to support the conclusion.
Tokenization and scoring
Tokenization is NFC plus lowercasing, Latin, digit, and underscore runs as words, Han runs emit single characters plus adjacent bigrams, query terms deduplicated. No stemming, stop words, embeddings, or translation. Titles are displayed but not indexed; I have not measured what indexing titles would change.
Scoring is fixed-parameter BM25, k1=1.2 and b=0.75. N and df count passages with zero-token passages in N, dl is the number of emitted tokens, a positive score is required, and the UI returns at most five hits (search() itself accepts 1 through 20). Ties sort by source id as Unicode code points then start offset, so tie order reproduces across locales. The IDF expression and defaults were checked against Lucene's BM25Similarity docs; no parity is claimed.
What the sample measures
Eighteen hand-written development queries split into 14 known-source lookups and 4 unanswerable examples. The extraction bug is a different layer: sample-card paragraph extraction (make_sample.py), not the CR/LF passage splitting in search.mjs. The first version pulled English headings in because of a leading newline, giving 12/14 Recall@1 and 13/14 Recall@3, then 14/14 after the fix. Queries and ranking parameters were unchanged, but the corrected corpus came after seeing the first result, so it is not held-out. The first corpus and full scores remain in sample-history-v0/.
The sample is 14 excerpts from seven locally accepted candidate cards, not a claim that those projects are public. What this tool verifies down to the character is a quote's position. Semantic support is not in its output.
Code is licensed under Apache-2.0. By Renchao Wu.
Originally published on widechaos.cn.
