How the scanner in Foilcase works out which printing of a trading card it is looking at: Vision feature prints for the artwork, text recognition for the collector number, and a strict rule for when to ask instead of guessing. Plus the storage underneath it: an append-only ledger and one daily price file for everyone. Foilcase is my iPhone app for scanning and pricing cards from five trading card games (App Store); the user-facing write-up covers what it does.
Why artwork matching picks the wrong print
Before release I built a harness on a Mac that calls the same Vision APIs as the app: 9,903 cards indexed, 400 queries, half drawn on purpose from collector numbers that more than one print shares. Each query was a synthetic degradation of an official card image (perspective, blur, glare, colour cast, JPEG).
| Sharp frame, 400 queries | Correct |
|---|---|
| Artwork, top-1 | 87.8% |
| Artwork, top-1 on shared numbers | 78.5% |
| Artwork, top-5 | 99.8% |
| Artwork, top-1 when given the correct number | 89.5% |
The last row changed the design. Even with the right collector number, artwork matching picks the right print 89.5% of the time, because alternate arts and higher-rarity parallels often share one number and one illustration. No embedding separates two prints of one picture, so the app asks which print, as it asks whether a card is foil. And since five candidates nearly always include the right card, the five-card sheet is the main flow, not an error state.
The embedding is VNGenerateImageFeaturePrintRequest, revision 2: 768 floats, about 9 ms an image. One option matters more than it looks:
static func featurePrint(of image: CIImage) -> [Float]? { let request = VNGenerateImageFeaturePrintRequest() request.imageCropAndScaleOption = .scaleFill try? VNImageRequestHandler(ciImage: image, options: [:]).perform([request]) guard let observation = request.results?.first else { return nil } var vector = [Float](repeating: 0, count: observation.elementCount) _ = vector.withUnsafeMutableBytes { observation.data.copyBytes(to: $0) } return vector } static let artworkMatchingAvailable: Bool = { let probe = CIImage(color: .gray).cropped(to: CGRect(x: 0, y: 0, width: 64, height: 90)) return featurePrint(of: probe) != nil }() The default, .centerCrop, cuts the top and bottom off a tall card: 82.4% against 93.2% for .scaleFill on the same 1,295-card index. The probe exists because feature prints fail in the Simulator ("Failed to create espresso context"). The app checks once and falls back to the collector number alone, as it would on any device without a usable neural engine.
Distance is not a confidence score
The obvious next step is a distance threshold. Measured, it says almost nothing: median distance was 0.450 when the top match was right and 0.420 when it was wrong. But distance does tell whether there is a card at all: photographed cards sat at a median of 0.459, a phone pointed at a laptop at 0.966. I found that out when the scanner, with no ceiling, turned a browser window into twenty-six confident entries. The ceiling is now 0.80, which keeps 99% of photographed cards and rejects 89% of everything else.
Reading the collector number
The number is unambiguous whenever it is legible, so most of the work went into legibility.
Drop soft frames. Between a handheld frame and a sharp one, the artwork gained 5.6 points and the collector number 46 (32.8% read against 78.5%). The gate is the variance of the Laplacian on a greyscale buffer of the rectified card. My first version chained Core Image filters, a Laplacian then a multiply blend, and returned negative variances: blend modes clamp and premultiply, they are not arithmetic.
Don't ask Vision to find the card. On iOS, VNDetectDocumentSegmentationRequest returned the card's text box at 0.96 confidence, and VNDetectRectanglesRequest found artwork panels, never the border. So the scanner draws a card-shaped guide, tries three crops around it and keeps whichever embeds closest to the index.
Read in passes, at full size. Downscaling crops is how my first benchmark reported an OCR accuracy of zero. VNRecognizeTextRequest runs over the whole card, then the top and bottom 22% enlarged 2.5 times, then three corners at 4 times if nothing turned up:
private func scan(_ image: CIImage, into found: inout Set<String>, minimumTextHeight: Float = 0.008) { let request = VNRecognizeTextRequest() request.recognitionLevel = .accurate request.usesLanguageCorrection = false // these are codes, not words request.recognitionLanguages = ["en-US"] request.minimumTextHeight = minimumTextHeight // the default floor skips the number try? VNImageRequestHandler(ciImage: image, options: [:]).perform([request]) for observation in request.results ?? [] { for candidate in observation.topCandidates(2) { found.formUnion(PrintedNumber.extract(from: candidate.string)) } } } The recognizer swaps look-alikes (S and 5, O and 0, B and 8), so readings are folded before comparing. That tolerance is what kept producing wrong additions. The fix was two keys: a loose one, padding removed, for finding candidates, and a literal one, padding and variant marks kept, for adding without asking. "45/195" fits a catalog "045/195", but it is also what "145/195" looks like with its first digit lost.
When the scanner adds and when it asks
Add without asking only when the artwork's first choice (or a card it can't tell apart from it) and a reading that spells that card's number literally name the same card, and nothing the artwork can't rule out shares that number. Everything else is the sheet. Trimmed from ScanPipeline.decide:
func blocks(_ card: CardRef) -> Bool { guard card.id != winner.id else { return false } guard baseName(card) != baseName(winner.card) else { return true } // same card, other print guard let margin = policy.distinctParallelMargin, let mine = winner.distance, let theirs = distances[card.id] else { return true } return theirs < mine * (1 + margin) } if parallels.contains(where: blocks) { return .ambiguous(candidates: offered, reason: .parallelPrints, numbers: numbers) } let artworkAgrees = winner.id == candidates[0].id || artworkTies(candidates[0], winner, policy.minimumRelativeMargin) let byNumber = winner.numberMatched && artworkAgrees && (spelled(winner.card, literals) || unpadded) guard byNumber else { return .ambiguous(candidates: ordered, reason: winner.numberMatched ? .numberDisagrees : .lowConfidence, numbers: numbers) } return .identified(winner) A different card sharing the number stops blocking only if the artwork puts it at least 15% farther away. unpadded lets an old card printed "18/130" match a catalog "018/130", but only when it lands on the artwork's own first pick.
The rule took three rounds of adversarial review to reach, each reported bug reproduced against the real catalogs. The first confirmed 33 defects; the second found 12 new ones, 11 introduced by the fixes. As my commit message put it, a comparison built to find candidates was also allowed to add one. Adding on artwork margin alone went too: a card missing from the index still has a nearest neighbour that can be clearly ahead and wrong.
Before a silent add, a second text request also counts Han, kana and hangul characters, because artwork and number are identical across languages and each catalog holds one. English cards read 0 to 3, Japanese ones 37 to 188; the cut-off is 8.
Then I measured on 529 real photographs of cards, labelled against the catalog, with all five games installed (62,331 cards in one index) and the app's recognition code running on a Mac. Every silent add was checked by eye:
| 529 real photos | Before | After |
|---|---|---|
| Added without asking, wrong card | 0 | 0 |
| Added without asking, right card | 21% | 42% |
| Right card added, or first in the sheet | 69% | 90% |
| Collector number read | 59% | 77% |
The gains came from the photos: a picked photo now skips the quality gates, since no better frame is coming; the number is read from the loosest crop when the tightest cut it off; and each crop is also embedded slightly blurred, because a holo's sparkle pulled matches away from the flat scans in the index (right card first: 377 to 397). The live camera path is still unmeasured; for that, the app keeps an on-device calibration log of each attempt and the card the user actually picked.
Keeping a 27,000-card index on the phone
Each game's index downloads separately, quantised to int8 with one scale per pack. On queries perturbed to the distance a handheld photo sits at, int8 and float16 returned the same card 99.8% of the time, for half the bytes. There's no approximate nearest-neighbour library: in the Mac harness, a brute-force scan of the largest index, 26,978 vectors, takes 1.6 ms widened to Float, but holds 83 MB. So the app memory-maps the int8 bytes and converts 512 rows at a time with vDSP_vflt8 and vDSP_distancesq: 5.6 ms, nothing resident. The scan runs only on a steady frame, so four milliseconds is invisible and 83 MB isn't. A pack from a different Vision revision is refused; its distances live in another space.
The index is built on a Mac and lags the catalog, so a clearly read number naming a card the artwork never offered goes to the top of the sheet, asked about and never added.
A ledger that never overwrites, and one price file for everyone
The collection is an append-only ledger table plus an items table derived from it in the same transaction. Removal is a tombstone; rebuild() replays the log. Replay needs an order every device agrees on, and the middle term of the sort key is a bug fix: the scanner adds a card and sets its finish within one millisecond, and a random event id sometimes sorted the edit first.
/// Creation sorts before every edit to the same item at the same instant. var replayRank: Int { if case .added = self { return 0 } return 1 } for record in records.sorted(by: { ($0.timestamp, $0.event.replayRank, $0.eventID) < ($1.timestamp, $1.event.replayRank, $1.eventID) }) { It paid off in October. The upstream catalog renamed sets, card ids include the set name, and 1,323 cards in one game got new ids overnight, 222 more across the others. Copies saved under old ids dropped off the collection screen. The fix is an alias table read in memory; the log stays as written.
Prices are one binary file per game, the same bytes for every user, so a request says nothing about what anyone owns. Names are content-addressed and the manifest is written last, so a client never sees half an update. Prices are integer cents, with a sentinel so a $0.00 promo isn't an unpriced card. Each carries the day it was captured, and past PriceProvenance.staleAfterDays (7) the portfolio headline says "These prices are 2 weeks old" instead of presenting an old total as current.
What the app does for the people who use it: Scan trading cards and see what they're worth with Foilcase.
FAQ
How do you tell apart card printings that share the same artwork?
Mostly not from the image: in my pre-release harness, artwork matching picked the right print 89.5% of the time even when given the correct collector number. Foilcase reads the number with Vision text recognition and, when other prints share it, shows the candidates and asks.
Is VNGenerateImageFeaturePrintRequest good enough for on-device image recognition?
For retrieval, yes, with no model to ship: the right card was in the top five 99.8% of the time on sharp synthetic frames of a 9,903-card index. Use .scaleFill, and test on a device, because it doesn't run in the Simulator.
Do I need an approximate nearest neighbour library for on-device image search?
Not at tens of thousands of vectors. A brute-force Accelerate scan over 26,978 feature prints took 1.6 ms on a Mac in Float, or 5.6 ms as memory-mapped int8, which avoids holding 83 MB in memory.
Can embedding distance be used as a confidence score?
Not between similar images: median distance was 0.450 for correct matches and 0.420 for wrong ones. It does separate cards (0.459) from non-cards (0.966), so Foilcase uses it only as a 0.80 ceiling.
How do you read small printed numbers with VNRecognizeTextRequest?
Lower minimumTextHeight, turn off language correction, never downscale the crop, and add passes over enlarged strips and corners. Rejecting blurry frames first matters most: it took collector-number accuracy from 32.8% to 78.5% in my harness.
Why store a collection as an append-only ledger?
Nothing is overwritten, so a mis-tap or a bug can't destroy data: removal is a tombstone and the item table is rebuilt from the log. Changed upstream ids resolve through an alias table at read time, without touching user data.
