AI & ML
Sycamore: Grounded Synthetic Evaluation for Visualization Systems
arXiv cs.HC0 views
arXiv:2605.08630v4 Announce Type: replace Abstract: Evaluating visualization systems in specialized domains, such as genomics, is challenging due to the scarcity of domain experts and difficulty recruiting relevant user types. LLM-based synthetic personas provide an appealing workaround, but they face skepticism as their claims often lack verifiable grounding. We present Sycamore, an exploratory approach to evaluating visualization systems with grounded synthetic personas whose responses are restricted to interview evidence. Sycamore enforces provenance with participant citations for each claim and an abstention mechanism when retrieval yields no relevant evidence. A protocol runner guides the synthetic personas through an evaluation of the live visualization system, producing multi-turn transcripts for qualitative coding. We instantiate the approach on Geranium, a multimodal genomics visualization search engine, with seven evaluators grounded in prior interviews on genomics visualization authoring. We report themes in the resulting transcripts, highlight an emergent requirement absent from the source data, and provide a preliminary comparison to a prior human user study. Recognizing that our findings derive from an exploratory perspective and that synthetic proxies cannot replace human evaluation, we discuss their implications for integrating synthetic evaluators alongside expert user studies. The protocol, codebook, extended results, transcripts, and demo video are in the Supplementary Materials and at https://osf.io/kdfr3/; the source code and project page are at https://huyen-nguyen.github.io/sycamore/.
Source:https://arxiv.org/abs/2605.08630PreviousMap2Video: Guiding Real-World-Grounded AI Video GenerationNextProfiling Handwriting-Process Deviations in Developmental Dysgraphia: An Open, Normatively-Referenced Instrument
Related posts

Your guide to containers at AWS re:Invent 2026
0AWS Containers Blog

Three Asian forwarders, one corridor: what Nippon Express, Yusen and CJ Logistics actually publish
0dev.to Data
Tương quan số liệu Nice vs PSG: Phép thử phòng thủ tại Allianz Riviera
0dev.to Data
Jacksonville, Florida Property Listings Dataset (2026-11-04): 6534 Rows, Excel Spreadsheet
0dev.to Data
Tương quan phong độ Parma vs AS Roma: Bài toán kiểm soát hỏa lực đội khách
0dev.to Data
Schalke 04 chạm trán Stuttgart: Bài toán hiệu suất chuyển hóa tại VELTINS-Arena
0dev.to Data