AI & ML
Pix2Evidence: A Multimodal RAG System for Text and Visual Evidence
arXiv cs.IR0 views
arXiv:2610.41335v1 Announce Type: new Abstract: Retrieval-augmented generation (RAG) over visually rich documents must find both relevant content and the evidence that supports an answer. Screenshot retrieval preserves images and page structure, but written facts within screenshots also need to be searchable. Moreover, topical relevance alone does not identify which retrieved tile contains evidence for the answer. We introduce Pix2Evidence, a multimodal RAG system that encodes each screenshot tile together with its available transcription into a single fused image--text representation, so that layout and readable text are considered jointly during retrieval. To guide evidence selection, a vision-language teacher describes the visible content that supports each training question. We use these descriptions to adapt the retriever, then rerank the retrieved candidates before answer generation. Extensive experiments demonstrate that Pix2Evidence outperforms state-of-the-art methods. Across five widely-used Wikipedia QA benchmarks, the non-reranked Pix2Evidence leads the main comparison in answer accuracy on four. On MMSearch, it reaches 26.0\%, 4.4 percentage points above PixelRAG. We will release the code and share derived data where licensing and privacy reviews permit.
Source:https://arxiv.org/abs/2610.41335PreviousMILES-Rec: Uncovering Implicit Intents from Mobility Behaviors for Cross-Domain POI RecommendationNextDefining Page-Centric Visual Document Retrieval and Evaluating Existing Benchmarks
Related posts
Kalshi court losses pile up as judges refuse to block state gambling laws
0Ars Technica

Tracking Vietnamese news trends with embeddings, HDBSCAN and an LLM
0dev.to
How to Organize and Improve Your AI Prompts Over Time
0dev.to
The Hackintosh Reborn
0Hacker News

Formally verifying agent-written full-stack web apps with PDD Kit
0Microsoft Research
I built an Android app with AI. Does that count as vibe coding?
0dev.to Discuss