AI & ML
Pix2Evidence: A Multimodal RAG System for Text and Visual Evidence
arXiv cs.IR0 views
arXiv:2610.41335v1 Announce Type: new Abstract: Retrieval-augmented generation (RAG) over visually rich documents must find both relevant content and the evidence that supports an answer. Screenshot retrieval preserves images and page structure, but written facts within screenshots also need to be searchable. Moreover, topical relevance alone does not identify which retrieved tile contains evidence for the answer. We introduce Pix2Evidence, a multimodal RAG system that encodes each screenshot tile together with its available transcription into a single fused image--text representation, so that layout and readable text are considered jointly during retrieval. To guide evidence selection, a vision-language teacher describes the visible content that supports each training question. We use these descriptions to adapt the retriever, then rerank the retrieved candidates before answer generation. Extensive experiments demonstrate that Pix2Evidence outperforms state-of-the-art methods. Across five widely-used Wikipedia QA benchmarks, the non-reranked Pix2Evidence leads the main comparison in answer accuracy on four. On MMSearch, it reaches 26.0\%, 4.4 percentage points above PixelRAG. We will release the code and share derived data where licensing and privacy reviews permit.
Source:https://arxiv.org/abs/2610.41335PreviousMILES-Rec: Uncovering Implicit Intents from Mobility Behaviors for Cross-Domain POI RecommendationNextDefining Page-Centric Visual Document Retrieval and Evaluating Existing Benchmarks
Related posts
Show HN: Grid – A Hold'em arena where your AI agent plays your strategy (Beta)
0Hacker News Show
Show HN: Mercury 3 cost calculator (launch promo vs. list pricing)
0Hacker News Show

'Gen Z uses AI the way their grandparents smoked': Students say they feel pressured to use AI, and many would ban it if they could
0TechRadar

Apple Store now offers extended holiday returns for all products
09to5Mac
I bought the knock-off Pixel Watch and Fitbit Air bands, and it’s crazy how good they are [Gallery]
09to5Google

Amazon will let you text Alexa your shopping list
0The Verge