Open Source
PEARL: Personalized Streaming Video Understanding with Runtime-Defined Concepts
arXiv cs.IR0 views
arXiv:2603.20422v2 Announce Type: replace-cross Abstract: Unlike humans, who learn new concepts from a continuous visual stream, existing personalized vision-language models (VLMs) internalize a predefined concept set during training and operate on static images or offline videos. They cannot register a concept that a user defines in the middle of a stream, nor recall the earlier moments in which it appeared. We formalize this task as Personalized Streaming Video Understanding (PSVU) and introduce PSVU-Bench, the first benchmark for this task, which annotates 132 videos with timestamped questions. Its Concept-Definition, Real-Time and Past-Time QA ask a model to register concepts at arbitrary timestamps, recognize their current state, including user-defined actions, and retrieve their historical evidence. We propose PEARL, a strong plug-and-play baseline that stores user-defined concepts apart from archived video clips and rewrites each query with concept descriptions to retrieve the relevant history. Grounding a named concept in video is a skill shared by all concepts. We therefore construct PSVU-10K, a training set of verified concept questions on public internet videos, and train the retriever and answering VLM of PEARL on it once. Concepts defined at test time still require no training. Applied to three architectures without training, PEARL improves Real-Time QA by 15.05% and Past-Time QA by 12.60% on average over the corresponding offline models, and all three surpass existing online video models on both QA types. Training further improves PEARL on Qwen3-VL-8B by 8.66% on Real-Time QA and 9.39% on Past-Time QA, with gains in every category. Code, data and models are available at https://github.com/Yuanhong-Zheng/PEARL.
Source:https://arxiv.org/abs/2603.20422PreviousUniRank: Benchmarking Ranking Models for Unified Sequential Modeling and Feature InteractionNextRecommendation Quality and the Concentration of Consumption: Experimental Evidence from Netflix
Related posts
Receita Federal publica código da Calculadora de Tributos no GitHub
0dev.to GitHub
Lifting 3D Gaussians: A Wavelet Multiresolution Representation for Scalable Gaussian Splatting Streaming
0arXiv eess.IV
Binarized Spectral Compressive Imaging
0arXiv eess.IV
HDNet: High-resolution Dual-domain Learning for Spectral Compressive Imaging
0arXiv eess.IV
GramStatTexNet: Efficient, Interpretable, and Neuro-Inspired Texture Analysis-by-Synthesis
0arXiv cs.GR
Webhook Rate-Limiting Queue Design for US/EU Delayed Jobs, Retry, and DLQ
0dev.to Go