Backend
Multimodal Person Concept Model for Person Identification under Varying Observation Conditions
arXiv cs.RO0 views
arXiv:2611.17802v1 Announce Type: new Abstract: Service robots in domestic and office environments must identify users to provide personalized services based on their preferences and previous interactions. This is challenging because only a few observations may be available for learning, while observation conditions change owing to appearance, environmental noise, and utterance content. To address this challenge, this study proposes a multimodal person concept model that learns person concepts from a small number of observations, without person labels, and enables robust identification under changing conditions. The model employs a nonparametric Bayesian generative model based on the Chinese restaurant process and integrates visual, acoustic, and linguistic features extracted using Contrastive Language-Image Pre-training (CLIP) and Contrastive Language-Audio Pretraining (CLAP), followed by principal component analysis (PCA). We evaluated six participants under eight conditions in which appearance changes, additive noise, and transcription errors were introduced individually and in combination. Averaged over the eight conditions, the proposed model achieved an identification accuracy of 0.89 while inferring the number of person concepts from the observations, which is comparable to k-means clustering that was supplied with the true number of participants (0.88). We also examined PCA dimensionality, modality availability, and variation strength, and demonstrated the model on a Toyota Human Support Robot. These results show that multimodal person concept learning supports robust person identification under changing conditions, and that identification can succeed even when the number of concepts is not correctly recovered. The project page and demonstration video are available at https://okitamisan01.github.io/person_concept/.
Source:https://arxiv.org/abs/2611.17802PreviousBeyond Prediction Gain: LASR as a Metric for Action-Grounded Latents in World ModelsNextPer-Agent-Safe Navigation of Globally Driven Robot Collectives via Affine Field Design
Related posts
General coupling-based f-divergence diagnostics for Markov chain Monte Carlo
0arXiv stat.TH
Self-Normalized Bounds for Time Series with Sample Variance Only and Their Application to Forecasting
0arXiv stat.TH
High-Dimensional Gaussian and Bootstrap Approximation: Near $1/\sqrt{n}$ Rates with General Covariance
0arXiv stat.TH
From Reading Code to Reading Spec: A Verified Layer for LLM-Driven Codebase Maintenance
0arXiv cs.SE
A Governance Methodology Layer for AI-Assisted Software Development: Defect Taxonomy, Controlled Ablation, and a Test of Process-Over-Capability
0arXiv cs.SE
Identifying and Mitigating API Misuse in Large Language Models
0arXiv cs.SE