AI & ML
Breaking the Modality Barrier: Multimodal Fusion With Missing Modalities for Cybersickness Prediction
arXiv cs.HC0 views
arXiv:2611.07386v1 Announce Type: new Abstract: Cybersickness is a major barrier to comfortable VR use, and reliable prediction often requires multiple physiological and behavioral sensors. However, existing models typically assume a fixed set of modalities, making them vulnerable when sensors are unavailable and difficult to generalize across datasets with different sensor configurations. We adapt a transformer-based model that handles arbitrary sensor combinations without imputing missing data. Anchor-based contrastive learning maps heterogeneous sensor embeddings into a shared latent space using eye tracking as the anchor modality, while masked self-attention excludes unavailable sensors during fusion. We evaluate our approach on three publicly available cybersickness datasets: VR.net, SIM21, and Blur-Rollercoaster, collected using different VR systems and sensor configurations. Across single-sensor dropout conditions, prediction error changes by less than 0.09 MAE. Joint training across all datasets achieves 0.51 MAE and 0.63 Pearson correlation, substantially outperforming single-dataset models. These results demonstrate robust and generalizable cybersickness prediction across diverse VR systems.
Source:https://arxiv.org/abs/2611.07386PreviousSlide to Report: Continuous Reporting Symptom-Specific Cybersickness in VRNextThinking Is Not Writing: Designing AI Chatbots to Connect Reasoning and Revision
Related posts

Running Local LLMs in the Browser with WebGPU and Transformers.js
0dev.to

ChatGPT Productivity for Developers: Build a Workflow, Not a Prompt Library
0dev.to AI
A human approval step for an LLM agent: the boundary that matters
0dev.to Security
如何像 osFoundry 一樣用 Python 包裝 Codex CLI
0dev.to Python

The Microsoft Lab That Worked on AutoGen Is Over. AutoGen Already Gets No Security Fixes
0dev.to Python
GitHub Copilot Just Separated Your BYOK Spend From Its Own — Here's What the New Usage API Actually Shows You
0dev.to DevOps