AI & ML
Exploring multimodal AI--XR interactions for software development: From expert perspectives to semantic code outlines
arXiv cs.HC0 views
arXiv:2611.24334v1 Announce Type: new Abstract: AI tools can generate code quickly, but developers still need to understand and check the results. We explore how the enriched multimodal context derived from the extended reality (XR) workspace might help developers manage this information. We first interviewed 13 experts in AI, XR, and related fields to identify opportunities and design challenges (Study 1). These interviews informed two prototypes explored with 12 professional developers (Study 2): four tried a version where the code faded to become invisible after it was looked at, and eight tried a version that replaced viewed code sections with short descriptions of their purpose. In the second prototype, head direction helped determine when to offer a change, while manual input let developers apply it and reopen the original code. A separate open-ended demonstration with seven software-engineering researchers provided complementary feedback. Participants' accounts suggested that descriptions could provide an overview of unfamiliar code and serve as reminders when revisiting it. However, difficulty comparing descriptions with source code, headset limitations, and controls that deviates from regular IDE routines could introduce additional effort. These exploratory findings identify requirements for attention support: make summaries easy to recognize and access, preserve straightforward inspection of the original code, and retain familiar navigation and editing practices. The study contributes to human-centered multimodal systems by showing how the coordination of input channels and AI-generated representations must account for users' attention, verification needs, and established ways of working.
Source:https://arxiv.org/abs/2611.24334PreviousTableControl: Gesture Interaction Through Surface VibrationsNextExploring Daily Emotional Well-Being through Personalized Data Analysis
Related posts
Re-Testing LLM-Based C-to-Rust Translations Against Their Original C
0arXiv cs.SE
Reliability is a user-specific property of large language models
0arXiv cs.SE
Safe Personal RSI: Evaluating and Repairing Personal-Agent Updates
0arXiv cs.SE
Option-GRPO: Hierarchical RL Against AI Slop in Code Generation for Unseen SDKs
0arXiv cs.SE
Systematic Mapping Protocol. Generative AI-Augmented Requirements Engineering: A Systematic Mapping Study of Practices, Specifications, and Reuse in Software Development
0arXiv cs.SE
RootSink: Reproducible C/C++ Crashes with Tested Repair Sites
0arXiv cs.SE