AI & ML
Shielded Trajectory Sampling for Personalized Relational Safety in Conversational AI
arXiv cs.HC0 views
arXiv:2609.30780v2 Announce Type: replace Abstract: Anthropomorphic conversational systems can provide companionship and emotional support, but repeated, seemingly harmless interactions can encourage excessive dependence and reduced human contact. We formulate safety in user-AI relationships as a sequential decision problem, using conversational cues to track the relationship and selecting responses according to their anticipated effects over time. A conditional Generative Flow Network learns from sampled response plans to select replies that preserve the largest estimated mass of safe and useful continuations. Model-based checks screen plans during training and responses during interaction, requiring at least one sequence of future response strategies that meets a prescribed safety bound under every candidate user model, independently of the learned network. Strategies are selected directly from learned first-step probabilities, avoiding online plan sampling. We evaluate the approach in simulation against reactive safeguards, filters with and without memory, and alternative planners, using exact computation to validate the learned approximation. We also evaluate relational cue extraction from real user messages in WildChat. To our knowledge, this is the first approach to select conversational replies by anticipating their effects on the user-AI relationship while imposing an explicit model-based safety bound on future response sequences. Results show close agreement with exact planning, lower decision costs at longer horizons, and improved safety-utility trade-offs when protective responses are less effective. The approach balances immediate support with the longer-term effects of repeated interaction. The source code is available at https://github.com/benedettapicano/ANTHROPOMORPHIC_SAFETY_TRAJ.
Source:https://arxiv.org/abs/2609.30780PreviousHuman-AI Writing as Distributed Cognition: Reassessing Collaborative Writing Theories in the Age of LLMsNextAI Simulations Grounded in Rich Self-Reports Increase Accuracy and Reduce Distortion
Related posts
How OpenAI scales PostgreSQL to 800 million users
0dev.to Database
Spec-Driven PBT: Deriving Semantic Invariants from Code Specifications for Agentic Property-based Testing
0arXiv cs.SE
A Grey Literature Review of Graduate Students' Research Practices and Skills in the LLM Era
0arXiv cs.SE
ReBoT-MiCS: Reliability-Aware Bounding Time for Enhancing Timing Behavior of Embedded Mixed-Criticality Systems
0arXiv cs.SE
VoxelSBFL: Voxel-Level Spectrum-Based Failure Prediction for Vision-Language-Action Models
0arXiv cs.SE
Discontinuity-Aware Solvers for Vapor-Compression Cycles
0arXiv cs.NA