AI & ML
When History Overrides Intent: Relational Schema Lock-In in Human-LLM Interaction
arXiv cs.HC0 views
arXiv:2610.41367v1 Announce Type: new Abstract: Long-context interaction and persistent memory allow LLM assistants to develop history-grounded understandings of users. When users change direction, however, an understanding built through earlier exchanges may conflict with their current instruction. We introduce relational schema lock-in to describe cases in which an assistant grants that accumulated understanding authority to override the current instruction without user authorization for the intervention. Study 1 documents reports of AI assistants using prior interactions to question or alter requested actions. Study 2 reproduces the pattern across controlled histories and six models: models retained history-grounded positions but differed in whether and how those positions shaped task execution. In Study 3 with 360 participants, artifact substitution preserved task completion while reducing faithful enactment: 75.0% sent the AI's suspension letter, including 11 participants who reported cancellation as their final stated intent. Overt resistance reduced task completion while leaving sent artifacts faithful.
Source:https://arxiv.org/abs/2610.41367PreviousToward Low-Burden Image-Based Dietary Assessment Through Human-AI Collaborative CorrectionNextHybrid Computer Use: A Unified Programmatic, Semantic, and Visual Interface for General Agents
Related posts
DeepRitzSplit: A Variational Neural Operator Framework for Gradient Flows with Application to Dendritic Growth
0arXiv cs.NA
Quantitative Universal Approximation for Noisy Quantum Neural Networks
0arXiv cs.NA
Vertex interpolation by bivariate $C^1$ cubic splines: counterexamples, geometric stability, and indecomposable defects
0arXiv cs.NA
Adaptive refinement based on local approximation power of hierarchical spline spaces
0arXiv cs.NA
Discrete hypocoercive estimates for discontinuous Galerkin methods: application to the Vlasov-Poisson-Fokker-Planck system
0arXiv cs.NA
Novel technique based on L\'eja Points Approximation for Log-determinant Estimation of Large matrices
0arXiv cs.NA