AI & ML
Beyond the Landing Page: Website Blocking, Tracker Visibility, and Human Alignment in Automated Web Crawls
arXiv cs.CR0 views
arXiv:2611.06197v1 Announce Type: new Abstract: Web tracking studies largely rely on automated crawlers, yet their measurements can differ from what people encounter while browsing. Conventional crawlers follow predefined navigation rules, whereas LLM-driven agents decide how to navigate and interact with websites, potentially changing the tracking they encounter. We compare agentic and non-agentic crawlers to examine how websites respond to them, what tracking they record, and how closely their observations match human browsing. Across 2,000 popular websites, we evaluate different navigation strategies, automation frameworks, and stealth configurations, alongside browsing data from 26 participants. We find that website denials increase beyond landing pages for standard browser automation, while a detection-resistant automation framework largely prevents this escalation without increasing recorded tracking. On the same URLs, agentic crawlers record tracking activity from 1.5-1.7 times as many third-party organizations, particularly on consent-managed pages. Agentic crawlers recover 82-87% of tracker organizations observed by participants, compared with 58-68% for non-agentic crawlers, yet roughly half of their own observations are absent from human participant visits. These findings show that avoiding website interventions, increasing tracking visibility, and achieving closer agreement with human observations do not necessarily go hand in hand, highlighting the need to select and validate crawlers according to the measurement objective.
Source:https://arxiv.org/abs/2611.06197PreviousNetwork Intrusion Forensic Detection System with Explainable Artificial Intelligence and Digital Chain of CustodyNextExplainable Anomaly Detection in 5G Networks through Continuous Time Graphs
Related posts
Uniform Race: Parameter-Free Approximate Rejection Sampling
0arXiv stat.ML
Efficient Robust Learning at the Information-Theoretic Limit
0arXiv stat.ML
Algorithmic Principles For Multiclass Learning Are Hard To Come By: Limits of Regularization and Proper Learning
0arXiv stat.ML
More Expressive Feedforward Layers: Part I. Token-Adaptive Mixing of Activations
0arXiv stat.ML
Fr\'echet Regression on the Bures-Wasserstein Manifold
0arXiv stat.ML
Decoupling Distance and Networks: Hybrid Graph Attention-Geostatistical Methods for Spatio-temporal Risk Mapping
0arXiv stat.ML