AI & ML
Is Randomness Necessary for Adaptive Data Analysis?
arXiv cs.CR0 views
arXiv:2607.07085v3 Announce Type: replace Abstract: The Adaptive Data Analysis (ADA) problem formalizes the challenge of preventing false discovery and overfitting when a dataset is repeatedly reused. Formally, our input is a dataset containing $n$ i.i.d. samples from an unknown distribution $\mathcal{P}$ over a domain $\mathcal{X}$, and our goal is to answer a sequence of $k$ adaptively chosen statistical queries with respect to $\mathcal{P}$. The main question is how many queries we can support (i.e., how large $k$ can be), primarily as a function of the number of samples $n$. This question has been intensively studied and is relatively well-understood for randomized mechanisms: there are computationally efficient mechanisms that support $k \approx n^2$ queries, and no computationally efficient mechanism can answer $k \gg n^2$ queries. In this paper, we address a fundamental question: is randomness necessary for ADA? Despite a decade of work on ADA, this question remains open. A folklore observation dating back to the initial works on ADA is that randomness is {\em not} necessary when the analyst is computationally bounded. Yet, the necessity of randomness against computationally unbounded analysts has remained elusive. Our main contribution resolves this gap in the information-theoretic setting. Perhaps surprisingly, we show that randomness is {\em strictly necessary} to answer a non-trivial number of adaptive queries: when the analyst is unbounded, any deterministic mechanism can be forced to fail after just $k = \tilde{O}(n)$ queries.
Source:https://arxiv.org/abs/2607.07085PreviousStatic Attribution of Android Residential Proxy Malware Using Graph KernelsNextLanguage Model Security Modules: Turning LLM Interpretability into Runtime Defense
Related posts
Getting cited by ChatGPT and Claude: what the research supports, and a free audit tool
0dev.to
Why Photoshot keeps the original color when you edit in black and white
0dev.to
ISTQB CTFL in 2027: is it still worth it? What changes, what does not
0dev.to

T3 Code with Codex and Claude Code: Verify the Provider Before You Run Parallel Work
0dev.to
The Songless Archive Is a Better Listening Log Than a Playlist
0dev.to
AI makes it easier to understand the whole project
0dev.to