AI & ML
IQS-BO: In-Context Query Selection for Bayesian Optimisation
arXiv stat.ML0 views
arXiv:2610.01269v2 Announce Type: replace-cross Abstract: Bayesian Optimisation (BO) is a powerful framework for optimising expensive black-box functions, but typically requires refitting a surrogate and maximising an acquisition function at every evaluation. In-context approaches based on Prior-data Fitted Networks (PFNs) amortise part of this cost by pre-training transformers on functions drawn from synthetic priors. PFNS4BO amortises the surrogate but still relies on a numerically maximised acquisition function, while FIBO performs BO fully in-context by sampling optimiser locations from a learned density, which fixes the decision rule and admits no surrogate. Learned acquisition functions score a finite candidate set with a network, but, lacking a label for the query, learn the score by reinforcement learning. We propose IQS-BO, a PFN that learns the query decision by supervised learning on synthetic priors. In a single forward pass, IQS-BO predicts the probability that each candidate is the best in the set, and we show that training minimises its expected KL divergence to the posterior probability of this event. The model can be pre-trained without a surrogate for fully in-context BO, or take the predictions of a fixed probabilistic surrogate as additional input, amortising only the decision step. Our method proposes queries at a fraction of the cost of acquisition-based methods, while matching or outperforming BO with Gaussian processes (GPs) and available in-context methods in sequential and batch optimisation, in low and high dimensions. Finally, we propose a mixture prior for pre-training PFNs which combines samples from GPs with functions exhibiting warped inputs, narrow optima, or plateaus that are poorly modelled by stationary GP kernels. We show that pre-training on this prior can improve performance and help the network generalise to a broader range of optimisation problems.
Source:https://arxiv.org/abs/2610.01269PreviousSwiss-Knife: A Framework for Reconfigurable Externalised Multi-Objective Alignment at Decode TimeNextPrescriptive Learning of Linear Policies with Feasibility Guarantees for Binary Decision-Making
Related posts
Ruby2ßhell: Pre-authentication RCE on stock Rails
0Hacker News
Generalizing self-normalized importance sampling with couplings
0arXiv stat.CO
Bayesian Computation of Copula-based Multivariate Models with an Application to Doping Detection
0arXiv stat.CO
WAIVES: Weak-form Asymptotically-unbiased Instrumental-Variable Estimation for SDEs
0arXiv stat.CO
Treat Your Honeypot Analysis Like Code: Git, Pull Requests and a Human Gate for AI [Guest Diary], (Wed, Nov 18th)
0SANS ISC
ISC Stormcast For Thursday, November 19th, 2026 https://isc.sans.edu/podcastdetail/10186, (Thu, Nov 19th)
0SANS ISC