AI & ML
GENSTRAT: Toward a Science of Strategic Reasoning in Large Language Models
arXiv cs.MA0 views
arXiv:2605.23238v2 Announce Type: replace-cross Abstract: Fixed game suites for evaluating strategic reasoning in large language models (LLMs) saturate and are hard to keep free of contamination. We introduce GENSTRAT, a system for evaluating the strategic capabilities of LLMs driven by procedural generation of two-player zero-sum imperfect-information betting games, allowing an evaluator to draw novel, fresh games on demand. Each game is characterized on a scale of five complexity axes (state space, temporal contingency, opponent dependence, private information, precision) and measured by Monte Carlo simulation. Using this generator, we construct a 50-game benchmark and hold a tournament of twelve frontier and open-weight models, gathering data from over 40,000 LLM vs. LLM matches. We further construct a composite complexity axis, and find that, on the eight hardest games, the gap between the best and worst model is 1.7x its value across the whole set of games. Ablations show that a higher thinking budget, disclosing the opponent's strategy, and higher reasoning matched to a game's specific demands (for private information and planning depth) were associated with higher margins for the LLM player.
Source:https://arxiv.org/abs/2605.23238PreviousSTRIATUM-CTF: A Protocol-Driven Agentic Framework for Autonomous CTF SolvingNextDenoising Multirobot Trajectories
Related posts
Meta Quest Promo Codes and Coupons for November 2026
0Wired
BullMQ in NestJS: Retries, Idempotency and Dead-Letter Queues
0dev.to
80.6% at Chores, 12% at Real Work: AI's Benchmark Week
0dev.to
Efficient estimation of regression models for recurrent events in presence of terminal events
0arXiv stat.CO
Local Whittle estimation for distributional time series: A regularization framework in Bayes Hilbert space
0arXiv stat.CO
Inventory and Assortment Optimization under General Choice: A Unified Approximation Framework
0arXiv stat.CO