Reinforcement Learning

Establishes the fundamental logarithmic regret lower bound; defines optimal exploration-exploitation tradeoff.

Introduces UCB1 with finite-time regret bounds; the practical workhorse algorithm still widely deployed.

Comprehensive survey unifying stochastic and adversarial bandits; essential theoretical reference.

Establishes lower bounds and optimal algorithms for best-arm identification; key for A/B testing.

Characterizes the sample complexity of best-arm identification in fixed-budget and fixed-confidence settings.

Introduces LinUCB; deployed at Yahoo! with 33M events; foundational industry paper.

Rigorous theoretical analysis providing SupLinUCB algorithm.

First near-optimal regret bounds for Thompson Sampling in contextual settings.

Efficient algorithm handling large policy classes for practical implementation.

Extends contextual bandits to neural network function approximation; bridges deep learning and bandit theory.

SquareCB algorithm achieves optimal regret using only regression oracles; practical for complex function classes.

Definitive survey covering theory, applications, and extensions; essential practitioner reference.

Demonstrates strong empirical performance; sparked renewed industry interest.

First proof of optimal O(√KT log T) regret bounds.

Theoretical foundations extending Thompson Sampling to general RL.

Generalizes Thompson Sampling using information ratio; provides tighter regret bounds for structured problems.

Introduces doubly robust estimator combining IPS and direct method; robust to model misspecification.

Principled framework for learning from logged data; widely used in industry.

Addresses high variance in IPS with self-normalization for real systems.

Minimax optimal bounds providing theoretical foundation for modern OPE.

High-confidence bounds for off-policy evaluation; safety-critical applications in healthcare and education.

Achieves minimax optimal rates for OPE in tabular MDPs; foundational theoretical result.

Extends OPE to slate/list recommendations; critical for search and recommendation systems.

Comprehensive tutorial defining the field's challenges and research directions.

Identifies extrapolation error as core challenge; introduces Batch-Constrained Q-learning.

Learns conservative Q-functions lower-bounding true value; SOTA on D4RL benchmarks.

Standard benchmark datasets enabling reproducible offline RL research.

Frames offline RL as sequence modeling using transformers; avoids bootstrapping entirely.

Simple algorithm avoiding explicit policy constraint; strong performance on D4RL benchmarks.

MDP framework for RTB with neural network value approximation for budget-constrained bidding.

DRL for sponsored search deployed at Alibaba scale; handles non-stationary environments.

Model-free RL for budget-constrained bidding with practical reward function design.

Thompson Sampling at web-scale in Bing; demonstrates industrial viability of Bayesian methods.

Regret bounds for bidding with budget constraints; foundational for pacing algorithms.

Neural networks learn near-optimal auction mechanisms; connects ML and mechanism design.

Learning across related auctions; critical for ad platforms with correlated contexts.

Combines ML feature learning with dynamic pricing; optimal regret in high dimensions.

Deployed at YouTube; addresses large action spaces in slate recommendation with off-policy correction.

Decomposes slate Q-values for tractable optimization; deployed at Google.

DRL for whole-page recommendations considering item interactions and user browsing patterns.

Learns user simulator for offline RL training; addresses exploration challenges in recommendations.

Optimizes long-term user retention metrics beyond immediate clicks; deployed at JD.com.

First practical algorithm for RL with safety constraints; foundational for safe RL.

Provides formal safety guarantees using Lyapunov functions; critical for robotics applications.

OpenAI Safety Gym benchmark suite; standard evaluation for safe RL algorithms.

Posterior sampling for deep RL exploration; efficient exploration without explicit uncertainty.

Pseudo-counts for exploration in high-dimensional spaces; breakthrough for sparse-reward problems.

Intrinsic curiosity module using prediction error as exploration bonus; widely influential approach.

SimPLe achieves 100x sample efficiency on Atari using learned world models.

PSRO framework unifying game theory and deep RL; foundational for competitive multi-agent systems.

LOLA accounts for opponent adaptation during learning; key insight for non-stationary multi-agent settings.

AlphaStar achieves grandmaster level; landmark result in complex multi-agent real-time strategy.

Pluribus beats top humans in 6-player poker; breakthrough in imperfect-information games.