Logarithmic Smoothing for Pessimistic Off-Policy Evaluation, Selection and Learning - École polytechnique Accéder directement au contenu
Pré-Publication, Document De Travail Année : 2024

Logarithmic Smoothing for Pessimistic Off-Policy Evaluation, Selection and Learning

Otmane Sakhi
  • Fonction : Auteur
  • PersonId : 1076039
Pierre Alquier
  • Fonction : Auteur
  • PersonId : 1396437

Résumé

This work investigates the offline formulation of the contextual bandit problem, where the goal is to leverage past interactions collected under a behavior policy to evaluate, select, and learn new, potentially better-performing, policies. Motivated by critical applications, we move beyond point estimators. Instead, we adopt the principle of pessimism where we construct upper bounds that assess a policy's worst-case performance, enabling us to confidently select and learn improved policies. Precisely, we introduce novel, fully empirical concentration bounds for a broad class of importance weighting risk estimators. These bounds are general enough to cover most existing estimators and pave the way for the development of new ones. In particular, our pursuit of the tightest bound within this class motivates a novel estimator (LS), that logarithmically smooths large importance weights. The bound for LS is provably tighter than all its competitors, and naturally results in improved policy selection and learning strategies. Extensive policy evaluation, selection, and learning experiments highlight the versatility and favorable performance of LS.
Fichier principal
Vignette du fichier
LS___NeuRIPS_Submission___Arxiv (1).pdf (613.36 Ko) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04629226 , version 1 (28-06-2024)

Identifiants

  • HAL Id : hal-04629226 , version 1

Citer

Otmane Sakhi, Imad Aouali, Pierre Alquier, Nicolas Chopin. Logarithmic Smoothing for Pessimistic Off-Policy Evaluation, Selection and Learning. 2024. ⟨hal-04629226⟩
0 Consultations
0 Téléchargements

Partager

Gmail Mastodon Facebook X LinkedIn More