A0529
Title: Equilibrium Control Learning under Cumulative Prospect Theory: Quantile Regression Meets Reinforcement Learning
Authors: Nixie Sapphira Lesmana - Nanyang Technological University (Singapore) [presenting]
Chi Seng Pun - Nanyang Technological University (Singapore)
Abstract: Cumulative prospect theory (CPT) optimization extends the well-known expected utility maximization by distorting probabilities and incorporating S-shaped utility functions to capture human decisions or preferences better. In this work, CPT optimization is investigated from the lens of reinforcement learning (RL). Firstly, noting that CPT can be rewritten as a function of quantiles, we review selected distributional RL techniques that deal with the prediction of quantile functions and highlight both their applicability and limitations for CPT Q-function prediction. Secondly, noting the time inconsistency introduced by probability distortions, it is shown that these RL algorithms aim to learn the subgame-perfect equilibrium (SPE) policy class. Drawing on these two perspectives, we propose a novel RL algorithm for learning SPE policy under CPT. We empirically evaluate our algorithm on several variant environments of casino gambling and demonstrate its plausible learning performance.
