Reinforcement Learning Under Uncertainty: Expected Versus Unexpected Uncertainty and State Versus Reward Uncertainty

• DOI: 10.1007/s42113-022-00165-y
• Publication Date: 2023-03-20T16:03:13Z
• Authors: Adnane Ez-zizi; Simon Farrell; David Leslie; Gaurav Malhotra; Casimir J.H. Ludwig
• Keywords: 0303 health sciences; 03 medical and health sciences; 150

Abstract Two prominent types of uncertainty that have been studied extensively are expected and unexpected uncertainty. Studies suggest humans capable learning from reward under both when the source variability is reward. How do people learn environment’s state rewards themselves deterministic? does their compare with case uncertainty? The present study addressed these questions using behavioural experimentation computational modelling. Experiment 1 showed human subjects were generally able to use feedback successfully task rules uncertainty, detect a non-signalled reversal stimulus-response contingencies. 2, which combined all four uncertainties—expected versus uncertainty—highlighted key similarities differences in between uncertainties. We found performed significantly better condition, primarily because they explored less improved disambiguation. also show simple reinforcement mechanism ignores updates state-action value only identified accounted for data than Bayesian model keeps track belief states acts based on sampling past experiences. Our findings common supports reward-based