US Patent 9536191 Reinforcement learning using confidence scores

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for reinforcement learning using confidence scores. One of the methods includes receiving a current observation; for each of multiple actions: determining a respective value function estimate that is an estimate of a return resulting from the agent performing the action in response to the current observation, determining a respective confidence score that is a measure of confidence that the respective value function estimate for the action is an accurate estimate of the return that will result from the agent performing the action in response to the current observation, adjusting the respective value function estimate for the action using the respective confidence score for the action to determine a respective adjusted value function estimate; and selecting an action to be performed by the agent in response to the current observation using the respective adjusted value function estimates.