Computational models of reinforcement learning: the role of dopamine as a reward signal

Reinforcement Learning: An Introduction

2005 • 27,230 citations

Reinforcement Learning: An Introduction

1998 • 23,138 citations

A Neural Substrate of Prediction and Reward

1997 • 8,991 citations

Reinforcement Learning: A Survey

1996 • 8,026 citations

Synaptic Modifications in Cultured Hippocampal Neurons: Dependence on Spike Timing, Synaptic Strength, and Postsynaptic Cell Type

1998 • 4,476 citations

Predictive Reward Signal of Dopamine Neurons

1998 • 4,446 citations

What is the role of dopamine in reward: hedonic impact, reward learning, or incentive salience?

1998 • 3,973 citations

Untitled

1988 • 3,901 citations

The Handbook of brain theory and neural networks

1996 • 3,654 citations

Neural systems of reinforcement for drug addiction: from actions to habits to compulsion

2005 • 3,624 citations

Untitled

1992 • 3,506 citations

Theoretical Neuroscience: Computational and Mathematical Modeling of Neural Systems

2001 • 3,213 citations

Dopamine, learning and motivation

2004 • 3,146 citations

Neuronlike adaptive elements that can solve difficult learning control problems

1983 • 3,122 citations

Getting Formal with Dopamine and Reward

2002 • 2,558 citations

Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control

2005 • 2,464 citations

The debate over dopamine’s role in reward: the case for incentive salience

2006 • 2,256 citations

Drugs of abuse: anatomy, pharmacology and function of reward pathways

1992 • 2,091 citations

Dissociable Roles of Ventral and Dorsal Striatum in Instrumental Conditioning

2004 • 2,078 citations

Emotion and motivation: the role of the amygdala, ventral striatum, and prefrontal cortex

2002 • 2,076 citations

A framework for mesencephalic dopamine systems based on predictive Hebbian learning

1996 • 2,003 citations

By Carrot or by Stick: Cognitive Reinforcement Learning in Parkinsonism

2004 • 1,951 citations

Brain Dopamine and Reward

1989 • 1,899 citations

Phasic versus tonic dopamine release and the modulation of dopamine system responsivity: A hypothesis for the etiology of schizophrenia

1991 • 1,819 citations

A Neostriatal Habit Learning System in Humans

1996 • 1,789 citations

Context and Behavioral Processes in Extinction: Table 1.

2004 • 1,677 citations

Neural Mechanisms of Extinction Learning and Retrieval

2007 • 1,555 citations

The incentive sensitization theory of addiction: some current issues

2008 • 1,555 citations

Temporal Difference Models and Reward-Related Learning in the Human Brain

2003 • 1,488 citations

The Neuroscience of Natural Rewards: Relevance to Addictive Drugs

2002 • 1,439 citations

The role of nucleus accumbens dopamine in motivated behavior: a unifying interpretation with special reference to reward-seeking

1999 • 1,420 citations

Two types of dopamine neuron distinctly convey positive and negative motivational signals

2009 • 1,331 citations

Conditioning and Associative Learning

1984 • 1,327 citations

Midbrain Dopamine Neurons Encode a Quantitative Reward Prediction Error Signal

2005 • 1,279 citations

Addiction, Dopamine, and the Molecular Mechanisms of Memory

2000 • 1,186 citations

Neurobehavioural mechanisms of reward and motivation

1996 • 1,177 citations

Models of Information Processing in the Basal Ganglia

1994 • 1,152 citations

Dichotomous Dopaminergic Control of Striatal Synaptic Plasticity

2008 • 1,152 citations

D1 and D2 dopamine-receptor modulation of striatal glutamatergic signaling in striatal medium spiny neurons

2007 • 1,125 citations

Differential Effect of Stress on In Vivo Dopamine Release in Striatum, Nucleus Accumbens, and Medial Frontal Cortex

1989 • 1,107 citations

Subsecond dopamine release promotes cocaine seeking

2003 • 1,086 citations

Neurobiology of addiction

1996 • 1,082 citations

Neural Responses during Anticipation of a Primary Taste Reward

2002 • 1,082 citations

Afferent modulation of dopamine neuron firing differentially regulates tonic and phasic dopamine transmission

2003 • 1,079 citations

Dopamine responses comply with basic assumptions of formal learning theory

2001 • 1,034 citations

Inverted-U dopamine D1 receptor actions on prefrontal neurons engaged in working memory

2007 • 991 citations

Responses of monkey dopamine neurons during learning of behavioral reactions

1992 • 986 citations

Computational roles for dopamine in behavioural control

2004 • 957 citations

Nucleus accumbens shell and core dopamine: differential role in behavior and addiction

2002 • 957 citations

Dynamic Dopamine Modulation in the Basal Ganglia: A Neurocomputational Account of Cognitive Deficits in Medicated and Nonmedicated Parkinsonism

2005 • 923 citations

Dopaminergic Modulation of Neuronal Excitability in the Striatum and Nucleus Accumbens

2000 • 916 citations

A cellular mechanism of reward-related learning

2001 • 863 citations

Amygdala Inhibitory Circuits and the Control of Fear Memory

2009 • 841 citations

Preferential activation of midbrain dopamine neurons by appetitive rather than aversive stimuli

1996 • 826 citations

Reward, Motivation, and Reinforcement Learning

2002 • 816 citations

Dopamine-mediated regulation of corticostriatal synaptic plasticity

2007 • 802 citations

Addictive Drugs and Brain Stimulation Reward

1996 • 777 citations

A Network Model of Catecholamine Effects: Gain, Signal-to-Noise Ratio, and Behavior

1990 • 767 citations

Temporal credit assignment in reinforcement learning

1984 • 746 citations

Neuroeconomics: The Consilience of Brain and Decision

2004 • 735 citations

Associative Processes in Addiction and Reward The Role of Amygdala‐Ventral Striatal Subsystems

1999 • 713 citations

Dopamine Operates as a Subsecond Modulator of Food Seeking

2004 • 686 citations

Genetic triple dissociation reveals multiple roles for dopamine in reinforcement learning

2007 • 685 citations

Long-term depression in the nucleus accumbens: a neural correlate of behavioral sensitization to cocaine

2001 • 622 citations

Temporal difference models describe higher-order learning in humans

2004 • 608 citations

Amygdala circuitry in attentional and representational processes

1999 • 606 citations

Orbitofrontal Cortex and Representation of Incentive Value in Associative Learning

1999 • 606 citations

Solving the Distal Reward Problem through Linkage of STDP and Dopamine Signaling

2007 • 603 citations

Neural Encoding in Orbitofrontal Cortex and Basolateral Amygdala during Olfactory Discrimination Learning

1999 • 602 citations

Ventral Tegmental Area Neurons in Learned Appetitive Behavior and Positive Reinforcement

2007 • 601 citations

Dopamine neurons encode the better option in rats deciding between differently delayed or sized rewards

2007 • 589 citations

Coincident but Distinct Messages of Midbrain Dopamine and Striatal Tonically Active Neurons

2004 • 573 citations

Dopaminergic modulation of limbic and cortical drive of nucleus accumbens in goal-directed behavior

2005 • 567 citations

Memory consolidation and the amygdala: a systems perspective

2002 • 562 citations

Neurotoxic Lesions of Basolateral, But Not Central, Amygdala Interfere with Pavlovian Second-Order Conditioning and Reinforcer Devaluation Effects

1996 • 562 citations

Decision theory, reinforcement learning, and the brain

2008 • 546 citations

Importance of unpredictability for reward responses in primate dopamine neurons

1994 • 543 citations

Dopamine-dependent plasticity of corticostriatal synapses

2002 • 509 citations

Midbrain dopamine neurons encode decisions for future action

2006 • 497 citations

Time-Derivative Models of Pavlovian Reinforcement

1990 • 494 citations

Prefrontal and striatal dopaminergic genes predict individual differences in exploration and exploitation

2009 • 483 citations

Actor–critic models of the basal ganglia: new anatomical and computational perspectives

2002 • 475 citations

Multiple Model-Based Reinforcement Learning

2002 • 468 citations

Dopamine Cells Respond to Predicted Events during Classical Conditioning: Evidence for Eligibility Traces in the Reward-Learning Network

2005 • 448 citations

Influence of Reward Delays on Responses of Dopamine Neurons

2008 • 431 citations

NMDA Receptors and L-Type Voltage-Gated Calcium Channels Contribute to Long-Term Potentiation and Different Components of Fear Memory Formation in the Lateral Amygdala

2002 • 431 citations

Adaptive Critics and the Basal Ganglia

1994 • 425 citations

Dopamine gates LTP induction in lateral amygdala by suppressing feedforward inhibition

2003 • 410 citations

Reward‐guided learning beyond dopamine in the nucleus accumbens: the integrative functions of cortico‐basal ganglia networks

2008 • 407 citations

Markov decision processes

1989 • 404 citations

Dopamine Receptor Activation Is Required for Corticostriatal Spike-Timing-Dependent Plasticity

2008 • 396 citations

Précis of The brain and emotion

2000 • 393 citations

Dopamine D-1/D-5 Receptor Activation Is Required for Long-Term Potentiation in the Rat Neostriatum In Vitro

2001 • 380 citations

D1/D5 Dopamine Receptor Activation Increases the Magnitude of Early Long-Term Potentiation at CA1 Hippocampal Synapses

1996 • 374 citations

Reconciling reinforcement learning models with behavioral extinction and renewal: Implications for addiction, relapse, and problem gambling.

2007 • 359 citations

Dopamine and N-Methyl-D- Aspartate Receptor Interactions in the Neostriatum

1998 • 358 citations

Dopamine neurons of the monkey midbrain: contingencies of responses to active touch during self-initiated arm movements

1990 • 350 citations

Reinforcement Learning Through Modulation of Spike-Timing-Dependent Synaptic Plasticity

2007 • 334 citations

Dopaminergic antagonists prevent long-term maintenance of posttetanic LTP in the CA1 region of rat hippocampal slices

1990 • 333 citations

Postsynaptic Induction and PKA-Dependent Expression of LTP in the Lateral Amygdala

1998 • 332 citations

The effect of dopaminergic D1 receptor blockade during tetanization on the expression of long-term potentiation in the rat CA1 region in vitro

1991 • 311 citations

Correlated Coding of Motivation and Outcome of Decision by Dopamine Neurons

2003 • 304 citations

Dopamine Neurons Can Represent Context-Dependent Prediction Error

2004 • 293 citations

Reward-Related Neuronal Activity During Go-Nogo Task Performance in Primate Orbitofrontal Cortex

2000 • 286 citations

Forebrain substrates of reward and motivation

2005 • 282 citations

Frequency of Dopamine Concentration Transients Increases in Dorsal and Ventral Striatum of Male Rats during Introduction of Conspecifics

2002 • 279 citations

Space, time and dopamine

2006 • 276 citations

Localization of drug reward mechanisms by intracranial injections

1992 • 274 citations

Dopaminergic Mechanisms in Actions and Habits: Figure 1.

2007 • 267 citations

Presynaptic induction of heterosynaptic associative plasticity in the mammalian brain

2003 • 257 citations

A Learning Theory for Reward-Modulated Spike-Timing-Dependent Plasticity with Application to Biofeedback

2008 • 253 citations

Distinct Roles of D₁and D₅Dopamine Receptors in Motor Activity and Striatal Synaptic Plasticity

2003 • 238 citations

Actor-Critic--Type Learning Algorithms for Markov Decision Processes

1999 • 234 citations

Neurocomputational models of basal ganglia function in learning, memory and choice

2008 • 226 citations

Time-limited modulation of appetitive Pavlovian memory by D1 and NMDA receptors in the nucleus accumbens

2005 • 217 citations

Simultaneous LTP of non-NMDA- and LTD of NMDA-receptor-mediated responses in the nucleus accumbens

1994 • 205 citations

Temporal Sequence Learning, Prediction, and Control: A Review of Different Models and Their Relation to Biological Mechanisms

2005 • 205 citations

Evidence for coexistence of GABA and dopamine in neurons of the rat olfactory bulb

1987 • 205 citations

Impulsive choice and response in dopamine agonist-related impulse control behaviors

2009 • 199 citations

Statistics of Midbrain Dopamine Neuron Spike Trains in the Awake Primate

2007 • 196 citations

Parallel Cortico-Basal Ganglia Mechanisms for Acquisition and Execution of Visuomotor Sequences—A Computational Approach

2001 • 195 citations

Dopaminergic Modulation of Long-term Synaptic Plasticity in Rat Prefrontal Neurons

2003 • 190 citations

Integrity of the mesocortical dopaminergic system is necessary for complete expression of in vivo hippocampal–prefrontal cortex long-term potentiation

1999 • 190 citations

Activity of dopamine neurons in the behaving primate

1992 • 178 citations

Dynamic Gain Control of Dopamine Delivery in Freely Moving Animals

2004 • 177 citations

Modifications of Reward Expectation-Related Neuronal Activity During Learning in Primate Orbitofrontal Cortex

2000 • 176 citations

Synaptic Plasticity in an In Vitro Slice Preparation of the Rat Nucleus Accumbens

1993 • 174 citations

Representation and Timing in Theories of the Dopamine System

2006 • 173 citations

Differential reactivity of dopaminergic neurons in the nucleus accumbens in response to different behavioral situations. An in vivo voltammetric study in free moving rats

1986 • 169 citations

D1/D5 Dopamine Receptors Inhibit Depotentiation at CA1 Synapses via cAMP-Dependent Mechanism

1998 • 167 citations

Dopamine Modulates Persistent Synaptic Activity and Enhances the Signal-to-Noise Ratio in the Prefrontal Cortex

2009 • 157 citations

Anatomy of the dopaminergic neuron systems in the rat brain.

1978 • 155 citations

The ability of the mesocortical dopamine system to operate in distinct temporal modes

2006 • 153 citations

Learning in neural networks by reinforcement of irregular spiking

2004 • 148 citations

Solving the distal reward problem through linkage of STDP and dopamine signaling

2007 • 144 citations

A Role for Dopamine in Temporal Decision Making and Reward Maximization in Parkinsonism

2008 • 141 citations

New insights on the subcortical representation of reward

2008 • 138 citations

The performance on learning tasks of patients in the early stages of Parkinson's disease

1989 • 136 citations

Activity-Dependent Synaptic Plasticity in the Central Nucleus of the Amygdala

2005 • 131 citations

Dopaminergic signaling in dendritic spines

2008 • 129 citations

Modeling functions of striatal dopamine modulation in learning and planning

2001 • 122 citations

Role of P/Q-Ca²⁺ Channels in Metabotropic Glutamate Receptor 2/3-Dependent Presynaptic Long-Term Depression at Nucleus Accumbens Synapses

2002 • 119 citations

Synaptic plasticity in the basal ganglia

2008 • 116 citations

Silencing the critics: understanding the effects of cocaine sensitization on dorsolateral and ventral striatum in the context of an actor/critic model

2008 • 115 citations

‘Passive stabilization’ of striatal extracellular dopamine across the lesion spectrum encompassing the presymptomatic phase of Parkinson's disease: a voltammetric study in the 6‐OHDA‐lesioned rat

2003 • 108 citations

Heterosynaptic Long-Term Potentiation of Inhibitory Interneurons in the Lateral Amygdala

2004 • 107 citations

Increased dopamine release in vivo in nucleus accumbens and caudate nucleus of the rat during drinking: A microdialysis study

1992 • 107 citations

A Spiking Neural Network Model of an Actor-Critic Learning Agent

2008 • 96 citations

Co-Transmission of Dopamine and GABA in Periglomerular Cells

2008 • 92 citations

Repeated Cocaine Administration Promotes Long-Term Potentiation Induction in Rat Medial Prefrontal Cortex

2006 • 84 citations

AMPA/kainate, NMDA, and dopamine D1 receptor function in the nucleus accumbens core: A context-limited role in the encoding and consolidation of instrumental memory

2005 • 84 citations

Extracellular dopamine dynamics in rat caudate–putamen during experimenter-delivered and intracranial self-stimulation

2000 • 79 citations

Neuroeconomics: Opening the Gray Box

2008 • 78 citations

Understanding Consolidation through the Architecture of Memories

2006 • 76 citations

Dopamine neuron glutamate cotransmission: frequency-dependent modulation in the mesoventromedial projection

2009 • 72 citations

Definitions of state variables and state space for brain-computer interface

2006 • 71 citations

L-type voltage-dependent Ca2+ channels mediate expression of presynaptic LTP in amygdala

2009 • 70 citations

Repeated exposure to amphetamine disrupts dopaminergic modulation of excitatory synaptic plasticity and neurotransmission in nucleus accumbens

2003 • 69 citations

Low-frequency stimulation induces a pathway-specific late phase of LTP in the amygdala that is mediated by PKA and dependent on protein synthesis

2007 • 66 citations

Reinforcement Learning, Spike-Time-Dependent Plasticity, and the BCM Rule

2007 • 64 citations

Interactions of glutamate and dopamine in a computational model of the striatum

1995 • 63 citations

Inter-module credit assignment in modular reinforcement learning

2003 • 59 citations

Rat Orbitofrontal Cortex Separately Encodes Response and Outcome Information during Performance of Goal-Directed Behavior

2008 • 57 citations

Dopamine release from pharmacologically distinct storage pools in rat striatum following stimulation at frequency of neuronal bursting

2000 • 56 citations

Background Dopamine Concentration Dependently Facilitates Long-term Potentiation in Rat Prefrontal Cortex through Postsynaptic Activation of Extracellular Signal-Regulated Kinases

2009 • 54 citations

Systematic presence of GABA-immunoreactivity in the tubero-infundibular and tubero-hypophyseal dopaminergic axonal systems: an ultrastructural immunogold study on several mammals

1991 • 51 citations

Domperidone, an inhibitor of the D2-receptor, blocks a late phase of an electrically induced long-term potentiation in the CA1-region in rats.

1989 • 49 citations

A neurocomputational account of catalepsy sensitization induced by D2 receptor blockade in rats: context dependency, extinction, and renewal

2009 • 45 citations

Facilitation of conditioned taste aversion learning by systemic amphetamine: role of nucleus accumbens shell dopamine D₁ receptors

2003 • 44 citations

Functional Heterogeneity at Dopamine Release Sites

2009 • 34 citations

Spiking neurons, dopamine, and plasticity: Timing is everything, but concentration also matters

2007 • 29 citations

Neural Encoding in the Orbitofrontal Cortex Related to Goal‐Directed Behavior

2007 • 28 citations

Multiple model-based reinforcement learning explains dopamine neuronal activity

2007 • 19 citations

A Roles of Dopamine

2002 • 16 citations

Parallel reinforcement learning for weighted multi-criteria model with adaptive margin

2008 • 13 citations

An implementation of reinforcement learning based on spike timing dependent plasticity

2008 • 10 citations

Postsynaptic modulation of AMPA- and NMDA-receptor currents by Group III metabotropic glutamate receptors in rat nucleus accumbens

2003 • 9 citations