Cortical and neuromodulatory mechanisms underlying reward identity-based reinforcement learning

Adaptive behavior requires accurate predictions of relevant rewards that can be expected in the future. Computational models posit that such predictions can be learned and updated by prediction error signals, computed as the difference between expected and experienced rewards. Neurobiologically, such prediction errors are reflected in the phasic firing of dopaminergic midbrain neurons, which serve as a “teaching signal” for downstream representations of reward expectations. Aberrant functionality within this circuit has a profound negative impact on motivated behavior, and is associated with a number of neuropsychological and neurological conditions. Schizophrenia, in particular, is associated with abnormal levels of dopamine in the brain, and dysregulated firing of midbrain dopamine neurons has been linked to both positive and negative symptoms of the disease. Accordingly, many studies have sought to characterize how prediction error-related processing is disrupted in patients with schizophrenia. However, the conceptual framework of these studies focuses on the role of dopaminergic prediction error signals in updating expectations about reward value, and thus can only inform model-free learning. More recently, work in animals and humans has shown that this same dopaminergic midbrain region responds to violations in the sensory properties, or identity, of expected rewards, thereby contributing to model-based learning. We have further shown that identity prediction errors are directly related to updated identity expectations in downstream orbitofrontal cortex (OFC) in healthy human subjects. Given the link between dopamine and schizophrenia, this raises the possibility that some symptoms of schizophrenia are related to abnormal reward identity prediction error signaling. To generate testable hypotheses along these lines within clinical populations, a more definitive understanding of the role of OFC and dopamine in model-based identity learning in healthy subjects is needed. The overarching goal of the current proposal is to test for causal roles for OFC and dopamine in identity prediction error signaling in humans. We aim to first utilize non-invasive transcranial magnetic stimulation (TMS) to temporarily modulate OFC function in the context of an established identity/value reversal learning paradigm. We predict that OFC-targeted stimulation will attenuate model-based identity learning, while leaving model-free value learning intact. In a subsequent experiment, we aim to implement a double-blind, within-subjects, placebo-controlled pharmacological study in which healthy human subjects receive low doses of either a dopamine receptor antagonist (haloperidol), agonist (L-DOPA), or placebo prior to performing an identity-based reversal learning task during functional magnetic resonance imaging (fMRI). If identity prediction errors are indeed dopamine dependent, we expect bidirectional effects of the drugs on behavioral learning rates and associated identity error signals in the midbrain. Taken together, these experiments could provide causal evidence for a role of dopamine in model-based learning processes. Moreover, such findings could provide novel neurocomputational insights into the genesis of positive symptoms in schizophrenia, such as delusions.