dopamine
Latest
Understanding reward-guided learning using large-scale datasets
Understanding the neural mechanisms of reward-guided learning is a long-standing goal of computational neuroscience. Recent methodological innovations enable us to collect ever larger neural and behavioral datasets. This presents opportunities to achieve greater understanding of learning in the brain at scale, as well as methodological challenges. In the first part of the talk, I will discuss our recent insights into the mechanisms by which zebra finch songbirds learn to sing. Dopamine has been long thought to guide reward-based trial-and-error learning by encoding reward prediction errors. However, it is unknown whether the learning of natural behaviours, such as developmental vocal learning, occurs through dopamine-based reinforcement. Longitudinal recordings of dopamine and bird songs reveal that dopamine activity is indeed consistent with encoding a reward prediction error during naturalistic learning. In the second part of the talk, I will talk about recent work we are doing at DeepMind to develop tools for automatically discovering interpretable models of behavior directly from animal choice data. Our method, dubbed CogFunSearch, uses LLMs within an evolutionary search process in order to "discover" novel models in the form of Python programs that excel at accurately predicting animal behavior during reward-guided learning. The discovered programs reveal novel patterns of learning and choice behavior that update our understanding of how the brain solves reinforcement learning problems.
Understanding reward-guided learning using large-scale datasets
Understanding the neural mechanisms of reward-guided learning is a long-standing goal of computational neuroscience. Recent methodological innovations enable us to collect ever larger neural and behavioral datasets. This presents opportunities to achieve greater understanding of learning in the brain at scale, as well as methodological challenges. In the first part of the talk, I will discuss our recent insights into the mechanisms by which zebra finch songbirds learn to sing. Dopamine has been long thought to guide reward-based trial-and-error learning by encoding reward prediction errors. However, it is unknown whether the learning of natural behaviours, such as developmental vocal learning, occurs through dopamine-based reinforcement. Longitudinal recordings of dopamine and bird songs reveal that dopamine activity is indeed consistent with encoding a reward prediction error during naturalistic learning. In the second part of the talk, I will talk about recent work we are doing at DeepMind to develop tools for automatically discovering interpretable models of behavior directly from animal choice data. Our method, dubbed CogFunSearch, uses LLMs within an evolutionary search process in order to "discover" novel models in the form of Python programs that excel at accurately predicting animal behavior during reward-guided learning. The discovered programs reveal novel patterns of learning and choice behavior that update our understanding of how the brain solves reinforcement learning problems.
Dopaminergic Network Dynamics
The Neurobiology of the Addicted Brain
SWEBAGS conference 2024: Shared network mechanisms of dopamine and deep brain stimulation for the treatment of Parkinson’s disease: From modulation of oscillatory cortex – basal ganglia communication to intelligent clinical brain computer interfaces
Learning and Memory
This webinar on learning and memory features three experts—Nicolas Brunel, Ashok Litwin-Kumar, and Julijana Gjorgieva—who present theoretical and computational approaches to understanding how neural circuits acquire and store information across different scales. Brunel discusses calcium-based plasticity and how standard “Hebbian-like” plasticity rules inferred from in vitro or in vivo datasets constrain synaptic dynamics, aligning with classical observations (e.g., STDP) and explaining how synaptic connectivity shapes memory. Litwin-Kumar explores insights from the fruit fly connectome, emphasizing how the mushroom body—a key site for associative learning—implements a high-dimensional, random representation of sensory features. Convergent dopaminergic inputs gate plasticity, reflecting a high-dimensional “critic” that refines behavior. Feedback loops within the mushroom body further reveal sophisticated interactions between learning signals and action selection. Gjorgieva examines how activity-dependent plasticity rules shape circuitry from the subcellular (e.g., synaptic clustering on dendrites) to the cortical network level. She demonstrates how spontaneous activity during development, Hebbian competition, and inhibitory-excitatory balance collectively establish connectivity motifs responsible for key computations such as response normalization.
Contribution of computational models of reinforcement learning to neurosciences/ computational modeling, reward, learning, decision-making, conditioning, navigation, dopamine, basal ganglia, prefrontal cortex, hippocampus
The cell biology of Parkinson’s disease: a role for primary cilia and synaptic vesicle pleomorphism in dopaminergic neurons
Personalized medicine and predictive health and wellness: Adding the chemical component
Wearable sensors that detect and quantify biomarkers in retrievable biofluids (e.g., interstitial fluid, sweat, tears) provide information on human dynamic physiological and psychological states. This information can transform health and wellness by providing actionable feedback. Due to outdated and insufficiently sensitive technologies, current on-body sensing systems have capabilities limited to pH, and a few high-concentration electrolytes, metabolites, and nutrients. As such, wearable sensing systems cannot detect key low-concentration biomarkers indicative of stress, inflammation, metabolic, and reproductive status. We are revolutionizing sensing. Our electronic biosensors detect virtually any signaling molecule or metabolite at ultra-low levels. We have monitored serotonin, dopamine, cortisol, phenylalanine, estradiol, progesterone, and glucose in blood, sweat, interstitial fluid, and tears. The sensors are based on modern nanoscale semiconductor transistors that are straightforwardly scalable for manufacturing. We are developing sensors for >40 biomarkers for personalized continuous monitoring (e.g., smartwatch, wearable patch) that will provide feedback for treating chronic health conditions (e.g., perimenopause, stress disorders, phenylketonuria). Moreover, our sensors will enable female fertility monitoring and the adoption of more healthy lifestyles to prevent disease and improve physical and cognitive performance.
Dopamine Acetylcholine interactions
Closed-loop deep brain stimulation as a neuroprosthetic of dopaminergic circuits – Current evidence and future opportunities; Spatial filtering to enhance signal processing in invasive neurophysiology
On Thursday February 15th, we will host Victoria Peterson and Julian Neumann. Victoria will tell us about “Spatial filtering to enhance signal processing in invasive neurophysiology”. Besides his scientific presentation on “Closed-loop deep brain stimulation as a neuroprosthetic of dopaminergic circuits – Current evidence and future opportunities”, Julian will give us a glimpse at the person behind the science. The talks will be followed by a shared discussion. Note: The talks will exceptionally be held at 10 ET / 4PM CET. You can register via talks.stimulatingbrains.org to receive the (free) Zoom link!
Dopamine, transcriptome, and new players in the reward game
Neuromodulation of subjective experience
Many psychoactive substances are used with the aim of altering experience, e.g. as analgesics, antidepressants or antipsychotics. These drugs act on specific receptor systems in the brain, including the opioid, serotonergic and dopaminergic systems. In this talk, I will summarise human drug studies targeting opioid receptors and their role for human experience, with focus on the experience of pain, stress, mood, and social connection. Opioids are only indicated for analgesia, due to their potential to cause addiction. When these regulations occurred, other known effects were relegated to side effects. This may be the cause of the prevalent myth that opioids are the most potent painkillers, despite evidence from head-to-head trials, Cochrane reviews and network meta-analyses that opioids are not superior to non-opioid analgesics in the treatment of acute or chronic non-cancer pain. However, due to the variability and diversity of opioid effects across contexts and experiences, some people under some circumstances may indeed benefit from prolonged treatment. I will present data on individual differences in opioid effects due to participant sex and stress induction. Understanding the effects of these commonly used medications on other aspects of the human experience is important to ensure correct use and to prevent unnecessary pain and addiction risk.
Use of brain imaging data to improve prescriptions of psychotropic drugs - Examples of ketamine in depression and antipsychotics in schizophrenia
The use of molecular imaging, particularly PET and SPECT, has significantly transformed the treatment of schizophrenia with antipsychotic drugs since the late 1980s. It has offered insights into the links between drug target engagement, clinical effects, and side effects. A therapeutic window for receptor occupancy is established for antipsychotics, yet there is a divergence of opinions regarding the importance of blood levels, with many downplaying their significance. As a result, the role of therapeutic drug monitoring (TDM) as a personalized therapy tool is often underrated. Since molecular imaging of antipsychotics has focused almost entirely on D2-like dopamine receptors and their potential to control positive symptoms, negative symptoms and cognitive deficits are hardly or not at all investigated. Alternative methods have been introduced, i.e. to investigate the correlation between approximated receptor occupancies from blood levels and cognitive measures. Within the domain of antidepressants, and specifically regarding ketamine's efficacy in depression treatment, there is limited comprehension of the association between plasma concentrations and target engagement. The measurement of AMPA receptors in the human brain has added a new level of comprehension regarding ketamine's antidepressant effects. To ensure precise prescription of psychotropic drugs, it is vital to have a nuanced understanding of how molecular and clinical effects interact. Clinician scientists are assigned with the task of integrating these indispensable pharmacological insights into practice, thereby ensuring a rational and effective approach to the treatment of mental health disorders, signaling a new era of personalized drug therapy mechanisms that promote neuronal plasticity not only under pathological conditions, but also in the healthy aging brain.
Social and non-social learning: Common, or specialised, mechanisms? (BACN Early Career Prize Lecture 2022)
The last decade has seen a burgeoning interest in studying the neural and computational mechanisms that underpin social learning (learning from others). Many findings support the view that learning from other people is underpinned by the same, ‘domain-general’, mechanisms underpinning learning from non-social stimuli. Despite this, the idea that humans possess social-specific learning mechanisms - adaptive specializations moulded by natural selection to cope with the pressures of group living - persists. In this talk I explore the persistence of this idea. First, I present dissociations between social and non-social learning - patterns of data which are difficult to explain under the domain-general thesis and which therefore support the idea that we have evolved special mechanisms for social learning. Subsequently, I argue that most studies that have dissociated social and non-social learning have employed paradigms in which social information comprises a secondary, additional, source of information that can be used to supplement learning from non-social stimuli. Thus, in most extant paradigms, social and non-social learning differ both in terms of social nature (social or non-social) and status (primary or secondary). I conclude that status is an important driver of apparent differences between social and non-social learning. When we account for differences in status, we see that social and non-social learning share common (dopamine-mediated) mechanisms.
Dopamine and Acetylcholine waves in the striatum
Light-driven dopamine release in the adult and developing retina
Learning to Express Reward Prediction Error-like Dopaminergic Activity Requires Plastic Representations of Time
The dominant theoretical framework to account for reinforcement learning in the brain is temporal difference (TD) reinforcement learning. The TD framework predicts that some neuronal elements should represent the reward prediction error (RPE), which means they signal the difference between the expected future rewards and the actual rewards. The prominence of the TD theory arises from the observation that firing properties of dopaminergic neurons in the ventral tegmental area appear similar to those of RPE model-neurons in TD learning. Previous implementations of TD learning assume a fixed temporal basis for each stimulus that might eventually predict a reward. Here we show that such a fixed temporal basis is implausible and that certain predictions of TD learning are inconsistent with experiments. We propose instead an alternative theoretical framework, coined FLEX (Flexibly Learned Errors in Expected Reward). In FLEX, feature specific representations of time are learned, allowing for neural representations of stimuli to adjust their timing and relation to rewards in an online manner. In FLEX dopamine acts as an instructive signal which helps build temporal models of the environment. FLEX is a general theoretical framework that has many possible biophysical implementations. In order to show that FLEX is a feasible approach, we present a specific biophysically plausible model which implements the principles of FLEX. We show that this implementation can account for various reinforcement learning paradigms, and that its results and predictions are consistent with a preponderance of both existing and reanalyzed experimental data.
How curiosity affects learning and information seeking via the dopaminergic circuit
Over the last decade, research on curiosity – the desire to seek new information – has been rapidly growing. Several studies have shown that curiosity elicits activity within the dopaminergic circuit and thereby enhances hippocampus-dependent learning. However, given this new field of research, we do not have a good understanding yet of (i) how curiosity-based learning changes across the lifespan, (ii) why some people show better learning improvements due to curiosity than others, and (iii) whether lab-based research on curiosity translates to how curiosity affects information seeking in real life. In this talk, I will present a series of behavioural and neuroimaging studies that address these three questions about curiosity. First, I will present findings on how curiosity and interest affect learning differently in childhood and adolescence. Second, I will show data on how inter-individual differences in the magnitude of curiosity-based learning depend on the strength of resting-state functional connectivity within the cortico-mesolimbic dopaminergic circuit. Third, I will present findings on how the level of resting-state functional connectivity within this circuit is also associated with the frequency of real-life information seeking (i.e., about Covid-19-related news). Together, our findings help to refine our recently proposed framework – the Prediction, Appraisal, Curiosity, and Exploration (PACE) framework – that attempts to integrate theoretical ideas on the neurocognitive mechanisms of how curiosity is elicited, and how curiosity enhances learning and information seeking. Furthermore, our findings highlight the importance of curiosity research to better understand how curiosity can be harnessed to improve learning and information seeking in real life.
Richly structured reward predictions in dopaminergic learning circuits
Theories from reinforcement learning have been highly influential for interpreting neural activity in the biological circuits critical for animal and human learning. Central among these is the identification of phasic activity in dopamine neurons as a reward prediction error signal that drives learning in basal ganglia and prefrontal circuits. However, recent findings suggest that dopaminergic prediction error signals have access to complex, structured reward predictions and are sensitive to more properties of outcomes than learning theories with simple scalar value predictions might suggest. Here, I will present recent work in which we probed the identity-specific structure of reward prediction errors in an odor-guided choice task and found evidence for multiple predictive “threads” that segregate reward predictions, and reward prediction errors, according to the specific sensory features of anticipated outcomes. Our results point to an expanded class of neural reinforcement learning algorithms in which biological agents learn rich associative structure from their environment and leverage it to build reward predictions that include information about the specific, and perhaps idiosyncratic, features of available outcomes, using these to guide behavior in even quite simple reward learning tasks.
Off-policy learning in the basal ganglia
I will discuss work with Jack Lindsey modeling reinforcement learning for action selection in the basal ganglia. I will argue that the presence of multiple brain regions, in addition to the basal ganglia, that contribute to motor control motivates the need for an off-policy basal ganglia learning algorithm. I will then describe a biological implementation of such an algorithm that predicts tuning of dopamine neurons to a quantity we call "action surprise," in addition to reward prediction error. In the same model, an implementation of learning from a motor efference copy also predicts a novel solution to the problem of multiplexing feedforward and efference-related striatal activity. The solution exploits the difference between D1 and D2-expressing medium spiny neurons and leads to predictions about striatal dynamics.
Obesity and Brain – Bidirectional Influences
The regulation of body weight relies on homeostatic mechanisms that use a combination of internal signals and external cues to initiate and terminate food intake. Homeostasis depends on intricate communication between the body and the hypothalamus involving numerous neural and hormonal signals. However, there is growing evidence that higher-level cognitive function may also influence energy balance. For instance, research has shown that BMI is consistently linked to various brain, cognitive, and personality measures, implicating executive, reward, and attentional systems. Moreover, the rise in obesity rates over the past half-century is attributed to the affordability and widespread availability of highly processed foods, a phenomenon that contradicts the idea that food intake is solely regulated by homeostasis. I will suggest that prefrontal systems involved in value computation and motivation act to limit food overconsumption when food is scarce or expensive, but promote over-eating when food is abundant, an optimum strategy from an economic standpoint. I will review the genetic and neuroscience literature on the CNS control of body weight. I will present recent studies supporting a role of prefrontal systems in weight control. I will also present contradictory evidence showing that frontal executive and cognitive findings in obesity may be a consequence not a cause of increased hunger. Finally I will review the effects of obesity on brain anatomy and function. Chronic adiposity leads to cerebrovascular dysfunction, cortical thinning, and cognitive impairment. As the most common preventable risk factor for dementia, obesity poses a significant threat to brain health. I will conclude by reviewing evidence for treatment of obesity in adults to prevent brain disease.
Hallucinating mice, dopamine and immunity; towards mechanistic treatment targets for psychosis
Hallucinations are a core symptom of psychotic disorders and have traditionally been difficult to study biologically. We developed a new behavioral computational approach to measure hallucinations-like perception in humans and mice alike. Using targeted neural circuit manipulations, we identified a causal role for striatal dopamine in mediating hallucination-like perception. Building on this, we currently investigate the neural and immunological upstream regulators of these dopaminergic circuits with the goal to identify new biological treatment targets for psychosis
Integration of 3D human stem cell models derived from post-mortem tissue and statistical genomics to guide schizophrenia therapeutic development
Schizophrenia is a neuropsychiatric disorder characterized by positive symptoms (such as hallucinations and delusions), negative symptoms (such as avolition and withdrawal) and cognitive dysfunction1. Schizophrenia is highly heritable, and genetic studies are playing a pivotal role in identifying potential biomarkers and causal disease mechanisms with the hope of informing new treatments. Genome-wide association studies (GWAS) identified nearly 270 loci with a high statistical association with schizophrenia risk; however each locus confers only a small increase in risk therefore it is difficult to translate these findings into understanding disease biology that can lead to treatments. Induced pluripotent stem cell (iPSC) models are a tractable system to translate genetic findings and interrogate mechanisms of pathogenesis. Mounting research with patient-derived iPSCs has proposed several neurodevelopmental pathways altered in SCZ, such as neural progenitor cell (NPC) proliferation, imbalanced differentiation of excitatory and inhibitory cortical neurons. However, it is unclear what exactly these iPS models recapitulate, how potential perturbations of early brain development translates into illness in adults and how iPS models that represent fetal stages can be utilized to further drug development efforts to treat adult illness. I will present the largest transcriptome analysis of post-mortem caudate nucleus in schizophrenia where we discovered that decreased presynaptic DRD2 autoregulation is the causal dopamine risk factor for schizophrenia (Benjamin et al, Nature Neuroscience 2022 https://doi.org/10.1038/s41593-022-01182-7). We developed stem cell models from a subset of the postmortem cohort to better understand the molecular underpinnings of human psychiatric disorders (Sawada et al, Stem Cell Research 2020). We established a method for the differentiation of iPS cells into ventral forebrain organoids and performed single cell RNAseq and cellular phenotyping. To our knowledge, this is the first study to evaluate iPSC models of SZ from the same individuals with postmortem tissue. Our study establishes that striatal neurons in the patients with SCZ carry abnormalities that originated during early brain development. Differentiation of inhibitory neurons is accelerated whereas excitatory neuronal development is delayed, implicating an excitation and inhibition (E-I) imbalance during early brain development in SCZ. We found a significant overlap of genes upregulated in the inhibitory neurons in SCZ organoids with upregulated genes in postmortem caudate tissues from patients with SCZ compared with control individuals, including the donors of our iPS cell cohort. Altogether, we demonstrate that ventral forebrain organoids derived from postmortem tissue of individuals with schizophrenia recapitulate perturbed striatal gene expression dynamics of the donors’ brains (Sawada et al, biorxiv 2022 https://doi.org/10.1101/2022.05.26.493589).
Dopamine and cellular mechanisms of cognitive control in primate prefrontal cortex
Classification of Dopamine Cells
Mapping learning and decision-making algorithms onto brain circuitry
In the first half of my talk, I will discuss our recent work on the midbrain dopamine system. The hypothesis that midbrain dopamine neurons broadcast an error signal for the prediction of reward is among the great successes of computational neuroscience. However, our recent results contradict a core aspect of this theory: that the neurons uniformly convey a scalar, global signal. I will review this work, as well as our new efforts to update models of the neural basis of reinforcement learning with our data. In the second half of my talk, I will discuss our recent findings of state-dependent decision-making mechanisms in the striatum.
Dopamine receptors dysregulation in BG disease
Chemistry of the adaptive mind: lessons from dopamine
The human brain faces a variety of computational dilemmas, including the flexibility/stability, the speed/accuracy and the labor/leisure tradeoff. I will argue that striatal dopamine is particularly well suited to dynamically regulate these computational tradeoffs depending on constantly changing task demands. This working hypothesis is grounded in evidence from recent studies on learning, motivation and cognitive control in human volunteers, using chemical PET, psychopharmacology, and/or fMRI. These studies also begin to elucidate the mechanisms underlying the huge variability in catecholaminergic drug effects across different individuals and across different task contexts. For example, I will demonstrate how effects of the most commonly used psychostimulant methylphenidate on learning, Pavlovian and effortful instrumental control depend on fluctuations in current environmental volatility, on individual differences in working memory capacity and on opportunity cost respectively.
Dyskinesia: the failure of dopamine-dependent motor control
Learning in/about/from the basal ganglia
The basal ganglia are a collection of brain areas that are connected by a variety of synaptic pathways and are a site of significant reward-related dopamine release. These properties suggest a possible role for the basal ganglia in action selection, guided by reinforcement learning. In this talk, I will discuss a framework for how this function might be performed and computational results using an upward mapping to identify putative low-dimensional control ensembles that may be involved in tuning decision policy. I will also present some recent experimental results and theory – related to effects of extracellular ion dynamics -- that run counter to the classical view of basal ganglia pathways and suggest a new interpretation of certain aspects of this framework. For those not so interested in the basal ganglia, I hope that the upward mapping approach and impact of extracellular ion dynamics will nonetheless be of interest!
Mechanisms and Roles of Fast Dopamine Signaling
Dopamine is a neuromodulator that codes information on various time scales. I will discuss recent progress on the identification of fast release mechanisms for dopamine in the mouse striatum. I will present data on triggering mechanisms of dopamine release and evaluate its roles in striatal regulation. In the long-term, our work will allow for a better understanding of the mechanisms and time scales of dopamine coding in health and disease.
Brain and behavioural impacts of early life adversity
Abuse, neglect, and other forms of uncontrollable stress during childhood and early adolescence can lead to adverse outcomes later in life, including especially perturbations in the regulation of mood and emotional states, and specifically anxiety disorders and depression. However, stress experiences vary from one individual to the next, meaning that causal relationships and mechanistic accounts are often difficult to establish in humans. This interdisciplinary talk considers the value of research in experimental animals where stressor experiences can be tightly controlled and detailed investigations of molecular, cellular, and circuit-level mechanisms can be carried out. The talk will focus on the widely used repeated maternal separation procedure in rats where rat offspring are repeatedly separated from maternal care during early postnatal life. This early life stress has remarkably persistent effects on behaviour with a general recognition that maternally-deprived animals are susceptible to depressive-like phenotypes. The validity of this conclusion will be critically appraised with convergent insights from a recent longitudinal study in maternally separated rats involving translational brain imaging, transcriptomics, and behavioural assessment.
Neuromodulation of sleep integrity
The arousal construct underlies a spectrum of behaviors that include sleep, exploration, feeding, sexual activity and adaptive stress. Pathological arousal conditions include stress, anxiety disorders, and addiction. The dynamics between arousal state transitions are modulated by norepinephrine neurons in the locus coeruleus, histaminergic neurons in the hypothalamus, dopaminergic neurons in the mesencephalon and cholinergic neurons in the basal forebrain. The hypocretin/orexin system in the lateral hypothalamus I will also present a new mechanism underlying sleep fragmentation during aging. Hcrt neurons are hyperexcitable in aged mice. We identify a potassium conductance known as the M-current, as a critical player in maintaining excitability of Hcrt neurons. Genetic disruption of KCNQ channels in Hcrt neurons of young animals results in sleep fragmentation. In contrast, treatment of aged animals with a KCNQ channel opener restores sleep/wake architecture. These data point to multiple circuits modulating sleep integrity across lifespan.
Inter-individual variability in reward seeking and decision making: role of social life and consequence for vulnerability to nicotine
Inter-individual variability refers to differences in the expression of behaviors between members of a population. For instance, some individuals take greater risks, are more attracted to immediate gains or are more susceptible to drugs of abuse than others. To probe the neural bases of inter-individual variability we study reward seeking and decision-making in mice, and dissect the specific role of dopamine in the modulation of these behaviors. Using a spatial version of the multi-armed bandit task, in which mice are faced with consecutive binary choices, we could link modifications of midbrain dopamine cell dynamics with modulation of exploratory behaviors, a major component of individual characteristics in mice. By analyzing mouse behaviors in semi-naturalistic environments, we then explored the role of social relationships in the shaping of dopamine activity and associated beahviors. I will present recent data from the laboratory suggesting that changes in the activity of dopaminergic networks link social influences with variations in the expression of non-social behaviors: by acting on the dopamine system, the social context may indeed affect the capacity of individuals to make decisions, as well as their vulnerability to drugs of abuse, in particular nicotine.
Metabolic spikes: from rogue electrons to Parkinson's
Conventionally, neurons are thought to be cellular units that process synaptic inputs into synaptic spikes. However, it is well known that neurons can also spike spontaneously and display a rich repertoire of firing properties with no apparent functional relevance e.g. in in vitro cortical slice preparations. In this talk, I will propose a hypothesis according to which intrinsic excitability in neurons may be a survival mechanism to minimize toxic byproducts of the cell’s energy metabolism. In neurons, this toxicity can arise when mitochondrial ATP production stalls due to limited ADP. Under these conditions, electrons deviate from the electron transport chain to produce reactive oxygen species, disrupting many cellular processes and challenging cell survival. To mitigate this, neurons may engage in ADP-producing metabolic spikes. I will explore the validity of this hypothesis using computational models that illustrate the implications of synaptic and metabolic spiking, especially in the context of substantia nigra pars compacta dopaminergic neurons and their degeneration in Parkinson's disease.
Dynamic dopaminergic signaling probabilistically controls the timing of self-timed movements
Human movement disorders and pharmacological studies have long suggested molecular dopamine modulates the pace of the internal clock. But how does the endogenous dopaminergic system influence the timing of our movements? We examined the relationship between dopaminergic signaling and the timing of reward-related, self-timed movements in mice. Animals were trained to initiate licking after a self-timed interval following a start cue; reward was delivered if the animal’s first lick fell within a rewarded window (3.3-7 s). The first-lick timing distributions exhibited the scalar property, and we leveraged the considerable variability in these distributions to determine how the activity of the dopaminergic system related to the animals’ timing. Surprisingly, dopaminergic signals ramped-up over seconds between the start-timing cue and the self-timed movement, with variable dynamics that predicted the movement/reward time, even on single trials. Steeply rising signals preceded early initiation, whereas slowly rising signals preceded later initiation. Higher baseline signals also predicted earlier self-timed movement. Optogenetic activation of dopamine neurons during self-timing did not trigger immediate movements, but rather caused systematic early-shifting of the timing distribution, whereas inhibition caused late-shifting, as if dopaminergic manipulation modulated the moment-to-moment probability of unleashing the planned movement. Consistent with this view, the dynamics of the endogenous dopaminergic signals quantitatively predicted the moment-by-moment probability of movement initiation. We conclude that ramping dopaminergic signals, potentially encoding dynamic reward expectation, probabilistically modulate the moment-by-moment decision of when to move. (Based on work from Hamilos et al., eLife, 2021).
Primary Motor Cortex Circuitry in a Mouse Model of Parkinson’s Disease
The primary motor cortex (M1) is a major output center for movement execution and motor learning, and its dysfunction contributes to the pathophysiology of Parkinson’s disease (PD). While human studies have indicated that a loss of midbrain dopamine neurons alters M1 activation, the mechanisms underlying this phenomenon remain unclear. Using a mouse model of PD, we uncovered several shifts within M1 circuitry following dopamine depletion, including impaired excitation by thalamocortical afferents and altered excitability. Our findings add to the growing body of literature highlighting M1 as a major contributor in PD, and provide targeted neural substrates for possible therapeutic interventions.
NaV Long-term Inactivation Regulates Adaptation in Place Cells and Depolarization Block in Dopamine Neurons
In behaving rodents, CA1 pyramidal neurons receive spatially-tuned depolarizing synaptic input while traversing a specific location within an environment called its place. Midbrain dopamine neurons participate in reinforcement learning, and bursts of action potentials riding a depolarizing wave of synaptic input signal rewards and reward expectation. Interestingly, slice electrophysiology in vitro shows that both types of cells exhibit a pronounced reduction in firing rate (adaptation) and even cessation of firing during sustained depolarization. We included a five state Markov model of NaV1.6 (for CA1) and NaV1.2 (for dopamine neurons) respectively, in computational models of these two types of neurons. Our simulations suggest that long-term inactivation of this channel is responsible for the adaptation in CA1 pyramidal neurons, in response to triangular depolarizing current ramps. We also show that the differential contribution of slow inactivation in two subpopulations of midbrain dopamine neurons can account for their different dynamic ranges, as assessed by their responses to similar depolarizing ramps. These results suggest long-term inactivation of the sodium channel is a general mechanism for adaptation.
A Flash of Darkness within Dusk: Crossover inhibition in the mouse retina
To survive in the wild small rodents evolved specialized retinas. To escape predators, looming shadows need to be detected with speed and precision. To evade starvation, small seeds, grass, nuts and insects need to also be detected quickly. Some of these succulent seeds and insects may be camouflaged offering only low contrast targets.Moreover, these challenging tasks need to be accomplished continuously at dusk, night, dawn and daytime. Crossover inhibition is thought to be involved in enhancing contrast detectionin the microcircuits of the inner plexiform layer of the mammalian retina. The AII amacrine cells are narrow field cells that play a key role in crossover inhibition. Our lab studies the synaptic physiology that regulates glycine release from AII amacrine cellsin mouse retina. These interneurons receive excitation from rod and conebipolar cells and transmit excitation to ON-type bipolar cell terminals via gap junctions. They also transmit inhibition via multiple glycinergic synapses onto OFF bipolar cell terminals.AII amacrine cells are thus a central hub of synaptic information processing that cross links the ON and the OFF pathways. What are the functions of crossover inhibition? How does it enhance contrast detection at different ambient light levels? How is the dynamicrange, frequency response and synaptic gain of glycine release modulated by luminance levels and circadian rhythms? How is synaptic gain changed by different extracellular neuromodulators, like dopamine, and by intracellular messengers like cAMP, phosphateand Ca2+ ions from Ca2+ channels and Ca2+ stores? My talk will try to answer some of these questions and will pose additional ones. It will end with further hypothesis and speculations on the multiple roles of crossover inhibition.
Untitled Seminar
Laura Fenlon (Australia): Time shapes all brains: timing of a conserved transcriptional network underlies divergent cortical connectivity routes in mammalian brain development and evolution; Laurent Nguyen (Belgium): Regulation of cerebral cortex morphogenesis by migrating cells; Carol Ann Mason (USA): Wiring the eye to brain for binocular vision: lessons from the albino visual system. Thomas Perlmann (Sweden): Interrogating dopamine neuron development at the single cell level
Phasic dopamine signaling in the homeostasis to action arc
Network dynamics in the basal ganglia and possible implications for Parkinson’s disease
The basal ganglia are a collection of brain areas that are connected by a variety of synaptic pathways and are a site of significant reward-related dopamine release. These properties suggest a possible role for the basal ganglia in action selection, guided by reinforcement learning. In this talk, I will discuss a framework for how this function might be performed. I will also present some recent experimental results and theory that call for a re-evaluation of certain aspects of this framework. Next, I will turn to the changes in basal ganglia activity observed to occur with the dopamine depletion associated with Parkinson’s disease. I will discuss some of the potential functional implications of some of these changes and, if time permits, will conclude with some new results that focus on delta oscillations under dopamine depletion.
Synaptic health in Parkinson's Disease
Parkinson's disease (PD) is the second most common neurodegenerative disorder, affecting 1% of over 65's; there is currently no effective treatment. Dopaminergic neuronal loss is hallmark in PD and yet despite decades of intensive research there is still no known therapeutic which will completely halt the disorder. As a result, identification of interventive therapies to reverse or prevent PD are essential. Using genetically faithful models (induced pluripotent stem cells and knock-in mice) of familial late onset PD (LRRK2 G2019S and GBA N370S) we have contributed to the literature that neuronal dysfunction precedes degeneration. Specifically, using whole cell patch clamp electrophysiology, biochemical, behavioural and molecular biological techniques, we have begun to investigate the fundamental processes that make neurons specialised i.e., synaptic function and neurotransmission. We illustrate those alterations to spontaneous neurotransmitter release, neuronal firing, and short-term plasticity as well as Ca2+ and energy dyshomeostasis, are some of the earliest observable pathological dysfunctions and are likely precursors to late-stage degeneration. These pathologies represent targets which can be manipulated to address causation, rather than the symptoms of the PD, and represent a marker that, if measurable in patients, could form the basis of early PD detection and intervention.
A role for dopamine in value-free learning
Recent success in training artificial agents and robots derives from a combination of direct learning of behavioral policies and indirect learning via value functions. Policy learning and value learning employ distinct algorithms that depend upon evaluation of errors in performance and reward prediction errors, respectively. In mammals, behavioral learning and the role of mesolimbic dopamine signaling have been extensively evaluated with respect to reward prediction errors; but there has been little consideration of how direct policy learning might inform our understanding. I’ll discuss our recent work on classical conditioning in naïve mice (https://www.biorxiv.org/content/10.1101/2021.05.31.446464v1) that provides multiple lines of evidence that phasic dopamine signaling regulates policy learning from performance errors in addition to its well-known roles in value learning. This work points towards new opportunities for unraveling the mechanisms of basal ganglia control over behavior under both adaptive and maladaptive learning conditions.
Dopaminergic modulation of synaptic plasticity in learning and psychiatric disorders
Transient changes in dopamine activity in response to reward and punishment have been known to regulate reward-related learning. However, the cellular basis that detects the transient dopamine signaling has long been unclear. Using two-photon microscopy and optogenetics, I have shown that transient increases and decreases of dopamine modulate plasticity of dopamine D1 and D2 receptor-expressing cells in the nucleus accumbens, respectively. At the behavioral level, I characterized that these D1 and D2 cells cooperatively tune learning by generalization and discrimination learning. Interestingly, disturbance of the dopamine signaling impaired D2 cell plasticity and discrimination learning, which was analogous to salience misattribution seen in subjects with schizophrenia.
Sleepless in Vienna - how to rescue folding-deficient dopamine transporters by pharmacochaperoning
Diseases that arise from misfolding of an individual protein are rare. However, collectively, these folding diseases represent a large proportion of hereditary and acquired disorders. In fact, the term "Molecular Medicine" was coined by Linus Pauling in conjunction with the study of a folding disease, i.e. sickle cell anemia. In the past decade, we have witnessed an exponential growth in the number of mutations, which have been identified in genes encoding solute carriers (SLC). A sizable faction - presumably the majority - of these mutations result in misfolding of the encoded protein. While studying the export of the GABA transporter (SLC6A1) and of the serotonin transporter (SLC6A4), from the endoplasmic reticulum (ER), we discovered by serendipity that some ligands can correct the folding defect imparted by point mutations. These bind to the inward facing state. The most effective compound is noribogaine, the metabolite of ibogaine (an alkaloid first isolated from the shrub Tabernanthe iboga). There are 13 mutations in the human dopamine transporter (DAT, SLC6A3), which give rise to a syndrome of infantile Parkinsonism and dystonia. We capitalized on our insights to explore, if the disease-relevant mutant proteins were amenable to pharmacological correction. Drosopohila melanogaster, which lack the dopamine transporter, are hyperactive and sleepless (fumin in Japanese). Thus, mutated human DAT variants can be introduced into fumin flies. This allows for examining the effect of pharmacochaperones on delivery of DAT to the axonal territory and on restoring sleep. We explored the chemical space populated by variations of the ibogaine structure to identify an analogue (referred to as compound 9b), which was highly effective: compound 9b also restored folding in DAT variants, which were not amenable to rescue by noribogaine. Deficiencies in the human creatine transporter-1 (CrT1, SLC6A8) give rise to a syndrome of intellectual disability and seizures and accounts for 5% of genetically based intellectual disabilities in boys. Point mutations occur, in part, at positions, which are homologous to those of folding-deficient DAT variants. CrT1 lacks the rich pharmacology of monoamine transporters. Nevertheless, our insights are also applicable to rescuing some disease-related variants of CrT1. Finally, the question arises how one can address the folding problem. We propose a two-pronged approach: (i) analyzing the effect of mutations on the transport cycle by electrophysiological recordings; this allows for extracting information on the rates of conformational transitions. The underlying assumption posits that - even when remedied by pharmacochaperoning - folding-deficient mutants must differ in the conformational transitions associated with the transport cycle. (ii) analyzing the effect of mutations on the two components of protein stability, i.e. thermodynamic and kinetic stability. This is expected to provide a glimpse of the energy landscape, which governs the folding trajectory.
Dopamine
Dopamine release in the nucleus accumbens core signals perceived saliency
Brief Sensory Deprivation Triggers Cell Type-Specific Structural and Functional Plasticity in Olfactory Bulb Neurons
Can alterations in experience trigger different plastic modifications in neuronal structure and function, and if so, how do they integrate at the cellular level? To address this question, we interrogated circuitry in the mouse olfactory bulb responsible for the earliest steps in odor processing. We induced experience-dependent plasticity in mice of either sex by blocking one nostril for one day, a minimally invasive manipulation that leaves the sensory organ undamaged and is akin to the natural transient blockage suffered during common mild rhinal infections. We found that such brief sensory deprivation produced structural and functional plasticity in one highly specialized bulbar cell type: axon-bearing dopaminergic neurons in the glomerular layer. After 24 h naris occlusion, the axon initial segment (AIS) in bulbar dopaminergic neurons became significantly shorter, a structural modification that was also associated with a decrease in intrinsic excitability. These effects were specific to the AIS-positive dopaminergic subpopulation because no experience-dependent alterations in intrinsic excitability were observed in AIS-negative dopaminergic cells. Moreover, 24 h naris occlusion produced no structural changes at the AIS of bulbar excitatory neurons, mitral/tufted and external tufted cells, nor did it alter their intrinsic excitability. By targeting excitability in one specialized dopaminergic subpopulation, experience-dependent plasticity in early olfactory networks might act to fine-tune sensory processing in the face of continually fluctuating inputs. (https://www.jneurosci.org/content/41/10/2135)
Preference dynamics in economic decision-making explained by dopaminergic distributional codes
Bernstein Conference 2024
Utilizing Random Forest for Multivariate Analysis: Exploring the Influence of Dopaminergic Neurons on Drosophila Larvae Locomotion
Bernstein Conference 2024
Distinct dynamics in projection-specific midbrain dopamine populations for learning and motivation
COSYNE 2022
Dopamine and norepinephrine signaling differentially mediate the exploration-exploitation tradeoff
COSYNE 2022
VTA dopamine neurons signal phasic and ramping reward prediction error in goal-directed navigation
COSYNE 2022
Exploration of learning by dopamine D1 and D2 receptors by a spiking network model of the basal ganglia
COSYNE 2022
Improved striatal learning with vector-valued errors mediated by diffusely transmitted dopamine
COSYNE 2022
Improved striatal learning with vector-valued errors mediated by diffusely transmitted dopamine
COSYNE 2022
Learning and expression of dopaminergic reward prediction error via plastic representations of time
COSYNE 2022
Learning and expression of dopaminergic reward prediction error via plastic representations of time
COSYNE 2022
Linking tonic dopamine and biased value predictions in a biologically inspired reinforcement learning model
COSYNE 2022
Linking tonic dopamine and biased value predictions in a biologically inspired reinforcement learning model
COSYNE 2022
Mesolimbic dopamine encodes subjective value and predicts time investment decisions
COSYNE 2022
Mesolimbic dopamine encodes subjective value and predicts time investment decisions
COSYNE 2022
Modeling Hippocampal Spatial Learning Through a Valence-based Interplay of Dopamine and Serotonin
COSYNE 2022
Modeling Hippocampal Spatial Learning Through a Valence-based Interplay of Dopamine and Serotonin
COSYNE 2022
Probing neural value computations in the nucleus accumbens dopamine signal
COSYNE 2022
Probing neural value computations in the nucleus accumbens dopamine signal
COSYNE 2022
Reward-related dynamics of dopamine in the hippocampus
COSYNE 2022
Reward-related dynamics of dopamine in the hippocampus
COSYNE 2022
Cortical dopamine enables deep reinforcement learning and leverages dopaminergic heterogeneity
COSYNE 2023
Dopamine projections to the basolateral amygdala drive the encoding of identity-specific reward memories
COSYNE 2023
Dopamine release in the nucleus accumbens during backward conditioning
COSYNE 2023
Dopamine neurons reveal an efficient code for a multidimensional, distributional map of the future
COSYNE 2023
Mesolimbic dopamine release conveys causal associations
COSYNE 2023
Striatal dopamine encodes movement and value at distinct time points
COSYNE 2023
Timing-dependent modulation of working memory by dopaminergic release in the prefrontal cortex
COSYNE 2023
The vanishing dopamine in Parkinson's disease
COSYNE 2023
Altered sensory prediction error signaling and dopamine function drive speech hallucinations in schizophrenia
COSYNE 2025
Broadly-projecting mesolimbic dopamine neurons implement a distributional critic across the striatum
COSYNE 2025
Differential coding of valence and expectation signals across the dopaminergic system
COSYNE 2025
Dopamine ramps encode discounted future value on a moment-by-moment basis
COSYNE 2025
Dopamine controls neural coding of anxiety and valence in the mouse anterior insula
COSYNE 2025
Dopamine signaling for perceptual learning in the sensory striatum
COSYNE 2025
Hunger modulates exploration through dopamine signaling at the tail of striatum
COSYNE 2025
Layered, hierarchical behavioral control underlies dopamine signals across the striatum during decision-making
COSYNE 2025
Midbrain dopamine activity produces regionally localized decision substrates
COSYNE 2025
Minimal neural circuit elements for dopaminergic temporal difference learning
COSYNE 2025
Rapid learning of nonlinear network dynamics via dopamine-gated non-Hebbian plasticity
COSYNE 2025
Dopamine dysregulation in Parkinson's Disease
Bernstein Conference 2024
dopamine coverage
90 items