Thoughts of Mice: Understanding the Role Serotonin Plays in Cognitive Flexibility

2021 Bio-Rad Science Writing Competition 2^nd Place

Ashlea is currently a PhD candidate in the Neurobiology and Behavior Program at Columbia University in New York, USA. She works in the laboratory of Dr. Mark Ansorge to investigate the role of serotonin signaling in cognitive and emotional behavior in a mouse model.

Ashlea impressed the judges with her eloquent description of her PhD project and style throughout. A really interesting read that highlights the role of translational animal models in neuroscience research.

We are delighted to publish Ashlea’s entry below.

“Describe what you do (wrong answers only)!” I saw this Facebook post and wanted to respond. After thinking for a bit, I got it: “I put mice in boxes in the hope that one day they will tell me what they’re thinking.” It was a joke, but in some ways, I wasn’t kidding.

I research how decision-makers change their minds and actions. I specifically want to know how the neurotransmitter, serotonin, helps. Many know of serotonin for its role in depression and “promoting happiness,” but it helps us with much more. From nausea to sleep to sexual function, serotonin is intimately involved in the emotional, autonomic, and unconscious aspects of life. Through my doctoral research, I set out to study the role serotonin plays in the cognitive aspects of life, and specifically, cognitive flexibility.

Most accurately, cognitive flexibility is the ability of an individual to change cognitive patterns to adapt to changing external stimuli. Although popular psychology magazines and blogs tote cognitive flexibility as essential for tackling life’s challenges, the mechanisms underlying our ability to think and behave flexibly are not well understood. Initial hints of how the brain carries this out came from researchers who manipulated serotonin in the brain of primates.

When researchers disrupted serotonin projections to the prefrontal cortex (PFC), they observed that primates made more errors during reversal (or when a previously rewarded response no longer yields a reward) (Clarke et al. 2005, 2007). This parallels the reversal deficits researchers observed in humans when the amino acid precursor to serotonin (tryptophan) was left out of their diet (Rogers et al. 1999).

My research built on this work to study how serotonin is used in the PFC during this cognitive flexibility behavior... in mice. In mice, unlike humans, we can carefully study serotonin during specific times (for example before decision-making) and in specific brain subregions (for example medial or “middle of the” PFC). As you might guess, I cannot ask my mice directly, “How do you use serotonin to learn how to switch between two choices?” but I can observe their behavior and manipulate serotonin release.

To study their behavior, I use a two-choice digging task. I first train the mice to dig for a reward. Basically, I’m saying: "Hey mouse, there are these two identical clay dishes filled with bedding, and if you dig to the bottom of each, you will get a delicious piece of that Honey Nut Cheerio you love.” They learn that dish digging = reward. No matter what. 

After they get good at this (that is digging and finding the rewards quickly), I introduce a rule that digging in one of two dishes results in a reward. The rule is based on how the bedding smells (like cinnamon or paprika) or feels (like soft paper or rough corn cob). I mix each odor with each bedding to get four different paired options. Say the rule is odor-based, and specifically cinnamon. Each time the mouse digs in the cinnamon-scented dish (regardless of bedding type), it gets a Cheerio bit whereas, digging in bedding mixed with paprika yields nothing. Within the first dozen or so tries, mice reach 80% correct (that is the criterion).

This is nice, but what I really want to ask my mice is: “How long does it take you to figure out a rule change? How flexible is your behavior and how does serotonin help?”. To ask these questions, I swap out which aspect of the test is rewarding. If cinnamon scent = reward before, now how the bedding feels is important (soft paper = reward).

To test the role of serotonin during the rule change, I use what’s called optogenetics. With this technique, I can express a light-sensitive protein (opsin) in all serotonin cells. These specific cells are then excited or inhibited in the presence of a particular wavelength of light. I shine laser light to activate or inhibit the release of serotonin in the medial PFC. What I found is that serotonin release in the medial PFC is critical for when the rules are changed across dimensions (e.g., from odor to texture). I observe that when I stimulate the release of serotonin during the rule change, the mice figure out the rule change sooner and with fewer errors than mice without this opsin. Further, inhibition of serotonin release using optogenetics impairs performance in this task. Together, this indicates that serotonin in this region is both necessary and sufficient.

My mice still may not tell me exactly what they're thinking, but with this knowledge, I use an imaging technique called fiber photometry to peek at how brain activity changes during this task. I can now ask “Are you using serotonin to help unlearn the previous association? Is serotonin helping to change the reward representation?” Using a virus to target the serotonin cells that project to the medial PFC, I get these cells to express a calcium-sensitive fluorophore. Because of the critical role calcium plays in action potentials, and thus cell activity, I can record the fluorescent signals to track the activity of the serotonin cells.

It is early days so I can’t answer all my “listening” questions yet. However, recent literature has indicated that serotonin neurons may play a role in reward coding (Bromberg-Martin et al. 2010), and in particular, the activity of serotonin neurons that project to the medial prefrontal cortex enhance the ability to wait for a reward (Miyazaki et al. 2020).

Together, these results and mine may reveal a target for treatment benefiting patients with cognitive impairments, due to prefrontal cortex dysfunction, and lead to the treatment of cognitive symptoms in neuropsychiatric disorders such as attention deficit disorders, schizophrenia, depression, anxiety, and Parkinson’s disease.

Whether human or mouse, life requires cognitive flexibility. Better understanding of cognitive flexibility may help us all wield this powerful ability to change our beliefs and actions, adapt to the current situation, and benefit ourselves and fellow humans.