How a single neurotransmitter can provide the basis for the explanation of all social phenomena
Those of you who read this blog for sometime, already know some of my views concerning psychology. I am an avid supporter of "hard science". Science that is based on solid facts and follows the reductionist paradigm. Even though solid facts are not always the answer and reductionism could be replaced in the following decades by something else, such as a holistic approach that focuses on "systems" rather than "reduction" (read this for example: Sacred Science: Using Faith to Explain Anomalies in Physics from Scientific American), I believe that this paradigm has just started to show its strength in cognitive science. Take this post for example: Neurons, politics and economics
The reason I am writing today is that I found this exciting article in Seed Magazine: A New State of Mind
The article speaks of Read Montague, director of the Human Neuroimaging Lab at Baylor College of Medicine in downtown Houston. He has participated in something very interesting researches which I was aware of. I was surprised, because I had forgotten his name, despite the fact that he was the mind behind some recent papers. However, what really suprised me was how all these researches are connected, what this man is studying and how his view on neuroscience coincides with mine. I am going to present the article and then I’ll make a few comments. If you are bored just skip the article. 
Read Montague studies the actions of dopamine in the brain. However, his methodology is not an experimental (even though he is a researcher), but rather a theoritical one.
Montague, who is uncommonly handsome, with a strong jaw and a Hollywood grin, first got interested in the brain while working in the neuroscience lab of Nobel Laureate Gerald Edelman as a post-doc. “I was never your standard neuroscientist,” he says. “I spent a lot of time thinking about how the brain should work, if I had designed it.” For Montague the cortex was a perfect system to model, since its incomprehensible complexity meant that it depended on some deep, underlying order. “You can’t have all these cells interacting with each other unless there’s some logic to the interaction,” he says. “It just looked like noise, though — no one could crack the code.” That’s what Montague wanted to do.
The code he chose was dopamine. Dopamine is a neurotransmitter linked with many things, but mainly with pleasure. It is also the neurotransmitter that is affected by most drugs (ecstasy, cocaine, methamphetamines)
That’s when Montague discovered the powers of dopamine, a neurotransmitter in the brain. His research on the singular chemical has drawn tantalizing connections between the peculiar habits of our neurons and the peculiar habits of real people, so that the various levels of psychological description — the macro and the micro, the behavioral and the cellular — no longer seem so distinct. What began as an investigation into a single neurotransmitter has morphed into an exploration of the social brain: Montague has pioneered research that allows him to link the obscure details of the cortex to all sorts of important phenomena, from stock market bubbles to cigarette addiction to the development of trust. “We are profoundly social animals,” he says. “You can’t really understand the brain until you understand how these social behaviors happen, or what happens when they go haywire.”
The importance of dopamine was discovered by accident. In 1954 James Olds and Peter Milner, two neuroscientists at McGill University, decided to implant an electrode deep into the center of a rat’s brain. The precise placement of the electrode was largely happenstance: At the time the geography of the mind remained a mystery. But Olds and Milner got lucky. They inserted the needle right next to the nucleus accumbens (NAcc), a part of the brain dense with dopamine neurons and involved with the processing of pleasurable rewards, like food and sex.
What is interesting about dopamine however, is not that it responds to rewards, but that it can predict rewards! This was discovered by Wolfram Schultz, a neuroscientist at Cambridge University.
After hundreds of experimental trials, Schultz began to believe his own data: He realized that he had found, by accident, the reward mechanism at work in the primate brain. “Only in retrospect can I appreciate just how lucky we were,” he says. After publishing a series of landmark papers in the mid-1980s, Schultz set out to decipher this reward circuitry in exquisite detail. How, exactly, did these single cells manage to represent a reward? His experiments observed a simple protocol: He played a loud tone, waited for a few seconds, and then squirted a few drops of apple juice into the mouth of a monkey. While the experiment was unfolding, Schultz was probing the dopamine-rich areas of the monkey brain with a needle that monitored the electrical activity inside individual cells. At first the dopamine neurons didn’t fire until the juice was delivered; they were responding to the actual reward. However, once the animal learned that the tone preceded the arrival of juice — this requires only a few trials — the same neurons began firing at the sound of the tone instead of the sweet reward. And then eventually, if the tone kept on predicting the juice, the cells went silent. They stopped firing altogether.
Professor Dr. Wolfram Schultz
So Schultz discovered by accident the reward mechanism and the mechanism of prediction of rewards in the brain. Then the article goes on to describe how Montague met with Peter Dayan, a researcher that introduced him into a computational learning model called temporal difference reinforcement learning (TDRL). The model was purely theoritical, but when Schultz’s results were out, the two discovered that this model could explain his findings on dopamine.
Peter Dayan, a colleague of Montague’s at Salk, had introduced him to a model called temporal difference reinforcement learning (TDRL). Computer scientists Rich Sutton and Andrew Barto, who both worked on models of artificial intelligence, had pioneered the model. Sutton and Barto wanted to develop a “neuronlike” program that could learn simple rules and behaviors in order to achieve a goal. The basic premise is straightforward: The software makes predictions about what will happen — about how a checkers game will unfold for example — and then compares these predictions with what actually happens. If the prediction is right, that series of predictions gets reinforced. However, if the prediction is wrong, the software reevaluates its representation of the game.
Montague was entranced by these software prototypes. “It was just so clearly the most efficient way to learn,” he says. The problem was that TDRL remained purely theoretical, a system both elegant and imaginary. Even though computer scientists had begun to adapt the programming strategy for various practical purposes, such as running a bank of elevators or determining flight schedules, no one had found a neurological system that worked like this.
But one spring day in 1991, Dayan burst into Montague’s small office. “He was very excited and shoved these figures from some new paper in my face,” Montague remembers. “He kept on saying to me, ‘What does this look like? What does this look like?’” The figures were from Schultz’s experiments with dopamine neurons, and they showed how these cells reacted to the tone and the juice. “I thought he had faked the data,” Montague says. “Dayan was a big prankster, and I assumed he’d photocopied some of our own figures [on TDRL] just to tease me. It looked too good to be true.” Montague immediately realized that he and Dayan could make sense of Schultz’s mysterious neurons. They knew what these dopamine cells were doing; they had seen this code before.
Dopamine pathways and functions
Montague believes that the reason for their success was the fact that they had a theoritical view from the beginning. He believes that sometimes to start with a theory than with data. I think that this can be true in many occasions. Data alone can’t provide a clue. A theory, however, can provide the basis upon which you create and reject hypotheses.
“The only reason we could see it so clearly,” Montague says, “is because we came at it from this theoretical angle. If you were an experimentalist seeing this data, it would have been extremely confusing. What the hell are these cells doing? Why aren’t they just responding to the juice?” That same day Montague and Dayan began writing a technical paper that laid out their insight, explaining how these neurons were making precise predictions about future rewards.
They tried to publish a paper on their findings, but no big journal would accept their research, so they ended up publishing it in smaller ones
But the paper — an awkward mix of Schultz’s dopamine recordings and equations borrowed from computer science — went nowhere. “We wrote that paper 11 times,” Montague says. “It got bounced from every journal. I came this close to leaving the field. I realized that neuroscience just wasn’t ready for theory, even if the theory made sense.”
Nevertheless, Montague and Dayan didn’t give up. They published their ideas in obscure journals, like Advances in Neural Information Processing Systems. When the big journals rejected their interpretation of monkey neurons, they instead looked at the nervous systems of honeybees, which relied on a version of TDRL when foraging for nectar. (That paper got published in Nature in 1995.) “We had to drag the experimentalists kicking and screaming,” Montague says. “They just didn’t understand how these funny-looking equations could explain their data. They told us, ‘We need more data.’ But what’s the point of data if you can’t figure it out?”
You can always publish your work on smaller journals 
And now, here’s where the fun begins. These findings are important not because they simply they provide the neuronal basis of reward, but that of prediction as well. As it seems, the brain is using a very simple behavioristic model of reward versus no-reward to computate future actions. This logic (reward-no reward) is something that the behaviorists had proposed as the explanation of all behavior at the beginning of the 20th century. However, their work was focused solely on behavior, since it was the only thing that it could be observed. At their time, neuropsychology was not so advanced as to take this model far into the brain. Even though, behaviorism is not very popular now (it is still however strong in clinical practice as cognitive-behavioristic therapy), I believe this to be a theoritical revival of its core principles.
The crucial feature of these dopamine neurons, say Montague and Dayan, is that they are more concerned with predicting rewards than with the rewards themselves. Once the cells memorize the simple pattern — a loud tone predicts the arrival of juice — they become exquisitely sensitive to variations on the pattern. If the cellular predictions proved correct and the primates experienced a surge of dopamine, the prediction was reinforced. However, if the pattern was violated — if the tone sounded but the juice never arrived — then the monkey’s dopamine neurons abruptly decreased their firing rate. This is known as the “prediction error signal.” The monkey got upset because its predictions of juice were wrong.
What’s interesting about this system is that it’s all about expectation. Dopamine neurons constantly generate patterns based upon experience: If this, then that. The cacophony of reality is distilled into models of correlation. And if these predictions ever prove incorrect, then the neurons immediately readjust their expectations. The discrepancy is internalized; the anomaly is remembered. “The accuracy comes from the mismatch,” Montague says. “You learn how the world works by focusing on the prediction errors, on the events that you didn’t expect.” Our knowledge, in other words, emerges from our cellular mistakes. The brain learns how to be right by focusing on what it got wrong.
Monkey juice
Then the story goes on to tell us how Montague published a paper that received 1.200 citations
Despite his frustrations with the field, Montague continued to work on dopamine. In 1997 he published a Science paper with Dayan and Schultz whose short title was audaciously grand: “A Neural Substrate of Prediction and Reward.” The paper has since been cited more than 1,200 times, and it remains the definitive explanation of how the brain parses reality into a set of accurate expectations, which are measured out in short bursts of dopamine. A crucial part of the cellular code had been cracked.
That’s where my views coincide with Montague’s. Montague describes how a simple system can evolve into a complex one.
But Montague was getting restless. “I wanted to start asking bigger questions,” he says. “Here’s this elegant learning system, but how does it fit with the rest of the brain? And can we take this beyond apple juice?”
At first glance the dopamine system might seem largely irrelevant to the study of human behavior. Haven’t we evolved beyond the brutish state of “reward harvesting,” where all we care about is food and sex? Dopamine might explain the simple psychology of a lizard, or even a monkey sipping juice, but it seems a stretch for it to explain the Promethean mind of a human. “One of the distinguishing traits of human beings is that we chase ideas, not just primary rewards,” Montague says. “What other animal goes on hunger strike? Or abstains from sex? Or blows itself up in a cafe in the name of God?” These unique aspects of human cognition seem impossible to explain with neurons that track and predict rewards. After all, these behaviors involve the rejection of rewards: We are shrugging off tempting treats because of some abstract belief or goal.
Montague’s insight, however, was that ideas are just like apple juice. From the perspective of the brain, an abstraction can be just as rewarding as the tone that predicts the reward. Evolution essentially bootstrapped our penchant for intellectual concepts to the same reward circuits that govern our animal appetites. “The guy who’s on hunger strike for some political cause is still relying on his midbrain dopamine neurons, just like a monkey getting a treat,” Montague says. “His brain simply values the cause more than it values dinner.” According to Montague, the reason abstract thoughts can be so rewarding, is that the brain relies on a common neural currency for evaluating alternatives. “It’s clear that you need some way to compare your options, even if your options come from very different categories,” he says. By representing everything in terms of neuron firing rates, the human brain is able to choose the abstract thought over the visceral reward, as long as the abstraction excites our cells more than apple juice. That’s what makes ideas so powerful: No matter how esoteric or ethereal they get, they are ultimately fed back into the same system that makes us want sex and sugar. As Montague notes, “You don’t have to dig very far before it all comes back to your loins.”
Complexity derived from simplicity
Montague, as he had grown frustrated with the limitations of the regural fMRI he inveted an fMRI where the participants could communicate with each other. He used this to perform simple experiments on economics.
And so Montague pioneered a technique known as hyper-scanning, allowing subjects in different fMRI machines to interact in real time. His experiment revolved around a simple economic game in which getting the maximum reward required the strangers to trust one another. However, if one of the players grew especially selfish, he or she could always steal from the pot and erase the tenuous bond of trust. By monitoring the players’ brains, Montague was able to predict whether or not someone would steal money several seconds before the theft actually occurred. The secret was a cortical area known as the caudate nucleus, which closely tracked the payouts from the other player. Montague noticed that whenever the caudate exhibited reduced activity, trust tended to break down.
But what exactly is the caudate computing? How do we decide whom to trust with our money? And why do we sometimes decide to stop trusting those people? It turned out that the caudate worked just like the reward cells in the monkey brain. At first the caudate didn’t get excited until the subjects actually trusted one another and garnered their separate rewards. But over time this brain area started to expect trust, so that it fired long before the reward actually arrived. Of course, if the bond was broken — if someone cheated and stole money — then the neurons stopped firing; social assumptions were proven wrong. (Montague is currently repeating this experiment with a collaborating lab in China so that he can detect the influence of culture on social interactions.) The point, he says, is that people were using this TDRL strategy — a strategy that evolved to help animals find caloric rewards — to model another mind. Instead of predicting the arrival of juice, the neurons were predicting the behavior of someone else’s brain.
The the article describes the revision of his theory as well as a very interesting experiment
A few years ago, Montague was reviewing some old papers on TDRL theory when he realized that the system, while effective and efficient, was missing something important. Although dopamine neurons excelled at measuring the mismatch between their predictions of rewards and those that actually arrived — these errors provided the input for learning — they’d learn much quicker if they could also incorporate the prediction errors of others. Montague called this a “fictive error learning signal,” since the brain would be benefiting from hypothetical scenarios: “You’d be updating your expectations based not just on what happened, but on what might have happened if you’d done something differently.” As Montague saw it, this would be a very valuable addition to our cognitive software. “I just assumed that evolution would use this approach, because it’s too good an idea not to use,” he says.
The question, of course, is how to find this “what if” signal in the brain. Montague’s clever solution was to use the stock market. After all, Wall Street investors are constantly comparing their actual returns against the returns that might have been, if only they’d sold their shares before the crash or bought Google stock when the company first went public.
The experiment went like this: Each subject was given $100 and some basic information about the “current” state of the stock market. After choosing how much money to invest, the players watched nervously as their investments either rose or fell in value. The game continued for 20 rounds, and the subjects got to keep their earnings. One interesting twist was that instead of using random simulations of the stock market, Montague relied on distillations of data from famous historical markets. Montague had people “play” the Dow of 1929, the Nasdaq of 1998, and the S&P 500 of 1987, so the neural responses of investors reflected real-life bubbles and crashes.
This is called neuroeconomics, that is, the study of neuronal procesess when the brain performs economic decisions. It is a much disputed new-found discipline, but I believe it to be part of what will be the core of cognitive science in the next decades. Montague describes some exciting results
The scientists immediately discovered a strong neural signal that drove many of the investment decisions. The signal was fictive learning. Take, for example, this situation. A player has decided to wager 10 percent of her total portfolio in the market, which is a rather small bet. Then she watches as the market rises dramatically in value. At this point, the regret signal in the brain — a swell of activity in the ventral caudate, a reward area rich in dopamine neurons — lights up. While people enjoy their earnings, their brain is fixated on the profits they missed, figuring out the difference between the actual return and the best return “that could have been.” The more we regret a decision, the more likely we are to do something different the next time around. As a result investors in the experiment naturally adapted their investments to the ebb and flow of the market. When markets were booming, as in the Nasdaq bubble of the late 1990s, people perpetually increased their investments.
But fictive learning isn’t always adaptive. Montague argues that these computational signals are also a main cause of financial bubbles. When the market keeps going up, people are naturally inclined to make larger and larger investments in the boom. And then, just when investors are most convinced that the bubble isn’t a bubble — many of Montague’s subjects eventually put all of their money into the booming market — the bubble bursts. The Dow sinks, the Nasdaq collapses. At this point investors race to dump any assets that are declining in value, as their brain realizes that it made some very expensive prediction errors. That’s when you get a financial panic.
Neuroeconomics, the study of how the brain determines economic behavior
Remember what I had said in previous articles that what is most important in science is the structure, rather than the content? Read these findings to see how a simple neuronic model, that in the above example explained the economic behavior, now explains other social behaviors. In that example we have substance addiction.
Such fictive-error learning signals aren’t relevant only for stock market investors. Look, for instance, at addiction. Dopamine has long been associated with addictive drugs, such as cocaine, that overexcite these brain cells. The end result is that addicts make increasingly reckless decisions, forgoing longterm goals for the sake of an intensely pleasurable short-term fix. “When you’re addicted to a drug, your brain is basically convinced that this expensive white powder is worth more than your marriage or life,” Montague says. In other words addiction is a disease of valuation: Dopamine cells have lost track of what’s really important.
Montague wanted to know which part of the dopamine system was distorted in the addicted brain. He began to wonder if addiction was, at least in part, a disease of fictive learning. Addicted smokers will continue to smoke even when they know it’s bad for them. Why can’t they instead revise their models of reward?
Last year Montague decided to replicate his stock market study with a large group of chronic smokers. It turned out that smokers were perfectly able to compute a “what if” learning signal, which allowed them to experience regret. Like nonsmokers they realized that they should have invested differently in the stock market. Unfortunately, this signal had no impact on their decision making, which led them to make significantly less money during the investing game. According to Montague, this data helps explain why smokers continue to smoke even when they regret it. Although their dopamine neurons correctly compute the rewards of an extended life versus a hit of nicotine — they are, in essence, asking themselves, “What if I don’t smoke this cigarette?” — their brain doesn’t process the result. That feeling of regret is conveniently ignored. They just keep on lighting up.
So then, the article closes supporting the same opinion I had expressed in Neurons, politics and economics
Montague exudes the confidence of a scientist used to confirming his hypotheses. He buzzes with ideas for new experiments — “ I get bored rather easily,” he says — and his lab is constantly shifting direction, transitioning from dopamine to neuroeconomics to social neuroscience. Montague is currently consumed with questions about how people interact when they’re part of a group. “A mob or a market is not just a collection of discrete individuals,” he says. “It’s something else entirely. You would do things in a group that you would never do by yourself. But what’s happening in your brain? We’ve got all these sociological studies but no hard data.” Montague’s been warned that the project is too complicated, that social interactions are too subtle and complex to pick up in a scanner, but he’s convinced otherwise. “If I’d listened to the naysayers,” he says, “I’d still be studying honeybees.”
Montague’s experiments take advantage of his unique fMRI setup. He has four people negotiate with one another as they decide how much to offer someone else during an investing game. While the group is bickering, Montague is monitoring the brain activity of everyone involved. He’s also infiltrated the group with a computer player that has been programmed to act just like a person with borderline personality disorder. The purpose of this particular experiment is to see how “one bad apple” can lead perfect strangers to also act badly. While Montague isn’t ready to share the results — he’s still gathering data — what he’s found so far is, he says, “stunning, shocking even…. For me the lesson has been that people act very badly in groups. And now we can see why.”
Such exuberance is well earned. In the space of a few short years, Montague has taken his theoretical model of learning — a model he borrowed from some old computer science textbooks — and shown that it’s an essential part of the human brain. He’s linked the transactions of a single neurotransmitter to a dizzying array of behaviors, so that it’s now possible to draw a straight line between monkeys craving juice and stock market bubbles. A neurotransmitter that wasn’t supposed to matter is now our most important clue into the secret messages of the mind and the breakdown of social graces. The code hasn’t been broken. But for the first time, it’s getting cracked.
Dr. Read Montague
So to recapitulate what we just presented (or to present it to those of you who were too bored to take a look at the article 
)
1. Read Montague, director of the Human Neuroimaging Lab at Baylor College of Medicine in downtown Houston, studies the brain from a theoritical perspective, that’s it, how it should work.
2.Wolfram Schultz, a neuroscientist at Cambridge University, discovers how dopamine takes part not only in the reward system in the brain, but also, in the system that predicts rewards.
3.Read Montague and Peter Dayan discover how a model called temporal difference reinforcement learning (TDRL) can explain Scultz’s findings.
4.The research community was not very fond of their views. However, they managed to publish their work on smaller journals.
5.In 1997 they publish a paper, along with Schultz whose short title was audaciously grand: “A Neural Substrate of Prediction and Reward.” The paper has since been cited more than 1,200 times, and it remains the definitive explanation of how the brain parses reality into a set of accurate expectations, which are measured out in short bursts of dopamine. A crucial part of the cellular code had been cracked.
6.Now Montague performs research on how the dopaminergic system can participate in other actions, leading to the creation of complex and larger systems, which until now they could not be studied at a level so close to the bottom (following the reductionist paradigm) as that of neuroscience. Examples include: simple economic behavior, stock market, substance addiction.
7.The article closes with the final quote which finds me in complete agreement:
Montague’s been warned that the project is too complicated, that social interactions are too subtle and complex to pick up in a scanner, but he’s convinced otherwise. “If I’d listened to the naysayers,” he says, “I’d still be studying honeybees.”
Complex systems arise from a few basic principles. There are many things we have to mention if we are to dig into this concept, but just for now, know that complex systems constitute the core of our world. Every science, from physics to economics, one way or another, deals with complex systems to some degree. I believe that the core problems that psychology poses deal only with complex systems. However, the limited scientific tools we had until now, limited our vision, data and theories. Montague’s work proves that for the first time, we can start to study psychological phenomena at their most basic level and then see the effects of this most basic level to the higher ones. I can only say that the next decades will provide even more exciting data that, hopefully, will start to lead us to the beginning of a unified theory of psychology.
