Postings on science, wine, and the mind, among other things.
Seeing the Social Future
Pop summary of Thornton, Weaverdyck & Tamir: The social brain automatically predicts others' future mental states. In press at The Journal of Neuroscience.
Walking past your colleague's office, you remember that you had a question for them. You peek in the door, and notice something about their demeanor: they are hunched over their laptop, shoulders tense, brow furrowed, lip bitten, eyes intensely focused on the screen. Do you choose this moment to ask your question? Unless it is terribly pressing, you will probably choose to wait. Why is that? The answer may seem so obvious to you that you wouldn't normally bother to say it: your colleague is clearly busy. You know that busy people tend to react negatively when they are needlessly interrupted, and you wish to maintain a positive relationship with your colleague, so you choose not to interrupt them with your question.
This line of reasoning may appear so blindingly obvious that it doesn't seem to contain much insight. The whole episode might pass in the span of seconds, and barely occupy your conscious mind as it happens. However, this apparent simplicity starts to dissolve into shocking complexity if you think in detail about how all this happens.
First, you must take visual input - patterns of photons reach your retinas - and turn them into representations of the external world: your friend's face, body posture, desk, computer, and so forth. Next, you have to interpret these visual representations: you need to know that lip biting and brow furrowing and an intense stare are signs of this invisible state in your colleague's mind you call "being busy." You then need to think about how people in the state of busyness tend to react to a stimulus: interruption. Finally, you have to evaluate their likely reaction to this stimulus - annoyance - and decide whether it is worth asking your question, given how it will impact your long-term goals, like maintaining good workplace relations. In other words, you have to take in information from the world, process it into something useful, and then use it to predict the likely futures that would result from possible actions.
Humans are a fantastically social species, in large part because our brains make the complex process described above feel effortless. Modern digital computers can easily outperform us in many tasks, such as playing chess. However, no currently existing computer - including those running sophisticated "artificial intelligence" programs like deep neural networks - could come close to performing the sequence of operations described above. These operations amount to little more than basic common sense in the social world, and yet they are supported by a complex and as-yet only dimly understood set of neural processes.
The Predictive Social Mind
My research aims to understand how our brains handle the complexity of the social world, and allow us to make efficient, accurate, and automatic social predictions. The first work in this vein revealed three psychological dimensions - rationality (vs. emotionality), social impact (social, intense states like love or envy vs. others), and valence (positive vs. negative states) - that shape the way our brain represents other people's mental states (i.e., how we think about others' thoughts and feelings).
Subsequently, we found that people can pick up on statistical regularities in emotion transitions - which emotions are likely to precede or follow which others - to make accurate, far-reaching predictions about others' future emotions. Essentially, if you know how someone is feeling now, you have a good clue about how they are likely to be feeling later. The same three psychological dimensions from our earlier work explained much of people's accuracy: the closer any two states were on these dimensions, the more likely a transition was between them. For example, "awe" and "joy" occupy similar coordinates on those three dimensions: both emotional, moderately social impactful, and highly positive. In contrast, "rage" occupies a very different set of coordinates, at the opposite (i.e., negative) end of the valence dimension. Thus we would (rightly) expect that transitions from awe to joy (or vice versa) to be likely, but transitions from either of these states to rage to be very unlikely.
These results led us to propose a framework for modeling the predictive social mind. We suggested that people use psychological dimensions to organize their social knowledge - not just knowledge of mental states, but also of individuals' personality traits, the actions people undertake, or even the situations people find themselves in. By using the right set of dimensions to do this, our brain could get social predictions effectively "for free". That is, if the dimensions which organize all of these different types of social knowledge have the same feature that we discovered in the case of mental states (i.e., that transitions are more likely between closer states) than the current location of (for instance) an action in action space would imply which future actions that person was likely or unlikely to engage in. Thus, merely by encoding a person in terms of their location in these spaces, we could automatically predict the social future, because a person's current position implies their likely future.
To put this another way, you can imagine that your brain makes maps of the social world. The points on these maps do not represent physical locations, but instead social "locations" - for instance, a particular mental state like happiness or sadness. Just like on a map of the world, the distance between two locations tells you something about how hard it is to get from one to another. You probably make lots of trips to locations within 5 or 10 miles from your home, but relatively few trips 50 to 100 miles away. Similarly, a person experiencing sadness might be likely to visit neighboring places on the state-map, like melancholy or ennui, but be much less likely to make it to a distant state like joy in the same time frame. By placing a person's current mental state, (or action, personality, situation, etc.) on such a map, you are thus implicitly predicting where they are more or less likely to "go" in the future. This prediction can then help you navigate your own social world in the best way possible.
Testing the Social Predictive Coding Hypothesis
Our most recently published research - just accepted at The Journal of Neuroscience - tested this theory of predictive social cognition. Participants in the fMRI scanner engaged in the mental state judgment task illustrated in the schematic below. On each trial, they judged to what degree a particular scenario would elicit a given mental state in another person. By averaging over multiple trials with different scenarios but the same state, we could estimate the stable pattern of brain activity that represented each of the 15 mental states in the study.
We analyzed these state-specific activity patterns using a statistical approach called representational similarity analysis (RSA). The basic idea of RSA is simple: you start with some predictions about how similar "things" should be to each other. In this case, those things were mental states, and the predictions came from ratings of the transitional probabilities between them. Next, you measure how similar those things actually are. In this case, we estimated how similar the brain activity patterns were to each other by correlating them: the higher the correlation, the more similar the patterns. Finally, you correlate your predictions about similarity with the actual similarity to see if the predictions bear out in the data. If you're interested in learning more about RSA, you can read this earlier blog post, or check out this tutorial which I taught at the Methods in Neuroscience at Dartmouth (MIND) computational summer school.
What we found using this approach was that neural pattern similarity was indeed correlated with the transitional probabilities between mental states. In other words, if people thought that one state was likely to precede or follow another, then those two states elicited more similar patterns of activity than if people thought the states were unlikely to precede or follow each other. Consistent with our theoretical framework, this indicates that when the brain represents the current state of others' minds, it automatically incorporates a prediction of their likely futures.
We also found that proximity on the three dimensions of mental state representation described earlier in this post statistically explain much of the association between transitional probabilities and neural pattern similarities. That is, proximity between mental states on dimensions such as valence was correlated with both transitional probability ratings, and the neural similarity between mental states, and can statistically account for correlation between these two variables. In other words, the automatic social prediction that we observed can potentially be attributed to how close states are to each other on our social "map" of mental states. This analysis is correlational, and thus does not necessarily imply that the psychological dimensions are causally involved in this predictive coding process, but it does provide circumstantial evidence in favor of this possibility.
Regions of the "social brain" network, independently defined in a separate study, within which we found RSA-based evidence for automatic social prediction.
In addition to the traditional RSA described above, we also developed a new, asymmetric form of RSA. This allowed us to rule out the possibility that the perceived similarity between the mental states - rather than transitional probability - was driving the observed effect. Ratings of similarity were generally highly symmetric (e.g., awe is just as similar to joy as joy is to awe) but the same was not true of the transitions between states, which were sometimes dramatically asymmetric (e.g., you're much more likely to feel tired after feeling energetic than vice versa). Asymmetries in the neural pattern similarity were correlated with asymmetries in the transitional probabilities, but not with the much smaller asymmetries in similarity ratings.
Two other pieces of evidence further supported our conclusion that the social brain automatically predicts others' future mental states. The first piece was the observation that the brain also encodes the long-term frequencies of these same states. These frequencies are useful for social prediction because, even if you don't know what state someone is currently feeling, you can still make a reasonable guess about how they are likely to feel in the future simply by thinking about how often they experience various states. For example, if someone is chronically happy, then even if you don't know how they're currently feeling, you can guess that they'll probably be happy tomorrow. You won't always be right, but you'll be right more often than if you just guess randomly. Since frequencies can be used for social prediction in this way, we would expect the brain to encode them as well as transition probabilities. Conveniently, the long term frequencies of states are implied by the transitions between those states. For example, if people are always transitioning into happiness, then happiness will end up being a frequent state. This fact allowed us to use the transitional probability ratings to predict state frequencies. Conducting another RSA between these states frequencies and neural pattern similarity indicated that the brain does seem to encode how often each state occurs overall. This result further supported our interpretation of brain activity in terms of social prediction.
Finally, the design of our study permitted another, completely different form of analysis called "repetition suppression" or "fMRI adaptation." The logic of repetition suppression is that, if the brain is constantly predicting the future, it should be more active when its predictions are wrong than when they are right. This is because, when it makes a prediction error, it needs to process that error and update its new predictions accordingly. We analyzed the data to identify whether any brain regions showed activity consistent with detecting and updating from prediction errors.
We found one brain region that showed this activity profile, responding more to unanticipated than anticipated sequences: a portion of the posterior precuneus. The precuneus - which is in the back, top, middle of your head - is a bit of a mysterious region because it is implicated in so many different mental functions. In previous work, we and others have found it to be sensitive to thinking about others' mental states. However, it has also been implicated portions in everything from mental imagery (i.e., the "mind's eye") to semantic memory. As a result, it's hard to draw a very strong conclusion from the location of the repetition suppression we observed. However, the fact that at least one portion of the social brain network does respond in this way further supports our hypothesis that the brain automatically predicts others' mental states.
Conclusion
As you've just read, the findings of our latest research support the model of predictive social cognition that we described earlier this year. This model suggests that, by thinking about the social world using certain psychological dimensions, we make automatic predictions about the social future. We are currently extending this research from mental states to actions, to identify which dimensions people might use to make sense of and predict others' behaviors. We are also studying how people make predictions across different "layers" of social cognition. For example, how knowledge of someone's traits may shape the way we predict their mental states. Stand by for more updates, coming soon!
© 2018 Mark Allen Thornton. All rights reserved.