Tag Archives: Evolutionary Psychology

A P-Curve Exercise That Might Restore Some of Your Faith in Psychology

I teach my university’s Graduate Social Psychology course, and I start off the semester (as I assume many other professors who teach this course do) by talking about research methods in social psychology. Over the past several years, as the problems with reproducibility in science have become more and more central to the discussions going on in the field, my introductory lectures have gradually become more dismal. I’ve come to think that it’s important to teach students that most research findings are likely false, that there is very likely a high degree of publication bias in many areas of research, and that some of our most cherished ideas about how the mind works might be completely wrong.

In general, I think it’s hard to teach students what we have learned about the low reproducibility of many of the findings in social science without leaving them with a feeling of anomie, so this year, I decided to teach them how to do p-curve analyses so that they would at least have a tool that would help them to make up their own minds about particular areas of research. But I didn’t just teach them from the podium: I sent them away to form small groups of two to four students who would work together to conceptualize and conduct p-curve analysis projects of their own.

I had them follow the simple rules that are specified in the p-curve user’s guide, which can be obtained here, and I provided a few additional ideas that I thought would be helpful in a one-page rubric. I encouraged them to make sure they were sampling from the available population of studies in a representative way. Many of the groups cut down their workload by consulting recent meta-analyses to select the studies to include. Others used Google Scholar or Medline. They were all instructed to follow the p-curve manual chapter-and-verse, and to write a little paper in which they summarized their findings. The students told me that they were able to produce their p-curve analyses (and the short papers that I asked them to write up) in 15-20 person-hours or less. I cannot recommend this exercise highly enough. The students seemed to find it very empowering.

This past week, all ten groups of students presented the results of their analyses, and their findings were surprisingly (actually, puzzlingly) rosy: All ten of the analyses revealed that the literatures under consideration possessed evidentiary value. Ten out of ten. None of them showed evidence for intense p-hacking. On the basis of their conclusions (coupled with the conclusions that previous meta-analysts had made about the size of the effects in question), it does seem to me that there really is license to believe a few things about human behavior:

(1) Time-outs really do reduce undesirable behavior in children (parents with young kids take notice);

(2) Expressed Emotion (EE) during interactions between people with schizophrenia and their family members really does predict whether the patient will relapse in in the successive 9-12 months (based on a p-curve analysis of a sample of the papers reviewed here);

(3) The amount of psychological distress that people with cancer experience is correlated with the amounts of psychological distress that their caregivers manifest (based on a p-curve analysis of a sample of the papers reviewed here);

and

(4) Men really do report more distress when they imagine their partners’ committing sexual infidelity than women do (based on a p-curve analysis of a sample of the papers reviewed here; caveats remain about what this finding actually means, of course…)

I have to say that this was a very cheering exercise for my students as well as for me. But frankly, I wasn’t expecting all ten of the p-curve analyses to provide such rosy results, and I’m quite sure the students weren’t either. Ten non-p-hacked literatures out of ten? What are we supposed to make of that? Here are some ideas that my students and I came up with:

(1) Some of the literatures my students reviewed involved correlations between measured variables (for example, emotional states or personality traits) rather than experiments in which an independent variable was manipulated. They were, in a word, personality studies rather than “social psychology experiments.” The major personality journals (Journal of Personality, Journal of Research in Personality, and the “personality” section of JPSP) tend to publish studies with conspicuously higher statistical power than do the major journals that publish social psychology-type experiments (e.g., Psychological Science, JESP and the two “experimental” sections of JPSP), and one implication of this fact, as Chris Fraley and Simine Vazire just pointed out is that the former set of experiment-friendly journals are more likely, ceteris paribus, to have higher false positive rates than is the latter set of personality-type journals.

(2) Some of the literatures my students reviewed were not particularly “sexy” or “faddish”–at least not to my eye (Biologists refer to the large animals that get the general public excited about conservation and ecology as the “charismatic megafauna.” Perhaps we could begin talking about “charismatic” research topics rather than “sexy” or “faddish” ones? It might be perceived as slightly less derogatory…). Perhaps studies on less charismatic topics generate less temptation among researchers to capitalize on undisclosed researcher degrees of freedom? Just idle speculation…

(3) The students went into the exercise without any a priori prejudice against the research areas they chose. They wanted to know whether the literatures the focused on were p-hacked because they cared about the research topics and wanted to base their own research upon what had come before–not because they had read something seemingly fishy on a given topic that gave them impetus to do a full p-curve analysis. I wonder if this subjective component to the exercise of conducting a p-curve analysis is going to end up being really significant as this technique becomes more popular.

If you teach a graduate course in psychology and you’re into research methods, I cannot recommend this exercise highly enough. My students loved it, they found it extremely empowering, and it was the perfect positive ending to the course. If you have used a similar exercise in any of your courses, I’d love to hear about what your students found.

By the way, Sunday will be the 1-year anniversary of the Social Science Evolving Blog. I have appreciated your interest.  And if I don’t get anything up here before the end of 2014, happy holidays.

Advertisements

The Trouble with Oxytocin, Part III: The Noose Tightens for The Oxytocin–>Trust Hypothesis

https://i0.wp.com/media-cache-ak0.pinimg.com/736x/2b/1f/9b/2b1f9b4e930d47f31b1f7f3aecd0b0cf.jpgMight be time to see about having that Oxytocin tattoo removed…

When I started blogging six months ago, I kicked off Social Science Evolving with a guided tour of the evidence for the hypothesis that oxytocin increases trusting behavior in the trust game (a laboratory workhorse of experimental economics). The first study on this topic, authored by Michael Kosfeld and his colleagues, created a big splash, but most of the studies in its wake failed to replicate the original finding. I summarized all of the replications in a box score format (I know, I know: Crude. So sue me.) like so:

Box Score_Dec2013By my rough-and-ready calculations, at the end of 2013 there were about 1.25 studies’ worth of successful replications of the original Kosfeld results, but about 3.75 studies’ worth of failed replications (see the original post for details). Even six months ago, the empirical support for the hypothesis that oxytocin increases trust in the trust game was not looking so healthy.

I promised that I’d update my box score as I became aware of new data on the topic, and a brand new study has just surfaced. Shuxia Yao and colleagues had 104 healthy young men and women play the trust game with four anonymous trustees. One of those four trustees (the “fair” trustee) returned enough of the subject’s investment to cause the subject and the trustee to end up with equal amounts of money; the other three trustees (designated as the “unfair players”) declined to return any money to the subject at all.

Next, subjects were randomly assigned to receive either the standard dose of intranasal oxytocin, or a placebo. Forty-five minutes later, participants were told that they would receive an instant message from the four players to whom they had entrusted money during the earlier round of the trust game. The “fair” player from the earlier round, and one of the “unfair” players, sent no message at all. The second unfair player sent a cheap-talk sort of apology, and the third unfair player offered to make a compensatory monetary transfer to the subject that would make their payoffs equal.

Finally, study participants took part in a “surprise” round of the trust game with the same four strangers. The researchers’ key question was whether the subjects who had received oxytocin would behave in a more trusting fashion toward the four players from Round 1 than the participants who received a placebo instead.

They didn’t.

In fact, the only hint that oxytocin did anything at all to participants’ trust behaviors was a faint statistical signal that oxytocin caused female participants (but not male participants) to treat the players from Round 1 in a less trusting way. If anything, oxytocin reduced women’s trust. I should note, however, that this females-only effect for oxytocin was obtained using a statistically questionable procedure: The researchers did not find a statistical signal of an interaction between oxytocin and subjects’ sex, and without such a signal, their separation of the men’s and the women’s data for further analyses really wasn’t licensed. But regardless, the Yao data fail to support the idea that oxytocin increases trusting behavior in the trust game.

It’s time to update the box score:

Box_Score_Jun2014

In the wake of the original Kosfeld findings, 1.25 studies worth of results have accumulated to suggest that oxytocin does increase trust in the trust game, but 4.75 studies worth of results have accumulated to suggest that it doesn’t.

It seems to me that the noose is getting tight for the hypothesis that intransasal oxytocin increases trusting behavior in the trust game. But let’s stay open-minded a while longer. As ever, if you know of some data out there that I should be including in my box score, please send me the details. I’ll continue updating from time to time.

David Brooks Declares: “Evolutionary Biology and the Other Axis Sciences Declare War on Character and Depth!”

You know how Bill O’Reilly has made a yearly habit of declaring that the godless liberals who run America’s public institutions have declared war on Christmas? Reading David Brooks’s piece in yesterday’s (March 14, 2014) New York Times, The Deepest Self, I wondered whether Brooks had taken a page out of O’Reilly’s playbook. No, Brooks isn’t fretting about the demise of nativity scenes on courthouse lawns or bans on Have Yourself a Merry Little Christmas. Instead, Brooks is worried about the decline of character, and of depth, which he defines (a little obliquely, I think), as the possession of “permanent convictions about fundamental things…a web of unconditional loves…and permanent commitments to transcendent projects that cannot be completed in a single lifetime.

Brooks has met the enemy of character and depth: It’s a scientific axis of evil that includes “evolutionary biology” (by the way, when did “evolutionary biology” start sounding cooler to Opinion columnists than “evolutionary psychology?”), “the chemistry and biology of love and sex,” and “psychology and the cognitive sciences.” Together, these Axis scientific powers are pushing a view of human nature that is crowding “character” and “depth” out of the marketplace of ideas:

[T]he strictly evolutionary view of human nature sells humanity short. It leaves the impression that we are just slightly higher animals—thousands of years of evolutionary processes capped by a thin layer of rationality. It lops off entire regions of human possibility.

And, according to Brooks, that’s just wrong:

In fact, while we are animals, we have much higher opportunities. While we start with and are influenced by evolutionary forces, people also have the chance to make themselves deep in a way not explicable in strictly evolutionary terms.

Wow. This sounds bad indeed. Character is important! Depth is important! If evolutionary psychology and the other axis sciences are pushing a view of human nature that marginalizes character, then Brooks is right to be concerned.

But they aren’t, so Brooks isn’t.

First of all, Brooks is wrong when he accuses evolutionary psychology of drawing a profound contrast between the throbbing mass of irrational, unconscious, animalistic impulses and the thin veneer of rationality that gets slapped on top of that jumble. In his indictment, Brooks accuses evolutionary psychology and the other axis sciences of pushing the following view of human nature:

Deep in the core of our being, there are the unconscious natural processes built in by evolution. These deep unconscious processes propel us to procreate or strut or think in certain ways, often impulsively. Then, at the top, we have our conscious, rational processes. This top layer does its best to exercise some restraint and executive function…This evolutionary description has become the primary way we understand ourselves. Deep down we are mammals with unconscious instincts and drives. Up top there’s a relatively recent layer of rationality.

To me, this sounds like Brooks has mistaken Darwin for Freud. It was Freud, after all, and not Darwin, who bequeathed to us the Id, Ego, and Superego. Evolutionary psychology has bequeathed a view of the mind as an integrated set of reasoning systems that are good at solving problems that ancestral humans faced as our species was becoming modern. According to the evolutionary psychology view of mental processes, there is no mind at war with itself, but rather, a collection of systems that are integrated and interdependent, even if their agendas don’t always coincide for every possible course of action. (If they did, what would be the point in having multiple systems?) Whatever that “relatively recent layer of rationality” is, evolutionary psychology’s view of human nature suggests that it’s going to be good at doing the jobs that natural selection designed it to do, which means that it has to play nicely with the other evolved systems the mind contains.

And even though evolutionary social scientists don’t yet know for sure what those jobs are, they have some decent hypotheses. Humans in every society that has ever existed have formed friendships, sought to cultivate good reputations with their neighbors, tried to master their environments, worked to develop social and human capital (some of Brooks’s other darlings), and sought to share the wisdom they have acquired over a lifetime with old friends and loved ones. Not every human succeeds at these tasks, but most do, to varying degrees. It’s Brooks who is bifurcating human nature into a zoo of untamed, irrational, animalistic impulses and a thin layer of reason that tries, bumblingly, to play zookeeper. Don’t blame evolutionary psychology for a dichotomy between what’s animalistic/natural and what’s rational/restrained, for that’s a dichotomy that evolutionary psychology actively resists.

Second, Brooks pushes a very old and very worn-out dichotomy between “nature” and “development:”

But depth, the core of our being, is something we cultivate over time. We form relationships that either turn the core piece of ourselves into something more stable and disciplined or something more fragmented and disorderly. We begin with our natural biases but carve out depths according to the quality of the commitments we make. Our origins are natural; our depths are man-made — engraved by thought and action.

It’s always nice to have new examples of the nature/nurture fallacy to show the freshman psychology students I teach, so in a way I’m pleased to see Brooks’s fresh recapitulation of that fallacy here. The set of human traits that are natural and the set of human traits that develop over time are not mutually exclusive. In fact, the two sets overlap almost completely. The body’s ability to produce calluses, scars, and darker skin in response to sunlight are all natural. They are also all “things that develop over time.” Learning a language is natural for humans. It’s also something that happens—and something that humans can even “actively cultivate”—over time. The same goes for character. Why is it that humans strive to become better and wiser, whereas our dogs and cats and goldfish (apparently) don’t? Because the urge to develop character is part of human nature, but not part of dog nature, cat nature, or goldfish nature.

Finally—and this is just a hunch—I think Brooks is confusing evolutionary psychology’s (de facto) research agenda with evolutionary psychology’s (actual) view of human nature. What’s really bugging Brooks about evolutionary psychology, I think, is its preoccupation with “the chemistry and biology of love and sex.” The lede to his piece, after all, is a recitation of factoids (drawn, one presumes, from recent research articles) regarding the effects of humans’ sexual appetites on their behavior, perceptions, and judgments.

I get the sense that Brooks finds evolutionary psychology’s overweening emphasis on sex a little trying. On this point, I actually agree with him. Sex has become to evolutionary psychology what Fire and Rain is to a James Taylor concert. Even so, evolutionary psychologists also study things like trust, cooperation, generosity, gratitude, forgiveness, reconciliation, moral judgments, reputation, culture, reasoning, empathy, language, music, memory, the development of expertise, and even play. Are concepts like these relevant to Brooks’s definition of character? I hope so: They’re relevant to everybody else’s. Granted, you won’t find any scientific talks on “depth” at an annual meeting of the Human Behavior and Evolution Society, but can you imagine hearing anything interesting on the topic of “depth” at any scientific conference?

Brooks’s fretting strikes me as the sort of thing people write when they’re worried that science might actually succeed in explaining something important about human nature. But “explaining things” is what scientists are supposed to be doing. As Steven Pinker recently wrote, “The mindset of science [is] indispensable in all areas of human concern, including politics, the arts, and the search for meaning, purpose, and morality.” In my opinion, you can add character to Pinker’s list without reducing its accuracy in the slightest.

Brooks is obviously free to continue admiring how people acquire permanent convictions about fundamental things, webs of unconditional loves, and permanent commitments to transcendent projects that cannot be completed in a single lifetime,” but some of us actually want to understand the pillars of human character, not just ooh and ahh about them and yearn for a time when evolutionary psychology and the other axis sciences weren’t butting into conversations in which they weren’t welcome. And despite what Brooks or anybody else might lead you to believe, as far as character is concerned, evolutionary psychology is trying to make love, not war.