Groundbreaking Study On Gay Marriage Views May Have Been Faked
A seemingly groundbreaking and widely publicized study reported in Science magazine this past December may be a fake.
The study appeared to show that openly gay activists in California had persuaded conservative voters to change their minds in a lasting way by engaging the voters in “heartfelt, reciprocal and vulnerable conversations” about being gay during door-to-door advocacy campaigns. It was co-authored by Michael J. LaCour, a Ph.D. candidate in political science at the University of California, Los Angeles, and Donald P. Green, a professor at Columbia University.
For the gay rights movement, this was good news. It suggested that the country’s shift on gay rights was, at least in part, the movement’s doing, and it provided a template for advocacy going forward. Gay rights advocates in Ireland reportedly based their strategy before a national vote on same-sex marriage this week on LaCour and Green’s results.
But according to a report issued Tuesday by two University of California, Berkeley, graduate students and a Yale professor, there are enough questions about the data to warrant retracting the study. Retraction Watch broke the story Wednesday about what students David Broockman (soon to be an assistant professor at Stanford) and Joshua Kalla and Yale professor Peter Aronow found.
The LaCour-Green study had examined the work of activists with the Los Angeles LGBT Center. After California’s gay marriage ban passed in 2008, activists at the center had more than 12,000 one-on-one conversations in Los Angeles neighborhoods with people who overwhelmingly supported the ban. LaCour’s idea was to see if those conversations produced any lasting change. He purportedly designed a randomized experiment to replicate those conversations, with a series of follow-up surveys online to test how the anti-gay voters felt about gay rights and gay marriage over time. Those who were contacted by the openly gay canvassers showed substantially more positive attitudes toward gay marriage as much as nine months later.
At least, that’s what the published study said in December. But now it appears those critical follow-up surveys may not have been conducted as described.
After the LaCour-Green study was published, Broockman and Kalla were impressed by its findings and wanted to extend the research. In January 2015, they found some patterns in the data that seemed to be too perfect — statistically speaking, there was less variance in the results than there should have been. Some social scientists had noticed this when the study was first published.
As Broockman and Kalla continued their work, they wrote in their report, they uncovered more irregularities. When the pair noticed that their own study had a much lower response rate (the proportion of people contacted who actually respond to a survey), they asked the survey firm that allegedly gathered data for LaCour, Qualtrics, how it achieved such a high response rate. They said the firm replied that it had no record of the project.
This is what happened next according to their report and Green’s letter to Science: The statistical irregularities continued to mount, and the pair recruited Aronow to help with their analysis. Last weekend, Broockman and Kalla contacted Green. Green said that he had joined the study after the data had been collected and thought that the irregularities Broockman and Kalla had uncovered were, indeed, highly suspicious. Green reached out to LaCour’s adviser at UCLA, professor Lynn Vavreck, and the two of them decided that Vavreck would confront LaCour and ask him to provide his data. Initially, LaCour claimed he had accidentally deleted the file with the necessary information, but again Qualtrics said it could not verify that the data had been deleted or that the study took place. It seemed increasingly clear to Green that no follow-up surveys had ever been conducted and that LaCour may have taken data from existing studies and manipulated the numbers to achieve the results he wanted.
Green told The Huffington Post that he was shocked and dismayed by the revelations about the data set. “There was a mountain of fabrication,” he said. “Graphs and charts and anecdotes and stories of every possible sort about these surveys. So it didn’t occur to me that the whole thing was fabricated because every time I had a question, it seemed as though [LaCour] had an answer.”
Green has since issued a retraction of the study.
LaCour, for his part, declined a request for an interview from The Huffington Post. “I’m gathering evidence and relevant information so I can provide a single comprehensive response. I will do so at my earliest opportunity,” he wrote in an email.
How did this happen? Science is a highly regarded, peer-reviewed publication, meaning that other experts review research before it is published to evaluate its quality and contribution to the field.
The Huffington Post confirmed with Science editors that the article in question did go through the magazine’s in-depth review process. This requires at least two people unconnected to the research to read the paper and return comments within a week or two.
“In general, reviewers for Science look at the data presented in submitted manuscripts,” said Monica Bradford, Science executive editor. “Some reviewers might redo analyses to see if they obtain the same results as described by authors. In some rarer cases, referees may also ask for the raw underlying data, which Science obtains and provides for them.”
Bradford said, however, that reviewers aren’t likely to go to the lengths Broockman and Kalla did: “Even if the reviewers had asked for the raw survey data underlying these results, it is unlikely they would have contacted the survey company to confirm it had carried out the surveys and supplied LaCour with the data.”
So while peer review aims to prevent bad research from being published, it is not a perfect safeguard against all possible fraud. Reviewers, editors and even co-authors tend to trust that researchers wouldn’t fake data. However, Science’s requirements for data to be made available ultimately helped uncover the alleged fraud — others were able to access the data and identify major issues.
In a statement Wednesday, Marcia McNutt, editor-in-chief of Science Journals, said, “At this time, our Editorial staff is assessing the report. Given the fact that the [co-author] Dr. Green has requested retraction, Science will move swiftly and take any necessary action at the earliest opportunity. In the meantime, Science is publishing an Editorial Expression of Concern to alert our readers to the fact that serious questions have been raised about the validity of findings in this study.”
A spokesman for the Los Angeles LGBT Center said the center was dismayed by the news that the study results may have been fabricated. “We sought external and independent evaluation of our voter canvassing project to determine the efficacy of the work through unbiased analysis,” said David Fleischer. “We are not in a position to fully interpret or assess the apparent irregularities in the research as we do not have access to the full body of information and, by design, have maintained an arms-length relationship with the evaluation of the project.”
Last December, Green told The Huffington Post that the results of the study were a revelation for him. “I’m more excited about these results than I have been about anything in a long time,” he said. “I used to think that attitudes were stable.”
Green said Wednesday that he is more determined than ever to find out whether the types of conversations conducted by the Los Angeles advocates could indeed produce lasting change in anti-gay voters.
“An actual experiment was conducted, a very nicely designed and elegant experiment, and the irony is that we don’t know what effects the experiment had because the outcomes were never measured,” Green said. “I am determined to do this study again, this time for real.”
— This feed and its contents are the property of The Huffington Post, and use is subject to our terms. It may be used for personal consumption, but may not be distributed on a website.
www.huffingtonpost.com/2015/05/20/gay-marriage-study-fake_n_7344796.html?utm_hp_ref=gay-voices&ir=Gay+Voices
