Friday, September 1, 2017

Revoking PhDs: Questions But No Answers Around the Value of Transparency, Robust Research Communities, and the Construction of Culpability for Senior Researchers

It was reported that a "U of Arizona professor’s Ph.D. is withdrawn after her findings on violent video games are questioned. Some wonder why her mentor and co-author, a senior scholar, has not shared the blame." Colleen Flaherty, "Revoking a Doctorate," Inside Higher Education (September 1, 2017). Ms. Flaherty reports:
Ohio State University took the extraordinary step of revoking a graduate’s doctorate last week. Now her future at the University of Arizona, where she is an assistant professor of communication, is unclear. Jodi Whitaker’s problems started in 2015, after scholars in two countries noticed irregularities in the data in her 2012 paper on video games. The study in Communication Research, called “‘Boom, Headshot!’ Effect of Video Game Play and Controller Type on Firing Aim and Accuracy,” found that playing a violent video game improved real-life shooting skills. Initially, it was something of a boon for both Whitaker, then still a graduate student at Ohio State, and her co-author and dissertation committee chair, Brad J. Bushman, the Margaret Hall and Robert Randal Rinehart Chair of Mass Communication there. That’s because Bushman served on President Obama’s committee on gun violence and his research challenges what he calls myths about violence, including that violent media have a trivial effect on aggression.

But Patrick Markey, a professor of psychology at Villanova University -- whose own findings on video games clash with Bushman’s -- soon challenged the paper, as did Malte Elson, a postdoctoral researcher in educational psychology at Ruhr University Bochum in Germany. Together they alerted the Committee of Initial Inquiry at Ohio State to what they called irregularities in some of the variables of the data set. The values of questioned variables could not be confirmed because the original research records were unavailable, according to Communication Research, which in 2016 decided that a retraction was warranted.

Bushman was cleared of wrongdoing by Ohio State, but he agreed to the retraction. He also agreed to the retraction of another paper in which Whitaker was not involved -- one finding that watching violent cartoons inhibits children's learning -- earlier this year, as reported by Retraction Watch. Data on a second 2016 paper by Whitaker and Bushman (on which Bushman was the lead) also have been corrected; that study found that "catharsis beliefs" attract people to violent video games.

But Whitaker, the 2012 paper’s lead author, was found responsible for the errors. And Ohio State’s Board of Trustees voted unanimously last week to revoke her doctorate, granted in 2013.
With the strong caveat that I am not privy to the evidence adduced nor to the proceedings, here are some initial thoughts on potentially important implications that may flow from this altogether sad events:
With the strong caveat that I am not privy to the evidence adduced nor the proceedings, here are some initial thoughts:

1. The “case” makes a “case” for full transparency in research—especially as the academy moves from centering normative analysis to centering data driven “fact” finding. The anomalies of data, indeed, appear at the heart of the case. From a perspective, at the top of my list would be ensuring full transparency of data used to advance arguments that find their way into peer reviewed print. The Germans I believe are ahead of us on that score. Thus for me the most important sentence in the reporting: “The values of questioned variables could not be confirmed because the original research records were unavailable, according to Communication Research, which in 2016 decided that a retraction was warranted."

2. It is not unusual at all for the younger researcher to be thrown under the bus. The academy appears long to have operated under a principle of sacrificing the “least valuable” to protect the more valuable “asset.” But a strong caveat—it might well be that the difference in treatment is well deserved. We do not know. What is suspicious, of course, is that the advisor also received the “fruits” of research in terms of academic benefits—co-authorship must mean something, and maybe what it should mean is co-responsibility. On the other hand, the academy (especially when we judge junior people coming up for tenure or promotion, or even when we engage in periodic assessment, tend to make almost microscopic distinctions between the efforts of co authors (with sometimes strong effects)). It seems somewhat anomalous to, on the one hand, insist on giving credit grounded on different assessment of involvement in authorship, and, on the other insist even split of responsibility when things go wrong. Still, the academy has made this sort of irrationality (that serves interest) an art form, so it would not surprise me to see policy changes that effectively institutionalize this split (fractional credit when things go well; full responsibility when things go badly). And we will find just the right set of rationalizations to allow us to impose these new policies with a straight face. In this sense this episode will produce a number of buses under which m, more than the protagonists will be thrown—potentially anyway.

3. I await a key marker—either an appeal within the revoking institution or a lawsuit to challenge the revocation. That decision (which it is in the junior professor’s interest to pursue if she means to salvage her reputation and career) may be both a signal about the unfairness of the responsibility (and punishment) differences and about the contest ability of the findings that led to revocation. Too early to tell then when, as here, the story line may not have run its course. In that context it is worth noting that the challenging researchers were reported to have written: ““During the investigation pertaining to the article in question, we discovered two different data files on the senior author’s computer between which the codes for variables were altered. These alterations occurred in a manner which supported the original study’s hypotheses. Additionally, the authors of the original study were unable to provide the raw data in order to confirm which data file was correct.”
4. To some extent the more interesting point was that it was in the market place of academic investigation—where robust interchanges are possible—that the potential wrongdoing was uncovered. What that suggests are two things. First that the processes for the protection of the integrity of the granting of advanced degrees may need some tweaking—perhaps these sorts of problems ought to be caught sooner. On the other hand, that there are so few instances of this sort of scandal may speak well of the system—unless the reason for the rarity of catching this sort of thing is that it is easy to hide. The immediate site for adjustment—the role and function of an examination committee. The problem, that substantially adds to the transaction costs of administering advanced degrees. Second, even where institutional processes may be (as a function of the transaction costs of implementing them) less useful for ferreting out this sort of misconduct, it appears that contests among interested parties may be a more effective means of policing behaviors and conduct that violate peer norms and standards.
5. To go for the gender reflex may speak more to the times than to the case. If what we are worried about are appearances then of course the outcome would have to be different, but as a matter of politics, or of societal expectations. Would one have the same concerns if the gender identities of the parties were reversed (male junior and female senior) etc.? True enough where there is a pattern of gender based decisions, one would have cause for concern. And one must be sensitive to these possibilities. But what might appear to be a gender ad hominem thrown in near the end of the article speaks to many things, possibly, other than gender—starting with the feud among the lead researchers.

6. But beyond the process and legal issues, there are ethical and moral issues here that might be better explored. Beyond the moral and ethical issues of data and research methodologies (already well developing in the academy), there are the ethical and moral issues around the relationship of a graduate student and her supervisors. More interesting still, are the ethical and moral issues that might arise among a supervisory committee and the researcher.  The issue does not run to liability--that would be both too easy and far too much of the distorting domain of the lawyer and the administrator to serve the substantive missions of the academy well. I do not have the answers but suggest that here the nature of such relationships, and the assumptions about the nature of those relationships might be changing. That is clear form the reporting, but also from the reaction.  At the center of that discussion might be differences in ethical duty where a supervising researcher also takes derives benefits of co-authorship.
7. In this context it might be valuable to avoid centering discussion around principles of power relationships. While useful, these tend merely to shift deck chairs on the Titanic, that is they may eventually change behavior rules grounded in power relationships, but in doing so they may merely change power relationships while affirming that power is the foundation of relationships. I tend to find this centering less valuable in the long run as it strips relationships of their complexity and mutability over time. It also unnecessarily reduces the nature of relationship to a patriarchal foundation in ying-yang; passive and active, however does not describe the complexities of the relationships in this case. .  An alternative approach might center discussion on benefit relations rather than power.  That is, the extent of duty and of obligation, might be grounded not just in power but in the extent to which the parties derive value form the relationship.  How and to what extent those changes might be usefully made remains an open issue--and one to which vigorous discussion also might be useful.

That's all I have at the moment. The observations are offered as preliminary thoughts and opinions on the basis of nothing more than the reporting from Inside Higher Education.

No comments:

Post a Comment