By Megan Boreham
Last November, Nature Communications published one of their most controversial and divisive articles to date (AlShebli et al. 2020a). The research, authored by three researchers at New York University’s Abu Dhabi campus, aimed to investigate the value of informal mentorship during academic collaboration. Their analysis of over 3 million mentor-protégé pairs led the researchers to conclude that “policies promoting female-female mentorships, as well-intended as they may be, could hinder the careers of women who remain in academia.” Unsurprisingly, the publication incited a flood of backlash across social media from academics and students alike, culminating in the paper’s swift retraction on 21st December 2020.
”Let me be blunt: for the good of the global scientific community and for the reputation of Nature Communications, you must retract this paper.” The opening statement of the open letter penned by Dr Leslie B. Vosshall, Robin Chemers Neustein Professor at Rockefeller University and investigator at Howard Hughes Medical Institute, to the Nature Communications’ Editor in Chief in response to the publication, provides a poignant demonstration of the sheer intensity of the criticism the paper received (Vosshall, 2020).
Vosshall went on to say that “the conclusions reached by the authors that being mentored by a female scientist is detrimental to young scientists, particularly female scientists is based on flawed assumptions and flawed analysis.” In response to the letter and further complaints across social media platforms, the prestigious journal added an editor’s note to the paper, altering readers that, “this paper is subject to criticisms that are being considered by the editors. Those criticisms were targeted to the authors’ interpretation of their data that gender plays a role in the success of mentoring relationships between junior and senior researchers, in a way that undermines the role of female mentors and mentees” (AlShebli et al 2020b).
What the study tells us
The paper, authored by AlShebli et al, aimed to evaluate the benefits of mentor-protégé relationships between junior academics and senior mentors, by analysing over 3 million relationships. Academics were classified by their academic age, defined as the years since the scientists first publication, under 7 years and they were considered junior – anything over counted as senior.
A mentor-protégé relationship was established whenever a junior scientist published a paper with a senior scientist, as long the mentor had co-authored at least one other paper with less than 20 total authors and shared the same scientific discipline and USA academic institution as the junior. To assess the ‘impact’’ of the mentors prior to the mentorship, the authors determined the average number of yearly citations they received. This value was then used to classify the extent to which the mentor was considered a “big shot” in their field. Additionally, they quantified the amount of collaborative research the mentors were previously involved with in the first year before the mentorship began. This “hub experience” was measured as the degree of involvement within the mentor’s collaborative network. The “success” outcomes of the protégés were measured in similar ways, during their senior years once the examined mentorship had appeared to cease.
The authors’ subsequent analysis showed that being involved with a “big shot” mentor was more valuable to junior researchers compared to being mentored by those with a large “hub experience”. Being mentored by “big shots” was significantly associated with an increase in protégé success of up to 35%, compared to 13% for “hubs”. Focusing their subsequent analysis on the “big shot” mentorships only, the authors showed that having more female mentors was associated with a decrease of up to 35% in protégé outcomes. These findings were especially legitimate for female-female mentorships.
Interestingly, this apparent damage seems to run both ways. AlShebi concluded that female mentors damaged their prospective gains from mentorship when mentoring female scientists compared to males, finding that female mentors lost an average of 18% citations on their paper when they mentored other women. To add injury to insult, male mentors did not appear to be significantly affected in this manner.
The backlash begins
Almost immediately after Nature Communications released the paper, outrage broke out across social media. On Twitter, Joshua Miller, a postdoctoral associate and conservation genomist at the University of Alberta, voiced his concerns over the publication of the research, stating, “Nature Communications needs to do better” (Miller, 2020).
So what exactly were the critics replying to Joshua’s initial tweet concerned over? Criticism of the paper stemmed from two main concerns; the methodological approaches used, followed by the subsequent interpretations and suggestions drawn by the authors.
Using citation number alone as a measure of academic success has previously been condemned. The reductionist metric fails to account for any of the wealth of other benefits that can arise from mentorship, from soft skills development to perfecting a work-life balance. Furthermore, the paper makes the assumption that co-authorship is equatable to mentorship; anyone with even a year’s experience in academia would be hesitant to agree with this statement. Whilst the authors did try and supplement their data mining with a survey addressing the benefits gained by junior scientists from their mentorships, the survey only returned 167 completed results and no attempt was made to stratify the results by gender.
It is already common knowledge that women in science face a heavy publication bias. One study looking at the field of neuroscience found that “reference lists tend to include more papers with men as first and last authors than would be expected if gender were unrelated to referencing” (Dworkin et al. 2020). By evaluating the merit of female-female mentorships using citation number alone, this publication bias is massively amplified compared to in any other gender pairing, and thus provided a likely explanation for the gendered results.
In order to analyse their 3 million pairs, the authors utilised an automated programme to identify the gender of both mentors and protégés. The software was only successful at identifying 48% of the total names included in the research, with no explanation found in the methods as to how the software handled gender ambiguous names. Using automated software in this manner is a blatant act of non-binary and trans erasure, an issue highly confounded when one realises that the work was published by Nature the day before International LGBTQ+ in STEM day.
As for the data itself, no obvious controls were used for confounding variables such as the scientists actual age, or the seniority or salary of the mentors. Given that women are less likely to be in senior positions than their age-matched male colleagues, it seems ill-advised to ignore these staggering facts. Nonetheless, the researchers did make their data open-source, with AlShelbi noting, “We believe that free inquiry and debate are engineers of science, and welcome the review launched”. Whether or not the comment extends to the individuals racking their data with such a fine toothed comb remains unclear.
Worryingly, the paper was assessed by 4 independent peer-reviewers prior to publication. Whilst the majority found issues in the data analysis and suggested the authors toned down their causal influences, one reviewer stated, “This is a well done paper. The findings are not terribly surprising, but the analysis is unusually exhaustive.” Many academics believing that these negative reviewer comments were largely ignored by the authors, highlights the issues in the peer reviewing process that is evidently commonplace within top academic journals. “Big shot” researchers are more likely to get preferential reviewer comments due their superiority, and the whole peer review process is rendered redundant if a paper becomes published without the authors even addressing any concerns highlighted in the process.
Despite the errors in the analysis, due to variables highlighted above it is likely that the patterns outlined in the paper sadly remain true. However, the authors failed to adequately address any structural problems that lead to these findings, instead suggesting that “female scientists, in fact, may benefit from opposite-gender mentorships”. Furthermore, the authors suggest their findings offer a “new perspective to the policy debate on how to best elevate the status of women in science.” But is a paper suggesting that young female academics would benefit more from being mentored by a white, male, “big shot” colleague really news to anyone?
Responding to the defamation
Many individuals responded to the wealth of criticism by suggesting that retracting the paper allows scientists to shy away from the uncomfortable findings, but instead suggest that the publication should be used as a springboard to begin investigating why these mentorship schemes may be detrimental for science. Only by looking at the data can the world of academia understand how best to promote, and crucially support, female scientists. As the paper was authored by 2 women, many implied that retracting the paper silences the female researchers and therefore feeds into the issue at hand, despite the findings fitting with the general consensus that male authored papers receive more citations than female authored counterparts.
This apparent “silencing” ties in with the “Queen Bee” phenomenon, in which senior female professionals are perceived to fail to help, or even actively hold back their female subordinates. Tania A. Reynolds, assistant professor of psychology at the University of New Mexico, tweeted that “Many people are calling to have this paper retracted, but these findings are quite in line with extant work on female-female competition” (Reynolds, 2020a) . She went on to say she would “save retractions for cases where there are data fraud or coding errors rendering the results invalid”, stressing that retracting the work does nothing in helping the women affected by these structural barriers and instead buries the problem away from the community (Reynolds, 2020b).
Nonetheless, the authors do acknowledge the benefits that female-female mentorships may provide junior researchers, such as “ increasing the likelihood of female protégés staying in academia” (Gaule and Piacentini, 2018) and providing them with increases in career outcomes (Blau et al, 2010).. The authors also justify the rationale for their study by pointing out that studies such as those above only compare mentees with a female mentor to those without a mentor, thus highlighting that their research fills an important gap in determining the true value of female-female mentorships.
Despite the exaggerated recommendation for policy makers to “revisit first and second order consequences of diversity policies”, the authors did briefly attempt to consider the societal factors that may contribute to the gendered results. The authors go on to say, “one potential explanation could be that, historically, male scientists enjoy more privileges and access to resources than their female counterparts, and thus were able to provide more support to their protégés”. However, these alternative explanations, such those mentioned above, are then not touched upon again as these “potential drivers lie outside of the scope of current study”.
Ultimately, it was comments such as these that caused the paper to be subjected to such heavy scrutiny. Although it remains the case that female scientists do not experience as much success when mentored by female “big shots” as their male counterparts, reinforcing the need for reevaluation of information shifts the reality of the situation from a purely unsupported ‘bad science’ to damaging and harmful. Failing to comment in detail on the systemic societal factors that provide an alternative explanation for these findings means the research feeds into the rhetoric that women have not earned their place in STEM. The authors made tenuous conclusions from limited data on a topic of massive importance, and Nature Communications willingly acted as a platform to voice their agended interpretations.
- AlShebli, B. et al. (2020a) RETRACTED ARTICLE: The association between early career informal mentorship in academic collaborations and junior author performance. Nature Communications, 11(5855).
- AlShebli, B. et al. (2020b) Retraction Notice: The association between early career informal mentorship in academic collaborations and junior author performance. Nature Communications, 11(6446).
- Blau, F. et al. (2010). Can Mentoring Help Female Assistant Professors? Interim Results from a Randomized Trial. The American Economic Review, 100(2), pp. 348-352.
- Dworkin, J. et al. (2020) The extent and drivers of gender imbalance in neuroscience reference lists. Nature Neuroscience, 23 pp. 918-926.
- Gaule, P. & Piacentini, M. (2018) An advisor like me? Advisor gender and post-graduate careers in science. Research Policy, 47(4), pp. 805-813.
- Miller, J. (2020) 17 November. Available at https://twitter.com/millerjm86 (Accessed: 3 March 2021).
- Reynolds, T. (2020a) 19 November. Available at https://twitter.com/TaniaArline (Accessed: 3 March 2021)
- Reynolds, T. (2020b). Retracting a controversial paper won’t help female scientists. [online] Quillette.com. Available at: https://quillette.com/2020/11/23/retracting-a-controversial-paper-wont-help-female-scientists/ (Accessed: 3 March 2021).
- Vosshall, L. (2020) 19 November. Available at https://twitter.com/leslievosshall (Accessed: 3 March 2021)