Pervasive misuse of data, inadequate quantitative metrics, and disregard for qualitative explanations have undermined scientific research and made the results frail (Ioannidis, 2015; Leek & Peng, 2015; Leung, 2015; Nielson, 2004)
A recent paper titled The association between early career informal mentorship in academic collaborations and junior author performance exploring mentoring, gender, and career outcomes by Nature Communications brought into focus the pervading sexism that continues to exist in STEM research. The study evaluates apparent markers of career success (drawn from publication data) and the gender of senior co-authors to predict a junior co-author’s career as reflected by subsequent citation rates. They found that women who publish with women are less likely to be cited, and based on this finding, went on to propose that women mentees should avoid working with or being mentored by other women. After the furor that erupted that had scientists critically examining the paper, the consensus is that the paper doesn’t tell us much about the impact of gender on mentorship, but it sure does tell us that the statistics community needs to do a better job at teaching scientists about correlation, causation, and confounding variables!
Severe methodological concerns were overlooked in this publication. Operational definitions of the primary variables “mentorship” and “career success” were narrow and dependent exclusively on co-authorship. While ideal mentorship involves catalyzing a holistic development, arising from psychosocial support, professional and intellectual growth, and guidance towards pursuing several career trajectories, the narrow metric does not account for nor casually predict career outcomes. Not to forget, the prevalent gender-based citation disparity was not controlled for, and therefore the citation metrics used to define success fails to present objective findings!
Numerous studies have found that women in STEM publish less, are paid less for their research, and do not progress as far as men in their careers. They are predominantly mentored by men, with only the recent trend showing a snail’s pace progress for available women mentors. Additionally, women leave STEM careers at disproportionately higher rates than men, particularly those who happen to be working parents (Frank, 2019). Systems of bias that push women out of careers can skew results related to gender-based disparities. However, these findings fail to address the “why” and qualitative aspects of this research question. The presentation of the findings and proposed recommendations further perpetuates the effect. Besides, it poses a further question of “If remaining as mentees are the only plausible career trajectory, why retain women in STEM?”
Beyond the overtly biased interpretation of data, the study highlights the fundamental questions about scientific research. These results do not provide novel insights about the accumulated advantage or disparities for under-represented individuals in science. Across the field, longstanding gender disparities have favored men -- with systemic factors, interpersonal biases, and preferences perpetuating it. The scientific community is more valid, innovative, and impactful when diversity is acknowledged and viewed as catalyzing growth. The recommendations of the authors do not elevate this status but rather counters the dire needs in existence (in case of accurate results), and ostensibly proposes sexism as a solution. This also blinks on whether or not the publisher should gauge the impact of their published findings. As science inches towards truth, it may not be to everyone’s liking. If the objective ‘truth’ arrived at in the stated paper implies poor value through the influence of women’s mentorship, how does this lend itself to our subjective perceptions? On the tide towards equality, would subjectivity misguide truth’s intent? These are indeed troubling questions.
While the conclusions are based on false assumptions and false analysis, the data used doesn’t account for diversity either. The use of data mining to define success and mentorship shows the redundancy within the sciences to categorize human behavior into small subsets. The wide data set to define and capture human behavior was not developed nor sufficient data sets were considered to report findings. Studies with similar methodological faults pose questions of whether or not all types of data are equal and/or if all scientific disciplines could follow a standardized method to arrive at objective findings. Without addressing it, how meaningful are these studies?
The real revolution for scientific research and publication will happen when peer review is no more the work of 3 or 4 people. Papers need to be judged post-publication via online audience. Editors along with the authors will curate and take the risk during reviewing post-publication. The eLife Journal has recently transitioned to a new model based on author-driven publishing (preprints) and public post-publication peer review and curation. The traditional “review, then publish” is replaced by a promising “publish, then review” model directed towards a public assessment of “refereed preprints”. If embraced rightly, this would provide a platform for eventual open research communication. A monumental leap for scientific progress across disciplines!
(Disclaimer: Any views or opinions represented in this blog article are personal and belong solely to the author, and do not represent those of people, institutions or organizations that the author may be associated with in professional or personal capacity, unless explicitly stated.)
References
AlShebli, B., Makovi, K., & Rahwan, T. (2020). RETRACTED ARTICLE: The association between early career informal mentorship in academic collaborations and junior author performance. Nature communications, 11(1), 1-8. https://doi.org/10.1038/s41467-020-19723-8
Catalyst (2020, August 4). Quick Take: Women in Science, Technology, Engineering, and Mathematics (STEM). Retrieved from https://www.catalyst.org/research/women-in-science-technology-engineering-and-mathematics-stem/
Frank, K. (2019). A Gender Analysis of the Occupational Pathways of STEM Graduates in Canada. Analytical Studies Branch Research Paper Series. Statistics Canada. Retrieved from https://files.eric.ed.gov/fulltext/ED600827.pdf
Ioannidis, J. P. (2005). Why most published research findings are false. PLoS medicine, 2(8), e124. https://doi.org/10.1371/journal.pmed.0020124
Leek, J. T., & Peng, R. D. (2015). Statistics: P values are just the tip of the iceberg. Nature, 520(7549), 612-612. https://doi.org/10.1038/520612a
Leung, L. (2015). Validity, reliability, and generalizability in qualitative research. Journal of family medicine and primary care, 4(3), 324. https://dx.doi.org/10.4103/2F2249-4863.161306
Nielson, J. (2004, February 29). Risks of Quantitative Studies. Nielsen Norman Group. Retrieved from https://www.nngroup.com/articles/risks-of-quantitative-studies/
UNESCO (2021). Women in Science. UNESCO Institute of Statistics. Retrieved from http://uis.unesco.org/en/topic/women-science