Abstract
-
Background
- Growing evidence for clinically significant differences between the sexes has attracted the attention of researchers. However, failures to report a test animal sex and balance the sex ratios of study samples remain widespread in preclinical investigations. We analyzed the sex-reporting rate and sex distributions of test animals in published oncology studies.
-
Methods
- We selected five oncology journals included in the Scientific Citation Index (SCI) based on impact factors. We identified preclinical investigations with in vivo mouse experiments published in 2015 for inclusion in our study sample. We classified each article by whether or not it reported test subject sex, and by which sex was included. We also recorded whether there were justifications for using one particular sex in single-sex studies (e.g., anatomical reasons) and whether sex-based analyses were conducted for both-sex studies.
-
Results
- We surveyed a total of 382 articles. Half (50.3%) failed to report test animal sex. Among articles that did report sex, 91.7% were single-sex studies, of which 69.4% did not provide any justifications for using the sex included in the study. Relatively few studies 15.7 studies included animals of both sexes, and only 2.3 studies conducted sex-based analyses. These findings are consistent with those of previous research that used other methods to collect data from the literature such as text mining, but our analysis of the provision of justifications for using one sex versus the other is a novel feature.
-
Conclusions
- Many researchers overlook test subject sex as a factor, but test animal sex should be reported in all preclinical investigations to enhance the reproducibility of research and avoid faulty conclusions drawn from one-sided studies.
-
Keywords: animal experimentation; bias; data accuracy; data curation; research design; research subjects
INTRODUCTION
Historically, clinical investigations have often overlooked test subject sex as a variable and failed to report test subject sex. However, advancements in research have highlighted the importance of sex differences in multiple biological systems—endocrine, biochemical, pharmacological—which reflect disparity in incidences of diseases, immunological reactions, and responses to drugs [1-3].
Males and females can have different biological properties even on the most basic cellular levels [4,5]. For example, microglia-to-neuron signaling plays a central role in chronic pain hypersensitivity in males, but not in females. For females, adaptive immune cells, such as T lymphocytes, take the place of microglia. Such sexual dimorphism in pain mechanisms indicates that male and female mice are not interchangeable in pain studies [6]. Such differences in basic properties are also reflected in oncology [7]. The majority of anticancer drugs, including methotrexate, paclitaxel, fluorouracil, capecitabine, gemcitabine, topotecan, and etoposide show sex differences in clearance and survival rate [8-10]. However, failure to report test animal sex remains widespread in preclinical research. This potentially results in poor reproducibility [11].
Investigations of test animal sex selection and reporting have been conducted in various fields of medicine [12,13]. However, previous studies did not determine whether researchers justified the selection of one sex over the other or include multiple experiments in the same study. Therefore, we addressed these shortcomings by manually analyzing data that were informative about the state of test animal usage in oncological research.
MATERIALS AND METHODS
Literature Survey
We identified preclinical studies published in five journals included in the Scientific Citation Index (SCI) with the highest 2014 impact factors for the Journal Citation Reports subject category “oncology” (http://jcr.incites.thomsonreuters.com/) for analysis in this study. The five journals were Cancer Cell, Leukemia, Journal of the National Cancer Institute, Cancer Research, and Oncogene. Articles published in these journals during the year 2015 were selected as the target of analysis. For Oncogene and Cancer Research, which are published semi-monthly and weekly, respectively, only the first issues for each month were included in order to roughly balance the number of articles from each journal. For the other three journals that were published monthly, every issue was included.
Of the selected journals, we selected only articles that reported in vivo procedures on mice (e.g., tumor xenografting, drug administration) for inclusion in the present study. All clinical investigations, in vitro studies, and studies of human tissue samples or cell line studies were excluded. Furthermore, animal experiments that included animals other than mice, such as rats, rabbits, and zebrafish, were excluded from the analysis, because they were too few in number to have significant effects on our results. This extensive selection process was conducted manually by the authors.
Coding and Analysis
After the selection process, each article was first coded for sex (i.e., whether the study used both sexes, female subjects only, male subjects only, or did not specify the sex of subjects). For studies that included both sexes, we further noted whether the authors had conducted a sexbased analysis of at least one parameter, in which we coded the study as “yes” for presence of sex-based analysis. For studies that used single-sex samples, we recorded whether the authors provided a justification for using only one sex. A study was considered justified if the study dealt with sex-specific cancers such as cervical, uterine, or prostate cancer (breast cancer experiments conducted only in female animals were also considered justified) or if the authors explicitly stated a reason for including only one sex (Figure 1).
We determined how many publications used both sexes and how many used only female mice as test subjects. Some studies reported multiple separate experiments using different test subjects for each. We treated each experiment as a separate analysis; in other words, for “n” number of separate experiments, each experiment was counted as “1/n” studies. For example, in one publication [14] that reported the results of four separate xenograft experiments, the first experiment used female mice, the second experiment left sex unspecified, and the other two experiments used male mice. Thus, this publication included the equivalent of 0.25 female subject studies, 0.5 male subject studies, and 0.25 unspecified studies. This is why the article count given in the Results section does not include only whole numbers.
For each article, we also noted whether information about the genetic background information of the experimental mice was provided. Genetic background information included strain, breed, lineage, gene knockouts, and nude mice. We considered the study to have provided genetic background information if it provided data for any one of these variables. Finally, we assessed the policies of each journal to confirm whether the articles published in those journals were presented in a manner consistent with journal policies.
RESULTS
We surveyed a total of 382 articles. Of these, 50.3% failed to report the sex of the test animals (Figure 2). However, most (98.2%) reported genetic background information. In total, 67.1% of articles that reported sex used only female mice, 24.6% used male mice, and only 8.3% used both male and female mice. This trend was reflected in all five of the oncology journals, with each publishing similar percentages of studies that used males, females, or both sexes (Figure 3).
Of the single-sex studies, only 30.6% provided justifications for using only one sex, and 96.2% of all justifications were simply based on anatomical differences, while the remaining 3.8% were unique justifications. However, even after we excluded studies with justifications, the same trend persisted. Female mice were used most often, with only a small percentage of studies incorporating both sexes into their experimental groups (Figure 4). Of the 15.7 studies that used mice of both sexes as test subjects, only 2.3 studies used both male and female mice and included sexbased analyses (Figure 5).
DISCUSSION
Overall, our analysis revealed that researchers in the field of preclinical oncology do not pay adequate attention to test subject sex, as less than 50% of publications during 2015 reported subject sex, while the numbers of studies including both sexes and/or sex-based analyses were much lower. Ideal studies should include both sexes alongside sex-based subgroup analyses [15]. Studies that use both sexes without conducting sex-based analyses have ambiguous value in comparison to single-sex studies, because the use of both sexes can hinder the interpretation of results if there are differences between the sexes related to the topic of the study.
Another prerequisite of an ideal study is inclusion of justification(s) for not specifying sex or using a single sex. Female mice may be used for in vivo experiments due to evidence that female mice show more drastic effects than male mice after treatment [16], while male mice may be selected as test subjects because hepatocellular carcinomas are more prevalent in males [17]. Overall, only 14.4% of the studies we surveyed were ideal, either including both sexes and sex-based analyses, or single-sex studies with justifications.
Most studies, however, were less ideal. One of the articles we examined described an experiment using female mice to make prostate cancer xenograft models without commenting on the sex disparity between host and graft. This was puzzling, as female mice have different physiologic properties from male mice and prostate cancer xenografts may behave aberrantly when grafted from male donors to female hosts [18].
Some journals have their own policies regarding animal test subjects, while others recommend following pre-established guidelines, such as the Animal Research: Reporting In Vivo Experiments (ARRIVE) guidelines (Table 1). The ARRIVE guidelines consist of a checklist of 20 basic items describing the minimum information that all scientific publications involving animal experiments should report. These include recommendations regarding the number and specific characteristics of animals used such as species, strain, sex, and genetic background [19].
All five of the journals included in this study had policies recommending that the sexes of animal test subjects be delineated in the article. However, 28% to 64.5% of articles (50.3% average) left the sex of animal subjects unspecified, suggesting that journal policies are often not followed. A 2010 study examining reports in general biology, immunology, neuroscience, physiology, pharmacology, reproduction, endocrinology, behavioral physiology, behavior, and zoology (Journal Citation Reports subject categories) found similar results [12]. In that study, the authors found that subject sex was not reported in 22%–42% of articles in neuroscience, physiology, and interdisciplinary biology journals, or in more than 60% of immunology reports, while fewer than 8% of articles in the behavior, endocrinology, and pharmacology categories failed to specify the sexes of experimental animals or tissues. Another study published in the journal Pain revealed that 97% of all articles published from 1995 to 2005 reported test subject sex [13].
In comparison with the values observed in previous studies, the 49.7% of reporting test subject sex among the five oncology journals surveyed in this study was very low. A recent investigation found that cancer research articles using mice had the lowest rates of sex and age identification (48%) among many disciplines [20]. Those authors also noted that preclinical cancer research was more likely to use female mice as test subjects. Our findings agreed, with 57.5% to 77.9% of studies including only female mice. This was a surprising finding as it is often assumed that studies using male subjects dominate preclinical research. It was hypothesized that the explanation for the predominant use of male subjects was simply convention, reinforced by the assumption that female hormonal cycles act as a source of confounding variability in experimental studies. However, there is no documentation of female estrous cycles affecting experimental results in mice. Studies that include both sexes offer several advantages, including enhanced power of research, practical advantages, and conformation with moral imperatives [12,13]. The use of female mice in oncology research might be due to the greater ease of accommodating female mice in research facilities, but the present study does not provide any concrete answers to this question.
Our study has certain limitations. First, the scope of our investigation was limited to studies published in the top five oncology journals in 2015. It therefore illustrates a cross-sectional view of oncological research trends and does not reflect historical trends. Nonetheless, our findings are compliant with those of a recent large-scale study [20] and are therefore likely to be an accurate reflection of current research trends. Another limitation is that the criteria we used to identify justifications for single-sex studies were ambiguous, and we had difficulty in defining which studies adequately justified using test subjects of only one sex because there is no gold standard for classifications regarding justification [21]. Even so, when our results were adjusted to include justified single sex studies, the majority still lacked any explicit explanation for female preferences in test subject sex. Our manual text mining approach made the assessment of the presence of justifications in single sex studies possible, even if these assessments were not perfect. This allowed us to consider the hypothesis that because certain cancers are specific to sex, the disparity in the number of articles investigating female-specific cancers could be explained. Indeed, most studies that studied sex-specific cancers such as cervical, prostate, or even breast cancer utilized only one sex.
In the field of preclinical oncology, as represented by the five journals with the highest impact factors in the field, more than half of all analyzed articles did not report the sexes of test subject animals. This represents a widespread problem in oncological preclinical research, for these low levels of sex-reporting rates undoubtedly hinder the reproducibility of research and cloud accurate interpretations of results, especially since increasing evidence indicates that there are clinically significant differences between male and female physiologies. Based on our results, we suggest that researchers take care to report test subject sex and to conduct sex-based subgroup analyses, even at the preclinical level.
KEY MESSAGES
▪ In the field of preclinical oncology, more than half of all analyzed articles did not report the sexes of test subject animals.
▪ Unreported sexes of test subject animals may cause problems for reproducibility of research and interpretations of results.
▪ Researchers should take care to report test subject sex and conduct sex-based subgroup analyses, even at the preclinical level.
NOTES
-
No potential conflict of interest relevant to this article was reported.
Acknowledgments
This study was supported by Korea University grants.
Figure 1.Article coding flowchart. In the case of “n” number of experiments in one article, the number of the articles was counted by a sum of 1/n of experiments, in other words, decimal points.
Figure 2.Percentages of articles reporting the sexes of test animals in each journal. Overall, 50.3% of articles did not report subject sex, while 49.7% did. In the case of “n” number of experiments in one article, the number of the articles was counted by a sum of 1/n of experiments, in other words, decimal points. JNCI: Journal of the National Cancer Institute; Cancer Res: Cancer Research.
Figure 3.Distributions of test subjects’ sex. The percentage of female single-sex studies varied from journal to journal, ranging from 57.5% to 77.9%. For male single-sex studies, the percentages ranged from 14.1% to 32.5%, and for both-sex studies, 7.4% to 10.0%. In the case of “n” number of experiments in one article, the number of the articles was counted by a sum of 1/n of experiments, in other words, decimal points. JNCI: Journal of the National Cancer Institute; Cancer Res: Cancer Research.
Figure 4.Distribution of test subjects’ sex with justified cases excluded. Excluding single-sex studies that provided justifications for examining only one sex, 57.3% to 77.9% were female single-sex studies, while 14.1% to 32.6% were male single-sex studies. The percentages of studies including both sexes ranged from 8.1% to 15.4% depending on the journal. In the case of “n” number of experiments in one article, the number of the articles was counted by a sum of 1/n of experiments, in other words, decimal points. JNCI: Journal of the National Cancer Institute; Cancer Res: Cancer Research.
Figure 5.Percentages of justified and sex-based analysis. Of all studies, 67.1% used only female mice, while 24.6% used only male mice. The other 8.3% utilized both sexes. Among single-sex studies (studies that utilized only male or female mice), only 30.6% included justifications for using just one sex. Among studies that included both sexes, only 14.6% conducted sex-based subanalyses.
Table 1.Provided guidelines regarding test animal sex for each journal
|
Journal |
Publisher |
Policy |
|
Cancer Cell
|
Cell Press |
“We suggest that researchers carrying out experiments with animals refer to the ARRIVE guidelines and recommendations from an NIH-sponsored workshop regarding experimental design and reporting standards.” |
|
JNCI
|
Oxford University Press |
“Authors are encouraged to follow published standard reporting guidelines for the study discipline (e.g., ARRIVE for animal studies) ..." |
|
Leukemia
|
Nature Publishing Group |
“All manuscripts reporting animal research must be written up in accordance with the ARRIVE guidelines." |
|
Cancer Research
|
American Association for Cancer Research |
“In experiments involving animal models, details concerning sex, age, weight, strain, substrain, and source must be delineated." |
|
Oncogene
|
Nature Publishing Group |
“Sex and other characteristics of animals that may influence results must be described." |
References
- 1. Franconi F, Brunelleschi S, Steardo L, Cuomo V. Gender differences in drug responses. Pharmacol Res 2007;55:81-95.ArticlePubMed
- 2. Hilliard LM, Nematbakhsh M, Kett MM, Teichman E, Sampson AK, Widdop RE, et al. Gender differences in pressure-natriuresis and renal autoregulation: role of the angiotensin type 2 receptor. Hypertension 2011;57:275-82.ArticlePubMed
- 3. Wang X, Magkos F, Mittendorfer B. Sex differences in lipid and lipoprotein metabolism: it’s not just about sex hormones. J Clin Endocrinol Metab 2011;96:885-93.ArticlePubMedPMCPDF
- 4. Deasy BM, Lu A, Tebbets JC, Feduska JM, Schugar RC, Pollett JB, et al. A role for cell sex in stem cellmediated skeletal muscle regeneration: female cells have higher muscle regeneration efficiency. J Cell Biol 2007;177:73-86.ArticlePubMedPMC
- 5. Du L, Bayir H, Lai Y, Zhang X, Kochanek PM, Watkins SC, et al. Innate gender-based proclivity in response to cytotoxicity and programmed cell death pathway. J Biol Chem 2004;279:38563-70.ArticlePubMed
- 6. Sorge RE, Mapplebeck JC, Rosen S, Beggs S, Taves S, Alexander JK, et al. Different immune cells mediate mechanical pain hypersensitivity in male and female mice. Nat Neurosci 2015;18:1081-3.ArticlePubMedPMCPDF
- 7. Sechzer JA, Rabinowitz VC, Denmark FL, McGinn MF, Weeks BM, Wilkens CL. Sex and gender bias in animal research and in clinical studies of cancer, cardiovascular disease, and depression. Ann N Y Acad Sci 1994;736:21-48.ArticlePubMed
- 8. Huang RS, Kistner EO, Bleibel WK, Shukla SJ, Dolan ME. Effect of population and gender on chemotherapeutic agentinduced cytotoxicity. Mol Cancer Ther 2007;6:31-6.ArticlePubMedPMCPDF
- 9. Midgley R, Kerr DJ. Capecitabine: have we got the dose right? Nat Clin Pract Oncol 2009;6:17-24.ArticlePubMed
- 10. Wang J, Huang Y. Pharmacogenomics of sex difference in chemotherapeutic toxicity. Curr Drug Discov Technol 2007;4:59-68.ArticlePubMed
- 11. Begley CG, Ellis LM. Drug development: raise standards for preclinical cancer research. Nature 2012;483:531-3.ArticlePubMedPDF
- 12. Beery AK, Zucker I. Sex bias in neuroscience and biomedical research. Neurosci Biobehav Rev 2011;35:565-72.ArticlePubMed
- 13. Mogil JS, Chanda ML. The case for the inclusion of female subjects in basic science studies of pain. Pain 2005;117:1-5.ArticlePubMed
- 14. Schwartz S, Wongvipat J, Trigwell CB, Hancox U, Carver BS, Rodrik-Outmezguine V, et al. Feedback suppression of PI3Kα signaling in PTEN-mutated tumors is relieved by selective inhibition of PI3Kα. Cancer Cell 2015;27:109-22.ArticlePubMed
- 15. Langenkamp E, Zhang L, Lugano R, Huang H, Elhassan TE, Georganaki M, et al. Elevated expression of the C-type lectin CD93 in the glioblastoma vasculature regulates cytoskeletal rearrangements that enhance vessel function and reduce host survival. Cancer Res 2015;75:4504-16.ArticlePubMed
- 16. Adams GN, Rosenfeldt L, Frederick M, Miller W, Waltz D, Kombrinck K, et al. Colon cancer growth and dissemination relies upon thrombin, stromal PAR-1, and fibrinogen. Cancer Res 2015;75:4235-43.ArticlePubMedPMC
- 17. Wang SH, Yeh SH, Shiau CW, Chen KF, Lin WH, Tsai TF, et al. Sorafenib action in hepatitis B virus X-activated oncogenic androgen pathway in liver through SHP-1. J Natl Cancer Inst 2015;107:pii: djv190.ArticlePDF
- 18. Kim T, Jeon YJ, Cui R, Lee JH, Peng Y, Kim SH, et al. Role of MYC-regulated long noncoding RNAs in cell cycle regulation and tumorigenesis. J Natl Cancer Inst 2015;107:pii: dju505.ArticlePDF
- 19. Kilkenny C, Browne WJ, Cuthill IC, Emerson M, Altman DG. Improving bioscience research reporting: the ARRIVE guidelines for reporting animal research. PLoS Biol 2010;8:e1000412. ArticlePubMedPMC
- 20. Flórez-Vargas O, Brass A, Karystianis G, Bramhall M, Stevens R, Cruickshank S, et al. Bias in the reporting of sex and age in biomedical research on mouse models. Elife 2016;5:pii: e13615.ArticlePDF
- 21. McCullough LD, de Vries GJ, Miller VM, Becker JB, Sandberg K, McCarthy MM. NIH initiative to balance sex of animals in preclinical studies: generative questions to guide policy, implementation, and metrics. Biol Sex Differ 2014;5:15. ArticlePubMedPMC
Citations
Citations to this article as recorded by
KSCCM
, Kyung Sun3,4 
PubReader
ePub Link
Cite
XML Download
