Advertisement
Viewpoint
Open Access | 
10.1172/JCI173750
Adding robustness to rigor and reproducibility for the three Rs of improving translational medical research
Michael P. McGill1,2 and David W. Threadgill1,2,3,4
1Interdisciplinary Graduate Program in Genetics and Genomics,
2Department of Cell Biology and Genetics,
3Department of Nutrition, and
4Department of Biochemistry and Biophysics, Texas A&M University, College Station, Texas, USA.
Address correspondence to: David W. Threadgill, 206 Olsen Blvd., Reynolds Medical Sciences Building, Room 440, Texas A&M University, College Station, Texas 77843-1114, USA. Phone: 979.436.0852; Email: dwt@tamu.edu.
Find articles by
McGill, M.
in:
PubMed
|
Google Scholar
|
1Interdisciplinary Graduate Program in Genetics and Genomics,
2Department of Cell Biology and Genetics,
3Department of Nutrition, and
4Department of Biochemistry and Biophysics, Texas A&M University, College Station, Texas, USA.
Address correspondence to: David W. Threadgill, 206 Olsen Blvd., Reynolds Medical Sciences Building, Room 440, Texas A&M University, College Station, Texas 77843-1114, USA. Phone: 979.436.0852; Email: dwt@tamu.edu.
Find articles by
Threadgill, D.
in:
PubMed
|
Google Scholar
|
Published September 15, 2023 - More info
J Clin Invest. 2023;133(18):e173750. https://doi.org/10.1172/JCI173750.
© 2023 McGill, et al. This work is licensed under the Creative Commons Attribution 4.0 International License. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/.
-
Origin of translational failures
Therapeutic candidates tested in clinical trials primarily originate from results gathered using preclinical models. However, preclinical results frequently fail to be reproduced in the clinical setting. Only 11% of published “landmark” findings in oncology were validated in clinical trials (2), consistent with other disease areas, due to an inability to demonstrate safety and efficacy (Figure 1). Aside from the impact on enrolled patients, these failures produce an estimated economic burden of roughly $45 million per failed clinical trial.
Figure 1Four constraints contribute to translational failures of preclinical studies. Rigorous, reproducible, and robust experimental designs can ameliorate experimental constraints that have led to high rates of translational failure. Preclinical results can be affected by extrinsic and intrinsic factors that can be better modeled to improve translational success by embracing heterogeneity as an inherent biological phenomenon.
Variability in response to experimental therapeutics among patients is a common concern during clinical trials, yet preclinical models are almost always genetically and environmentally homogeneous. An analysis of over 2,000 drugs found that common genetically homogeneous preclinical species are inconsistent predictors of toxic responses in humans (3). However, retrospective studies using heterogeneous models support their utility for predicting therapeutic safety. For example, heterogeneous preclinical populations have predicted adverse responses to phosphatidylinositol 3-kinase inhibitors for the treatment of chronic lymphocytic leukemia and follicular lymphoma (4) and liver injury during the use of vasopressin V2-receptor antagonists in the treatment of hyponatremia (5). These discoveries could have informed clinical trial design if they had been identified proactively during the preclinical phase. In general, four areas contribute to translational failures from homogeneous preclinical models to heterogeneous human patient populations (Figure 1).
Mechanistic differences. Mechanistic differences between mice and humans can limit translation but are still informative for understanding molecular mechanisms. Classical examples include the loss of hypoxanthine-guanine phosphoribosyl transferase (HPRT) to model Lesch-Nyhan syndrome (6) and disruption of the tumor-suppressor RB transcriptional corepressor 1 (RB1) to model retinoblastoma (7). Deficiency of the X-linked purine salvage pathway gene HPRT causes Lesch-Nyhan syndrome in humans, which is characterized by self-injurious behavior and behavioral abnormalities. However, the development of Hprt-deficient mice resulted in no phenotypic neurobehavioral abnormalities (8). Further investigation showed that mice can use an alternative purine salvage pathway utilizing adenine phosphoribosyl transferase (APRT). Although subsequent treatment with an APRT inhibitor induced persistent self-injurious behavior in Hprt-deficient mice (9), this was not replicable in Hprt–/– Aprt–/– double-mutant mice (10). The failure of the Hprt, Aprt–deficient mice to generate any phenotypic abnormalities resembling the human disease suggests still-unknown species and/or genetic background-dependent differences between mice and humans.
Retinoblastoma in humans is the result of deficiency of RB1, and individuals who inherit one defective copy of RB1 frequently develop retinoblastoma early in life. Unlike children bearing germline RB1 mutations, heterozygous Rb1 mutant mice fail to develop retinoblastoma. It was later discovered that the related protein RBL1 (also known as p107) protects against tumorigenesis in mice and mutation in both Rb1 and Rbl1 is needed for retinoblastoma development in mice (11). The use of mice in translational studies evaluating mechanisms that differ between mice and humans can, even if used in a rigorous and reproducible study, lead to translational failures. Notably, neither Lesch-Nyhan syndrome nor retinoblastoma mouse models have been evaluated on more than one genetic background. However, extensive analysis of these diseases in humans now shows a continuum of disease severity ranging from clinically insignificant to very severe for Lesch-Nyhan (12) and low to high penetrance for retinoblastoma (13), likely related to genetic differences among patients that have not been explored in mice.
Inappropriate statistical analysis. Inappropriate statistical analysis can also lead to translational failures. Phenotypic traits observed in preclinical models may present similarities or differences with humans that do not exist due to inappropriate comparisons. Previous comparisons of inflammatory responses between humans and C57BL/6J (B6J) mice revealed significant differentially expressed genes (14). However, a low correlation in gene expression changes was observed between humans and mice, leading to the conclusion that mice do not accurately reflect human inflammatory diseases. Using the same gene expression data sets, another group arrived at an opposing conclusion (15). Instead of analyzing sets of genes that were changed in human inflammatory conditions regardless of expression changes in mice, researchers analyzed genes whose expression levels were significantly changed in both humans and mice. This led to the conclusion that gene expression patterns in mice do recapitulate those seen in human conditions and that mice faithfully model human inflammatory disease. Even without considering the questionable comparison of heterogeneous humans with a homogeneous mouse model, these contrasting analyses demonstrate the potential impact of different statistical approaches on translational interpretation.
Environmental variation. Environmental variation is a major confound in patient populations. Despite tight environmental control in preclinical studies, mice can display variable behavioral and molecular responses even when genetically identical (16). Phenotypic plasticity is complex and fluctuates depending on extrinsic factors, such as the fixed environment and social milieu, in addition to intrinsic factors, including genetic background and sex. Consequently, environmental variation can lead to poor reproducibility across laboratories and facilities and, in some cases, translational failures. While these environmental differences are difficult to control and can confound results in reproducibility studies, environmental variability could be leveraged to ensure robust results. Dividing large animal experiments into several smaller experiments spread over time has been shown to increase reproducibility compared with conventional experimental designs (17). Multilaboratory, environmentally heterogeneous experimental designs can also increase robustness by capturing biologically relevant variation that is largely absent under highly standardized conditions (18). A systematic analysis of the influence of gene-by-environment interactions on phenotypes spanning asthma, type 2 diabetes mellitus, obesity, anxiety, and immunological, biochemical, and hematological phenotypes using genetically heterogeneous mice demonstrated that genetic variation explains more observed phenotypic variation than the environment in over 75% of phenotypes analyzed, with genetics accounting for roughly 90% of the observed variation in some physiological phenotypes (19). Furthermore, subsequent analysis of environmental and physiological covariates indicated that sex is also influential and was involved in each case when a covariate explained over 25% of the observed variation. In cases where preclinical studies are highly standardized and environmental conditions are tightly controlled, genetically homogeneous models serve as poor models for the genetically and environmentally heterogeneous human patient population.
Poor preclinical experimental designs. Poor preclinical experimental designs that are not properly configured to translate results from homogeneous model systems to heterogeneous patient populations likely result in many translational failures. Although the experimental design may be sound for the scientific question at hand, the design can still be poor if the ultimate goal of the study is translation to heterogeneous patient populations. To generate robust conclusions, genetic heterogeneity of the human patient population should be captured in preclinical models. By modeling heterogeneity of the disease state in preclinical studies, more robust conclusions can be reached, illuminating the spectrum of patients likely to benefit from specific clinical interventions.
Copyright © 2025 American Society for Clinical Investigation
ISSN: 0021-9738 (print), 1558-8238 (online)