Executive Order 14303 outlines nine tenets to ensure federal data, research, and analyses are compatible with Gold Standard Science. Together these nine tenets promote the generation of knowledge and will restore the public’s trust in science and its subsequent application to public policy:
- replicable
- transparent
- communicative of error and uncertainty
- collaborative and interdisciplinary
- skeptical of its findings and assumptions
- structured for falsifiability of hypotheses
- subject to unbiased peer review
- accepting of negative results as positive outcomes; and
- without conflicts of interest.
- Research must be reproducible. The GSS tenet of reproducible science reflects two key concepts: replicability and reproducibility. Replicability is the ability of the same researchers and external reviewers to perform the same analysis and achieve the same results from the same methods, while reproducibility is the ability to use multiple methods to consistently achieve results that confirm or refute prior results. Together, these concepts increase confidence in the integrity and precision of the results and can validate broader scientific claims. To promote this tenet, OSTP has directed that agencies should require of their research clear, standardized, and justifiable protocols; comprehensive documentation; robust statistical methods; adequate sample sizes; validated methodologies; and appropriate controls.
- Research and results must be transparent. This tenet entails the open, accessible, and comprehensive sharing of all components of the research process—methodologies, data, analytical tools, and findings—to enable stringent scrutiny, validation, and reuse by research peers and the public. OSTP has charged agencies to prioritize clear, detailed reporting of methodologies, make raw data and analytical tools publicly available when feasible and lawful, and disclose funding sources or conflicts of interest.
- Results much acknowledge and communicate the limitations of assumptions and methodologies within research. That is, research should not claim too broad conclusions—though subsequent extensions of previous research may build on and expand the applicability of a study’s results. Communicating error and uncertainty in science requires the clear, precise, and accurate disclosure of limitations, variability, and potential sources of error, as well as acknowledging any limitations in data, measurements, or research findings. OSTP has instructed agencies to ensure research reporting includes quantitative measures of uncertainties—such as confidence intervals, error margins, or sensitivity analyses—alongside clear explanations of methodological constraints and assumptions and the intended scope of the research, including what the scientific findings do and do not establish. Moreover, agencies should promote cautious, evidence-based language in reports, publications, and public communications and discourage speculative claims or extrapolations that extend well beyond the data’s scope, especially when science is used in an operational or regulatory context.
- GSS also encourages collaborative and interdisciplinary science. Including multiple disciplines in the creation and review of research can increase insight into topics of study, and the integration of a wide range of expertise, methodologies, and perspectives can catalyze transformative discoveries. OSTP says that agencies shall prioritize collaborative and interdisciplinary approaches in scientific research to accelerate discovery and innovation. These approaches include recognizing limitations in an individual’s or an agency’s expertise and engaging other divisions within an agency, or other agencies, for complementary expert support when appropriate to address cross-disciplinary problems.
- In addition, researchers should have a healthy skepticism of assumptions and consequent findings. Critical evaluation of underlying assumptions, methodologies, and findings—as well as openness to alternative assumptions, model specifications, interpretations—promotes the validity, robustness, and reliability of research. As a result, the scientific process is strengthened. One key aspect of this tenet is to be aware of potential confirmation bias and avoid it. OSTP has mandated that agencies foster this GSS tenet through policies and programs that emphasize critical evaluation, transparency, and objectivity. One such way is through encouraging constructive collaborations among researchers with differing viewpoints to critically assess the reliability of research results.
- Gold Standard Science should also be structured that hypotheses can be disproven. The GSS tenet of falsifiability requires designing studies to so that hypotheses can be carefully tested and potentially disproven empirically. This approach promotes rigor and the generation of new knowledge by preventing the perpetuation of unproven assumptions. OSTP has directed that federal agencies prioritize research that is structured for falsifiability of hypotheses and advances knowledge through careful testing and is transparent about null or negative results in publications.
- Another tenet of GSS is to ensure that research is subject to unbiased peer review at various stages, including but not limited to the proposal, draft, and publication submission stages. Ensuring impartial and independent evaluation by qualified experts increases the credibility of research, minimizes bias, ensures rigor, and promotes objective scrutiny. OSTP has charged agencies to prioritize unbiased peer review to advance sound science in the review, selection, and awarding of Federal grants and contracts, including competitive and discretionary awards.
- It is also imperative that research accept negative results as beneficial to the goal of increasing knowledge. A study’s failure to confirm a hypothesis or its production of counterintuitive results still adds to the understanding of the world. Moreover, it leads to further inquiry to confirm the results or explore why the hypothesis was rejected. This tenet counters publication bias, encourages comprehensive reporting, and provides valuable insights into ineffective approaches, thereby guiding future research directions and avoiding redundant efforts. OSTP has instructed agencies to foster integrity and research innovation by recognizing that negative or null results are valuable contributions to scientific knowledge, such that research projects transparently report all outcomes, including null or negative results, in publications and publicly accessible data repositories, accompanied by clear, detailed documentation of methods, analyses, and limitations.
- The final tenet of GSS requires that those performing research do so without conflicts of interest. Research should be designed, executed, reviewed, and reported free from financial, personal, or institutional influences that could bias outcomes or undermine objectivity. This upholds scientific integrity, fosters public confidence, and ensures that results reflect evidence rather than external agendas. OSTP has says that agencies must require researchers, reviewers, and managers to disclose all relevant affiliations, funding sources, and relationships and adhere to stringent ethical standards that are supported by strong institutional oversight, transparent reporting systems, and independent expert review mechanisms.