Author: Carrie D. Wolinetz, PhD

Former Associate Director for Science Policy, NIH

Summary of “NIH Workshop on Optimizing Reproducibility in Nonhuman Primate Research Studies by Enhancing Rigor and Transparency” Now Available

Posted on

First, I want to take a moment to wish you all well as we collectively weather the unprecedented novel coronavirus pandemic. I hope all readers of this blog, and their loved ones and friends are staying safe and well. As you might imagine, COVID-19 response is occupying a great deal of the bandwidth of NIH right now, as we ensure the safety of our workforce and accelerate scientific solutions to the current crisis. However, even from remote workspaces and amidst the flurry of pandemic response, work to support the ongoing mission of the agency is continuing…

Today, NIH is posting a summary of a workshop held in February which brought together key stakeholders to explore the intersection of optimizing research rigor and research involving non-human primates (NHP). You may recall I blogged about this last fall, and I noted the continuity with other NIH efforts assessing the landscape of NHP research, as well as the agency’s ongoing initiative to enhance research reproducibility through improvements in rigor and transparency. In the interim, a neuroethics working group of the Advisory Committee to the NIH Director (ACD) produced a report in October 2019 “Enabling and Enhancing Neuroscience Advances for Society” which included ethical considerations for NHP research, particularly relative to emerging biotechnologies. Collectively, these presented a terrific backdrop to bring together experts in various scientific disciplines, primatologists, veterinarians, and bioethicists, including representatives from National Primate Research Centers, academic institutions, government, industry and non-profit groups for two days of in-depth discussion.

A number of common themes emerged, including:

  • Ethical considerations should extend beyond legal and regulatory requirements.
  • It is of critical importance to understand the course of human disease and to understand the strengths and limitations of the animal model proposed for use.
  • Rigor and transparency necessitates data sharing yet there are challenges.
  • Rigor, welfare, husbandry, environment, and behavior are all interrelated.
  • Rigorous and reproducible science is part of a virtuous learning cycle.

However, this concise summary gives short shrift to the rich and thoughtful discussion of the unique opportunities and challenges shared by the NHP research community, and I encourage you to read the full report to get a better sense of the full breath of issues discussed. As one participant noted, even the lunchtime conversations represented exciting exchanges of ideas at the nexus of scientific and clinical opportunities, ethical considerations, ensuring rigor and transparency, and the nuances of animal welfare and husbandry. It highlighted how interdisciplinary engagement – from experts in primatology, statisticians, experimental design, clinicians, ethicists, data scientists, and basic biologists – can be incredibly useful to maximize the knowledge generated by working with these critically important animal models. In addition, it became very clear that lessons learned from NHP can more broadly benefit enhancing rigorous study design in all animal research and that data sharing is key for facilitating rigor and reproducibility.

While the discussion itself was important in fostering those connections, a natural question to ask is: what happens next? The proceedings of the workshop feed into the broader efforts of the ACD Working Group on Enhancing Reproducibility and Rigor in Animal Research (of which I am a member) in its deliberations. This group has been charged with assessing and making recommendations to enhance the reproducibility and rigor of animal research by improving experimental design, optimizing translational validity, enhancing training, and increasing the transparency of research studies involving animal models, and is expected to present an interim report to the ACD in June.

Carrie D. Wolinetz, PhD
Former Associate Director for Science Policy, NIH
Blog Roll

‘Twas the Night Before……

Posted on

‘Twas the night before Christmas, And nobody’s here.

All of us Feds are on break, But please don’t you fear…

The policy gurus, soon will return,

NIH’s midnight oil will not cease to burn.

The NExTRAC working groups are now on their way,

Emerging biotech we need to think on today.

And the Draft Data Policy is finally out,

Waiting for comments, the cheers and the shouts.

The BESH researchers are nestled, snug in their delay,

As we work with them on ensuring results see light of day.

Rumors are flying on the public access front,

We have no comment, if I may be so blunt.

The Common Rule at long last will go into effect,

Genomic Summary Results, you can now freely detect.

Return of results, we’re still thinking deep thoughts,

How to respect research participants, who we care about lots.

We’re thinking about ethics for humans, monkeys, and BRAIN,

That doesn’t mean new policy, is our constant refrain!

And I, in my role as NIH Chief of Staff,

“What work-life balance?”, I’ve been known to laugh.

On sexual harassment, let me be very clear,

Unacceptable, immoral, and not welcome here.

The culture that allows bad behavior to occur,

Is something everyone of us is responsible for.

Science needs to change, NIH needs to drive it,

We must unite with those who have somehow survived it.

“So, on civility, on respect, on diversity and inclusion!

On safe research spaces, down with biased exclusion!

On polices that promote only science that is best,

Rigorous and translatable, when put to the test!”

And when the NSABB convenes in the first quarter of ’20,

Biosecurity and transparency, we’ll talk about plenty!

The NIH Guidelines need a refresh,

And improving clinical trials goes well beyond BESH.

Heritable gene editing is still all the rage,

Will the world’s governance bodies get on the same page?

Policy opportunities abound in the coming New Year!

From all of our stakeholders, we’re dying to hear…

But for now let’s pause, as we rarely do,

To say from NIH OSP, happy holidays to you!

Carrie D. Wolinetz, PhD
Former Associate Director for Science Policy, NIH
Blog Roll

Continuing to Work with the Community on Registration and Results Reporting for Basic Experimental Studies Involving Humans

Posted on

The research that NIH funds doesn’t always fall neatly into a single category.  Basic research involving humans that seeks to understand the fundamental aspects of phenomena also may meet the NIH-definition of a clinical trial.   We refer to these studies as BESH – Basic Experimental Studies involving Humans (see our previous blog). Since this type of research meets the NIH definition of a clinical trial, these trials must register and report summary results information for transparency and other purposes outlined in the NIH Policy on the Dissemination of NIH-Funded Clinical Trial Information. However, some researchers have faced challenges in fitting these studies into the data fields for submission in ClinicalTrials.gov.

In response to concerns from the basic science communities, the NIH issued a Request for Information (RFI; NOT-OD-18-217) seeking additional input from stakeholders on how best to implement the NIH clinical trials policies.  NIH asked for examples of BESH that pose challenges for registration and results reporting on ClinicalTrials.gov, asked for potential modifications to ClinicalTrials.gov that would better meet the needs of BESH, and asked about the strengths and weaknesses of potential alternative platforms.

Thanks to the outreach efforts of the basic science communities and organizations, we received a robust response to the RFI.  A predominant theme of these responses was agreement on the importance of registration and results reporting for research transparency and on ensuring that the contributions of human research participants advance science and public health. Examples of challenges included registering and reporting both cognitive and brain function studies, and studies involving a series of trials in which the design of subsequent trials are contingent on the findings of prior trials.

To further understand the challenges of registering and reporting results for BESH on ClinicalTrials.gov, staff at the National Library of Medicine performed an in-depth analysis of several recently published articles selected from respondents to the RFI. Four primary challenges to registering and reporting results of BESH in ClinicalTrials.gov were identified from this analysis:

  1. reporting for multiple, interrelated small studies,
  2. apparent absence of prespecified primary outcome measures in the published articles,
  3. results reported for a few individual participants in a non-aggregated manner, and
  4. iterative preliminary studies to develop or optimize procedures.

These characteristics are difficult to address in a system designed to represent one study per record with at least one pre-specified primary outcome measure and with aggregated summary results information provided in tabular format.

Although most BESH are not expected to have these specific challenges, for those that do, the requirement of registering and results reporting on ClinicalTrials.gov might not communicate what the studies planned to do, or was done, in a scientifically meaningful and useful manner.  The NIH has determined that more time is needed to address these challenges. Today, NIH published a Guide Notice (NOT-OD-19-126) announcing the extension of delayed enforcement of registering and results reporting of BESH on ClinicalTrials.gov through September 24, 2021.  Investigators conducting BESH are still expected to register and report results on alternative, publicly available platforms during this interim period, but are not required to do so on ClinicalTrials.gov at this time.

During this interim period, we will continue to work together with the basic science community to understand more fully the challenges of disseminating information about BESH and explore solutions that are meaningful and useful to other researchers and members of the public. In the coming months, NIH will be reaching out to the basic science community and asking for assistance in helping us reach this goal. Working together, we believe that we can create a system that can facilitate the registration and results reporting for all clinical trials, while maintaining the NIH commitment to stewardship and transparency of the research we support.

This blog was co-authored by Dr. William Riley, Director of the NIH Office of Behavioral and Social Sciences Research (OBSSR) and Dr. Mike Lauer, the NIH Deputy Director for Extramural Research (OER).

More information about OBSSR can be found at https://obssr.od.nih.gov/. Information about OER can be found at: https://grants.nih.gov/grants/oer.htm.

Dr. Mike Lauer
NIH Deputy Director for Extramural Research
William Riley
Director of the NIH Office of Behavioral and Social Sciences Research
Blog Roll