Capturing Impact: A Method for Measuring Progress

NIH’s mission is to seek fundamental knowledge about the nature and behavior of living systems and the application of that knowledge to enhance health, lengthen life, and reduce illness and disability.

Let me pose a simple question – how do we know if NIH is achieving its mission?  It’s tricky enough to assess how effective we are at generating fundamental scientific knowledge, though we have decent grasp on that side of the equation.  We can link tens of thousands of biomedical research articles published each year to the NIH grants that supported them.  But can we take it a few sizeable steps further and systematically connect our research efforts to advances in human health?  And how can we use what we learn to design policies and strategies to speed innovation and biomedical progress?

The pathways from research to practice to changes in public health are typically non-linear and unpredictable.  For a scientific discovery to make that journey may take decades or more and involves a complex ecosystem – academic scientists, research funders, policymakers, health product developers, regulators, clinicians, and a receptive public, just to name a few. To better understand these intricate pathways, we conducted a handful of case studies that help illuminate the types of evidence and data that NIH can draw from in order to measure our progress towards our ultimate goal – improving human health.

Today we are publishing three case studies in a new section on the “Impact of NIH Research” website, titled “Our Stories.”  These studies, developed with our partners in the Institutes and Centers, trace the chain of evidence between scientific discoveries to longer-term health impact, reaching back into basic research findings that set the stage for progress and noting NIH’s role as well as that of others along the way.  Study topics range from a childhood vaccine that dramatically reduced the incidence of a deadly infection, to a paradigm-shifting approach for treating cancer, to a suite of neurotechnologies for profound impairments like deafness, paralysis, and Parkinson’s disease.  Focusing on these topics gave us a chance to examine the factors that led to their success and broadly map the data sources and strategies we should cultivate in order to improve our capacity for assessing impact (positive, negative, and null) across NIH’s portfolio.  In an era of big data, when the ability to link and analyze multiple streams of information has never been better, this seemed an opportune time to go on a data hunt.

The data we drew from were wide-ranging, including grants, research publications, press releases, patents, FDA approvals, clinical guidelines, policy and regulatory decisions, industry reports, economic analyses, medical expenditures, and public health statistics.  In piecing these disparate sources together, some clear needs emerged – it would be fantastic to link NIH’s grants data to structured data from other Federal sources, like FDA, CDC, AHRQ, USPTO, and CMS.  Citations in patents, FDA approval packages, clinical trials and guidelines, and regulations could help link such outputs to federal funding.  One of the biggest challenges is the need for strategies to connect research advances to long-term changes in health practice and outcomes, for example data on healthcare utilization, disease statistics, and quality of life measures.  We at NIH, and particularly our colleagues in the Office of Extramural Research and the Office of Portfolio Analysis, are pushing to develop and bring just these kinds of data into our own administrative data systems. We’re hopeful that emerging data tools may one day keep track of our impacts, and our influence on the health of the Nation, almost as thoroughly as we track our grants.

The studies we post today are just a few examples of the continuing work of NIH to measure our progress in improving the health of all Americans and those across the globe.  I invite you to take a look, not just at the story itself, but also the backbone of evidence behind it.  Hopefully, you’ll get a sense of the vibrant and diverse ways that NIH turns discovery into health, and how we’re grappling with making that process even better.

Posted by Dr. Carrie D. Wolinetz, June 1, 2016

Building a Better Biomarker Glossary

Precise and clear communication across biological and clinical research disciplines supports efficient translation of results from basic research into applied therapeutics and interventions. Both the NIH and FDA are keenly interested in working together to help the biological and clinical research communities speak a common language, so that research results can be clearly understood by both groups.

This is especially true in considering the vocabulary used to describe measures of health, disease, or physiological processes. For example, “biomarkers,” “surrogate endpoints,” and “clinical outcome measures” are widely used in published research findings.  The terms above mean different things; in order to build a solid foundation of research for precision medicine (and medicine in general), it is important that researchers are communicating consistently and that the wider community understands what published results actually mean.  Inconsistent terminology undermines the strength of these tools.

We’d like to share a new resource focused on biomarkers, endpoints, and other related tools that we hope will assist researchers in the development of their research plans and reporting of research  findings. Earlier this year, the NIH and FDA published an open access textbook: the Biomarkers, EndpointS, and Other Tools (BEST) Resource.  BEST was developed by evaluating an extensive array of definitions — drawing from FDA guidance documents; the scientific literature; the 2010 Institute of Medicine study on the Evaluation of Biomarkers and Surrogate Endpoints in Chronic Disease; and a 2015 Brookings Institution meeting with academic and private sector stakeholders.

If your work involves the use of biomarkers or clinical outcomes, we hope you take a look at this resource and consider using it in a variety of contexts – reading and writing manuscripts, discussing your ideas with colleagues, and planning the next steps in your research. The glossary is intended to be a living document, with the goal of adding more terms and definitions based on your feedback. We welcome you to email the joint FDA-NIH Biomarker Working Group with your suggestions – including proposed edits. This input will be used by the Working Group to inform future editions of this glossary.

For additional perspectives on this resource, please see the recent JAMA article or blog in the FDA Voice.

Many thanks to Pamela McInnes, Lisa McShane, and Holli Hamilton of NIH for their contributions to this blog.

Dr. Mike Lauer is the NIH Deputy Director for Extramural Research and blogs about NIH research funding policies and data at his blog, Open Mic.

Posted by Dr. Carrie D. Wolinetz, April 18, 2016

Dr. Mike Lauer
NIH Deputy Director for Extramural Research

NIH-FDA Draft Clinical Trial Protocol Template: Stakeholder Feedback Needed!

In the recently released NIH-Wide Strategic Plan (FY 2016-2020), the agency declared our commitment to “fostering approaches to enhance the speed and efficiency with which [clinical] trials are conducted.” On March 17, we unveiled a piece of that ongoing effort for input from stakeholders: a draft clinical trial protocol template released in today’s Guide to Grants and Contracts.

Developed jointly by NIH and the Food and Drug Administration (FDA), the protocol template  provides a standard format with instructional and sample text that NIH funded investigators can use when preparing protocols for phase 2 or 3 clinical trials that require an Investigational New Drug application (IND) or Investigational Device Exemption (IDE) application.

The agencies’ goal in developing the template is to ensure investigators prepare protocols that are organized consistently and contain all the information necessary to enable efficient and timely review.   The draft template is consistent with guidance on protocol development found in the International Conference on Harmonisation (ICH) E6 Good Clinical Practice.  A copy of the draft template as well as instructions on how to comment can be viewed on the NIH OSP Website.

We encourage stakeholders to submit comments on the utility of the template and the clarity of the accompanying instructional guidance.  We would specifically welcome feedback from investigators, investigator-sponsors, institutional review board members, and any other stakeholders who are involved in protocol development and review on the readability and clarity of the instructions contained in the template.  Stakeholder feedback will be critical when NIH and FDA consider the next steps in this process.

Posted by Dr. Carrie D. Wolinetz, March 18, 2016

Biosafety at NIH and Beyond….A Shared Responsibility

Co-Authored by Carrie Wolinetz and Deborah Wilson

Following this week’s release of the Office of Science and Technology Policy’s joint memo on biosafety and biosecurity, coming at the tail end of National Biosafety Stewardship Month, it seems like an excellent time to discuss how NIH helps ensure the research we conduct at our own facilities as well as the research we fund across the globe is done safely.

Here at NIH, the NIH Division of Occupational Health and Safety (DOHS) is responsible for overseeing the day-to-day operations of a large and diverse biosafety program. A committed team of biosafety professionals helps ensure that the vital research being carried out by NIH is being done safely. To manage the unique challenges associated with the NIH intramural program, DOHS must be flexible in order to adapt to the changing research landscape.  Recently, DOHS has instituted changes to NIH policies for working with and storing potentially hazardous biological agents including human, plant, and/or animal infectious agents, poisons and toxins. These are significant changes to the way NIH has been doing business for almost four decades. Although NIH registers, reviews, and approves all active work with human pathogens and research involving non-exempt recombinant nucleic acids, it became clear we also needed to re-evaluate and optimize the methods used for keeping track of all biological agents that might have been stored in laboratories or repositories. In addition to support received from senior NIH management, a plan for interactions and information sharing was developed.  A continuous, open dialogue with NIH’s safety committees, such as the institutional biosafety committee (IBC), was also essential to invoking changes to long-standing programs and processes.

In support of the biosafety programs of the institutions that NIH funds, the NIH Office of Science Policy (OSP) conducts an extensive program of outreach and education on topics related to biosafety.  One of the signature programs in OSP is our extramural site visit program for grantee institutions.   The aim of these educational visits is to enable NIH to have a face to face dialog with institutions and to assist IBCs with their programs of biosafety oversight.  The visit includes a review of the policies and procedures that the institution is implementing to ensure the safe conduct of recombinant or synthetic nucleic acid research.  To date, OSP has visited over 110 institutions, and a write up of the program received the 2015 Richard C. Knudsen Memorial Publication Award from the American Biological Safety Association.

OSP has also used the information we have gathered from our site visits to develop a body of information on best practices, in particular the IBC Self-Assessment tool which institutions can use to evaluate their own IBC program.  We encourage all institutions to use the self-assessment tool, which addresses all of the major requirements of the NIH Guidelines.

These are just of few examples NIH is doing to ensure essential biomedical research is conducted safely.  Biosafety is a shared responsibility for all those involved in the research enterprise.  The close and collaborative relationship between the DOHS and OSP help ensure that NIH is at the top of the class with respect to biosafety oversight.  To learn more about NIH’s intramural biosafety program, please visit http://www.ors.od.nih.gov/sr/dohs/Pages/default.aspx.  More information on how NIH engages our extramural grantee institutions with respect to biosafety can be found at: /office-biotechnology-activities

RADM Deborah Wilson, Dr.P.H., is the Director of the Division of Occupational Health and Safety at NIH