Capturing Impact: A Method for Measuring Progress

NIH’s mission is to seek fundamental knowledge about the nature and behavior of living systems and the application of that knowledge to enhance health, lengthen life, and reduce illness and disability.

Let me pose a simple question – how do we know if NIH is achieving its mission?  It’s tricky enough to assess how effective we are at generating fundamental scientific knowledge, though we have decent grasp on that side of the equation.  We can link tens of thousands of biomedical research articles published each year to the NIH grants that supported them.  But can we take it a few sizeable steps further and systematically connect our research efforts to advances in human health?  And how can we use what we learn to design policies and strategies to speed innovation and biomedical progress?

The pathways from research to practice to changes in public health are typically non-linear and unpredictable.  For a scientific discovery to make that journey may take decades or more and involves a complex ecosystem – academic scientists, research funders, policymakers, health product developers, regulators, clinicians, and a receptive public, just to name a few. To better understand these intricate pathways, we conducted a handful of case studies that help illuminate the types of evidence and data that NIH can draw from in order to measure our progress towards our ultimate goal – improving human health.

Today we are publishing three case studies in a new section on the “Impact of NIH Research” website, titled “Our Stories.”  These studies, developed with our partners in the Institutes and Centers, trace the chain of evidence between scientific discoveries to longer-term health impact, reaching back into basic research findings that set the stage for progress and noting NIH’s role as well as that of others along the way.  Study topics range from a childhood vaccine that dramatically reduced the incidence of a deadly infection, to a paradigm-shifting approach for treating cancer, to a suite of neurotechnologies for profound impairments like deafness, paralysis, and Parkinson’s disease.  Focusing on these topics gave us a chance to examine the factors that led to their success and broadly map the data sources and strategies we should cultivate in order to improve our capacity for assessing impact (positive, negative, and null) across NIH’s portfolio.  In an era of big data, when the ability to link and analyze multiple streams of information has never been better, this seemed an opportune time to go on a data hunt.

The data we drew from were wide-ranging, including grants, research publications, press releases, patents, FDA approvals, clinical guidelines, policy and regulatory decisions, industry reports, economic analyses, medical expenditures, and public health statistics.  In piecing these disparate sources together, some clear needs emerged – it would be fantastic to link NIH’s grants data to structured data from other Federal sources, like FDA, CDC, AHRQ, USPTO, and CMS.  Citations in patents, FDA approval packages, clinical trials and guidelines, and regulations could help link such outputs to federal funding.  One of the biggest challenges is the need for strategies to connect research advances to long-term changes in health practice and outcomes, for example data on healthcare utilization, disease statistics, and quality of life measures.  We at NIH, and particularly our colleagues in the Office of Extramural Research and the Office of Portfolio Analysis, are pushing to develop and bring just these kinds of data into our own administrative data systems. We’re hopeful that emerging data tools may one day keep track of our impacts, and our influence on the health of the Nation, almost as thoroughly as we track our grants.

The studies we post today are just a few examples of the continuing work of NIH to measure our progress in improving the health of all Americans and those across the globe.  I invite you to take a look, not just at the story itself, but also the backbone of evidence behind it.  Hopefully, you’ll get a sense of the vibrant and diverse ways that NIH turns discovery into health, and how we’re grappling with making that process even better.

Posted by Dr. Carrie D. Wolinetz, June 1, 2016

Ensuring Continued Responsible Research with Non-Human Primates

Research with animals, including non-human primates, has enabled the development of treatments and cures for a host of devastating diseases and conditions in humans, and continues to revolutionize our understanding of health and disease.  Because non-human primates are anatomically, physiologically and behaviorally similar to humans, they are particularly valuable for answering some of the most complex questions germane to human health.  These research models have been instrumental to significant scientific and medical advances such as deep brain stimulation to treat Parkinson’s disease, experimental vaccines aimed at preventing the spread of Ebola virus, developing the polio vaccine, and new strategies that improve organ transplant survival today.  Non-human primate research retains a critical position in the biomedical research enterprise.

Equally important to their scientific value is upholding the highest possible standards of animal welfare, including ensuring that a proposed animal model is appropriate to the research and with an expectation of scientific rigor for every experiment. These precepts have been codified in research policy for many decades (https://www.nal.usda.gov/programs/awic) and are a central value of all biomedical research funded by the National Institutes of Health (NIH). NIH remains confident that the oversight framework for the use of non-human primates in research is robust and has provided sufficient protections to date. However, we believe that periodically reviewing agency policies and processes ensures that this framework evolves in a manner consistent with emerging scientific opportunities and public health needs.

Toward this end and in response to Congressional interest , the Office of Science Policy is taking the lead in planning a workshop on September 7th, 2016 that will convene experts in science, policy, ethics, and animal welfare.  Workshop participants will discuss the oversight framework governing the use of non-human primates in NIH-funded biomedical and behavioral research endeavors.  At this workshop, participants will also explore the state of the science involving non-human primates as research models and discuss the ethical principles underlying existing animal welfare regulations and policies.  NIH is committed to ensuring that research with non-human primates can continue responsibly as we move forward in advancing our mission to seek fundamental knowledge and enhance health outcomes.

The workshop will be broadcast live and archived for future viewing on the NIH Videocast website.  Comments regarding the workshop may be submitted online in advance of and during the workshop for consideration. Please save the date, and stay tuned for more information, including a detailed agenda.

 

Posted by Dr. Carrie D. Wolinetz, May 24, 2016

Protecting Data, Promoting Access: Improving Our Toolbox

You may recall from one of my previous blogs that there are policy challenges in balancing the sharing of valuable research data with the protection of the participants whose data is being shared. The NIH Office of Science Policy is taking a leading role in ensuring that genomic data is shared in a responsible way.

Under the NIH Genomic Data Sharing Policy, institutions must indicate the appropriate use of genomic data, including any limitations on the distribution and use of that data.  In order to assist institutions in recognizing potential data use limitations (DUL), NIH created several resources for investigators.  These include Points to Consider in Developing Effective Data Use Limitation Statements as well as a set of Standard Data Use Limitations.  However, even with these resources, there is still the possibility for multiple interpretations which may cause time delays and additional costs when trying to share genomic data.

This raises some interesting questions: could time delays and cost burdens be reduced if the conditions of potential data use and sharing were clearly communicated at the time the data is generated? What are the variations in the designation of data conditions, and what does that landscape look like, domestically and internationally? To begin to address these questions, OSP staff collaborated with members of the Global Alliance for Genomics and Health to a set of “Consent Codes” that could be used to assign genomic datasets to standardized data use groups in order to allow a consistent interpretation for the appropriate secondary use of vital genomic data.  As described in a recent article in PLOS Genetics, the consent codes should help in avoiding the introduction of unnecessary new restrictions on data use, while at the same time facilitating research with the greatest amount of data available.

The issue of genomic data sharing highlights the importance of international collaboration as well as the delicate balance between the broad sharing of valuable research and ensuring the protection of participants.  OSP will continue to develop resources on genomic data sharing while evaluating the existing policy landscape to ensure that researchers have appropriate access to data while simultaneously making sure that the data is not inadvertently or deliberately misused.

Posted by Dr. Carrie D. Wolinetz, May 2, 2016

Building a Better Biomarker Glossary

Precise and clear communication across biological and clinical research disciplines supports efficient translation of results from basic research into applied therapeutics and interventions. Both the NIH and FDA are keenly interested in working together to help the biological and clinical research communities speak a common language, so that research results can be clearly understood by both groups.

This is especially true in considering the vocabulary used to describe measures of health, disease, or physiological processes. For example, “biomarkers,” “surrogate endpoints,” and “clinical outcome measures” are widely used in published research findings.  The terms above mean different things; in order to build a solid foundation of research for precision medicine (and medicine in general), it is important that researchers are communicating consistently and that the wider community understands what published results actually mean.  Inconsistent terminology undermines the strength of these tools.

We’d like to share a new resource focused on biomarkers, endpoints, and other related tools that we hope will assist researchers in the development of their research plans and reporting of research  findings. Earlier this year, the NIH and FDA published an open access textbook: the Biomarkers, EndpointS, and Other Tools (BEST) Resource.  BEST was developed by evaluating an extensive array of definitions — drawing from FDA guidance documents; the scientific literature; the 2010 Institute of Medicine study on the Evaluation of Biomarkers and Surrogate Endpoints in Chronic Disease; and a 2015 Brookings Institution meeting with academic and private sector stakeholders.

If your work involves the use of biomarkers or clinical outcomes, we hope you take a look at this resource and consider using it in a variety of contexts – reading and writing manuscripts, discussing your ideas with colleagues, and planning the next steps in your research. The glossary is intended to be a living document, with the goal of adding more terms and definitions based on your feedback. We welcome you to email the joint FDA-NIH Biomarker Working Group with your suggestions – including proposed edits. This input will be used by the Working Group to inform future editions of this glossary.

For additional perspectives on this resource, please see the recent JAMA article or blog in the FDA Voice.

Many thanks to Pamela McInnes, Lisa McShane, and Holli Hamilton of NIH for their contributions to this blog.

Dr. Mike Lauer is the NIH Deputy Director for Extramural Research and blogs about NIH research funding policies and data at his blog, Open Mic.

Posted by Dr. Carrie D. Wolinetz, April 18, 2016

Dr. Mike Lauer
NIH Deputy Director for Extramural Research