NCCIH Research Blog

We were delighted to welcome Dr. Noni Byrnes, director of the National Institutes of Health (NIH) Center for Scientific Review (CSR), to the September meeting of our Advisory Council. During her Council presentation, Dr. Byrnes explained some modifications that are coming in the peer review process for NIH grant applications. Today, I want to let you know about some changes she discussed that will have a substantial impact both on investigators who write grant applications and on members of review panels.

The CSR has led a lengthy process to develop a revised framework for reviewing research project grant applications. One set of changes, intended to reduce the burden on reviewers, will shorten the list of items that reviewers must consider. For example, reviewers will no longer be asked to evaluate compliance with certain administrative and policy requirements, such as those regarding resource sharing plans and applications from foreign organizations. 

Another set of changes will alter the way in which investigator expertise and institutional resources are evaluated. The goal of these changes is to mitigate reputational bias in the peer review process by focusing the evaluation on the specific project that’s proposed. Instead of providing numerical scores for the investigator and environment, reviewers will be asked to judge whether the investigators are fully capable of completing the project and whether the research environment is appropriate, or whether there are gaps that would need to be filled. If reviewers identify gaps, they will be asked to explain them. Evaluation of expertise and resources will be made in the context of the proposed research rather than the general reputation of the investigator or institution.  

One very noticeable change that you will see when the new framework goes into effect is that there will no longer be five scored criteria (Significance, Investigator(s), Innovation, Approach, and Environment). Instead, the criteria will be consolidated into three factors:

1. Importance of Research (based on Significance and Innovation)
2. Feasibility and Rigor (based on Approach)
3. Expertise and Resources (based on Investigator and Environment)

This three-factor structure is intended to focus the evaluation of scientific merit on key questions: How important is the proposed research? Is the project both rigorous and feasible? Do the investigator(s) have the demonstrated background, training, and expertise to do the research? Are the institutional resources appropriate to ensure that the proposed work can be executed successfully? 

NIH gathered input from many sources to develop the revised framework, including in late 2022, a request for information (RFI) to obtain public input. Most respondents favored the proposed changes, but many commented on the need for NIH to develop strong, effective training for reviewers on the new framework and scoring system. They felt that the proposed changes will be successful only if reviewers fully understand the new criteria and can apply them properly. At our Council meeting, Dr. Byrnes explained that training will be provided for all reviewers, study section chairs, and NIH scientific review officers when the new framework is implemented. 

CSR has already put in place two other trainings for reviewers that are designed to promote effective and fair peer review: an updated training on review integrity and a new training on bias awareness and mitigation. Each of them is 30 minutes long, and both are based on real-life situations. Many reviewers have already completed the trainings and found them to be helpful. They will be required for all reviewers starting in early 2024. 

For More Information

• A Proposed Simplified Framework for NIH Peer Review Criteria
• NIH Center for Scientific Review
• Review Integrity and Bias Awareness Training Will Be Required for NIH Reviewers Effective May 2024 Council Round (NOT-OD-23-156)