Monday, December 10, 2007

NIH Panel Unveils Big Ideas for Revamping Peer Review and Grant Making

By JEFFREY BRAINARD

Bethesda, Md.

A committee studying ways to improve peer review and grant making at the National Institutes of Health called for major changes on Friday. They include slashing the length of grant applications and placing more weight in grant reviews on the scientific effects of the proposed research.

The working group, created by the advisory committee to NIH director Elias A. Zerhouni, also suggested providing more grants for young scientists who have never before received one. The agency should review grant proposals from such applicants separately from those of established, older investigators. The younger scientists make up a diminishing proportion of the agency's grantees, raising concerns about the future vigor of the biomedical-research work force.

In addition the working group proposed ways to improve the quality and efficiency of the NIH's peer review. One way was to require senior, established researchers to serve on the agency's review panels, which are made up largely of outside academics, as a condition of receiving certain grants. Those veterans are increasingly unwilling to volunteer because of the time commitment involved, but they possess the expertise and experience needed for quality reviews, the panel found.

The working group presented its suggestions at a meeting of the advisory committee and plans to continue refining its ideas in January. The agency—the largest source of money for academic research—is expected to decide by March whether to begin testing some of the proposed changes.

"The biggest impact will come" not piecemeal but "through a combination of multiple, decisive changes," said Keith R. Yamamoto, a co-chairman of the working group and executive vice dean at the University of California at San Francisco School of Medicine. And that, he said, in an understatement, "is kind of hard for a big bureaucracy to pull off."

Members of the director's advisory committee praised many of the ideas and said the agency should act boldly.

The recommendations were influenced by more than 2,600 comments about improving peer review that the NIH collected this year from scientists, universities, and others (The Chronicle, December 7). Many researchers see the agency's grant-review process as unpredictable and overly time-consuming, conservative, and focused on minutiae.

One of the big changes would transform the peer-review committees of outside scientists who now review and rate applications for NIH grants, making them operate more like the editorial boards of scholarly journals. The committees, called study sections, would "outsource" grant applications to specialists in the discipline to review technical aspects. The study sections would be made up of generalists who would discuss the applications' scientific significance, broadly construed.

Study sections now draw on the technical expertise of standing and ad-hoc members. Many scientists have said that the committees are overworked by record numbers of applications and pay too much attention to the fine points of an experiment's methodology and too little to its potential to advance science and medical care.

Among other suggestions offered by Mr. Yamamoto:

• Study sections should offer applicants a "prereview" based on technical evaluations, and the applicants should be allowed to provide a brief, one-page rebuttal. This idea was a response to complaints that the written evaluations sent to grant applicants by study sections often contain factual errors or indications that reviewers did not understand technical details.
• The review panels should bluntly tell applicants if their ideas are not creative enough to merit financing, ever. Instead, study sections tend to waffle, criticizing only a proposal's technical merits. As a result, applicants then revise and resubmit applications, only to have them rejected again—wasting everyone's time and sowing frustration.
• To save applicants' time, the maximum length of research-grant applications should be reduced to only seven pages from the current 25. This would force applicants to shrink details on their experiments' methodology and focus instead on the effect on science.
• The NIH should provide perhaps 1 percent of all research funding for "transformative" research likely to "crush" scientific paradigms and establish new ones.
• The study sections would eliminate a 51-point scale now used to prioritize grant applications, which some scientists have said creates a false sense of precision. In its place, the study sections would rank applications in priority order.
• The NIH should evaluate the performance of peer reviewers.

Please contact your ORPA Research Administrator with any questions.

Anthony Beckman - x5-1502

Donna Beyea - x5-8037

Marlene Boutet - x3-3960

Brenda Kavanaugh - x5-1504

Jane Tolbert - x5-4210

Cheryl Williams - x5-1503