博文
My (old) notes after severing on a NASA review panel (before 2005)
|||
Who should read this Blog?
Any first-time PI, who intends to write a proposal to NASA for funding. Or anyone who is interested in knowing how a U.S. funding agent decides on which proposals to fund or not.
Background for posting these old notes: A friend of mine is serving on a review panel to decide the fate of 350 proposals. This reminded me of my past experience as a panel member, which I shared (reluctantly) with some of my colleagues. If you don’t know some of the abbreviations in my notes below, you may assume it’s a government agent (such as NASA, NOAA, NSF), or a program (such as ESE/NRA), or an institute/center (such as JPL, APDRC), or an ocean model (OGCM), or a “data” product (such as ECCO).
Dear colleagues,
Toni (not her real name) encouraged me to forward my notes to you, the notes I took during and after I severed on a NASA panel to review proposals. I am reluctant, because I don't like to "put things in writing." However, I trust her good intention, so here it is, with some additions in bold-faced fonts.
Regards,
Zuojun
1) For this panel, there were 128 proposals. No mail reviews conducted.
2) Each of the 20 some panel members received a FedEx package a month before the panel meeting.
3) There were about three weeks of time for reviewers to read the assigned proposals, to write the reviews, and to submit them electronically. The web site was closed for review submission one week before the panel met in D.C.
In theory, each reviewer should have enough time to finish the assignment, working on one proposal per day (meaning reading a proposal and writing the review). In practice, I read on average two proposals per day, some read 4-5 proposals per day just before the deadline for web submission, and some never submitted much writing before the deadline. My point here is that the proposal has to be very well written to convince the reviewers of its value.
4) When the panel finally met, each proposal was given about 15 minutes of time for the primary reviewer to have the opening remark, followed by the two secondary reviewers, questions and comments were then welcomed by the panel members before their votes were counted. The reviewers were allowed to change their original scores for many reasons... (The most important one was triggered by the "exercise" the program manager did at the beginning of the panel meeting: He started the meeting with Group 0 proposals, about five of them all together. When Group 0 was finished, we realized that they were "post child" for "Excellent", "Very Good", "Good", "Fair" proposal ratings. Following that exercise, some of the reviewers adjusted their scores during the rest of the week.)
5) The panel's votes were calculated into one score, which was entered as 1/4 of the final score. So, a dynamics primary reviewer in favor of the proposal was critical, so were the opinions of the secondary reviewers, because the rest of the panel did not read the proposal in discussion. Each panel member was given the whole collection of the 128 abstracts before the meeting, and one may request a copy of a proposal of particular interest to him. There was a copy available for the panel at the time of discussion as well.
6) The role of the program manager: He read all the 128 proposals and made notes of his opinions. He also spoke of his views whenever he wanted to, and sometimes quite authoritatively.
7) Final show: three blocks for each subject, core physical oceanography, air-sea fluxes, new instrument development, etc. The program manager went over each block, citing his reasons for the placement (ranking of proposals) and asking for advice.
My notes/hints taken during the panel meeting:
1) Write well; including logically organizing the proposal, giving detailed work plan (year by year), and using larger fonts and italicized or bold-faced fonts to highlight the hypothesis, etc. (Toni asked:"How about the English?" Yes, good English is very important. This remark is not directed to non-native speakers, because I read some poorly written proposals by English-speaking colleagues.—This was before I became a freelance English editor.)
2) Emphasize the proposal's relevance to ESE/NRA, especially what the PI can give back to NASA!
3) Know the past literature well, including citing seminal papers in the field as (30-year-) old as they may be;
4) Attach unpublished manuscript and pubs in press (so the reviewers can't complain, even if they don't have time to read any). Also, some reviewers were critical about PIs' past experience (in terms of pubs in the proposed field), they might give a low score to a proposal simply because the PI never before did any work on the subject (say biology, or numerical modeling);
5) Don't propose "new field" research, use existing fund instead to get at least one pub out first);
6) Don't assume the reviewers are from the same field (unlike NSF's mail review process); e.g., some satellite people reviewed a numerical modeling proposal, and gave a low score because he didn't know what penetrative shortwave radiation meant in an OGCM, and didn't see an equation for it nor any explanation for how to implement the term; the last two points were valid criticisms though;
7) Validation, validation, validation; some numerical modeling proposals without validation component by field observations can be rated down;
8) Follow the budget rules (1/4 million dollars/year for this NRA);
9) Disclose related funding (especially those from NASA for similar type of work);
10) If collaborating with NOAA personnel, one should be extremely cautious about the relevance with NASA (why should NASA fund this proposal, not NOAA?) and ask for NO money for buying NOAA computing facility).
My other thoughts/observations:
1) The primary reviewer is very important, but the secondary reviewers with "strong-opinions" are also critical because each casts 1/4 of the overall score;
2) Consensus form: Wording was required to match the overall score, the only score that the PI will see, especially for those at 3 or lower; more "weaknesses" were asked for proposals on the "bubble" in case they are turned down at the end due to limited funds available;
3) Two okay proposals may help to bring one UP for funding?
4) ECCO is considered "NASA" product/investment, so someone should validate/use it. Right now, ECCO is mostly used by ECCO people at JPL; so you should write a proposal using ECCO;
5) If one might lose his job when this proposal doesn't get funded, the manager might want to save his career.
My advice for NOAA researchers:
One should look at the statistics: How many NOAA (related) people actually get funded by NASA (before spending any time on writing NASA proposals).
https://blog.sciencenet.cn/blog-306792-345934.html
上一篇：Basic English (3): however vs. nevertheless
下一篇：Green tides off China (4): More needs to be done and can be done …
