The Astonishing PF-01367338 'Cheat' That Hopefully Will Fool Virtually All

Only a handful of studies have experimentally examined the agreement of funding processes. In 1977, the US National buy PF-01367338 Science Foundation re-reviewed 150 proposals using a second independent peer review panel and found a 24�C30% disagreement in funding outcomes.17 A Canadian study of 248 proposals submitted to two major funding agencies with similar peer review processes found a 27% disagreement in funding.19 In 2009, the Academy of Finland randomly assigned peer reviewers to two panels assessing the same 65 proposals, and found a 31�C35% disagreement.20 These studies have a 65�C76% agreement and 24�C35% disagreement. We similarly found that a 75% agreement was the median acceptable agreement for funding peer review in a survey of Australian researchers based on a hypothetical peer review scenario.3 The scenario was that researchers were asked to imagine that 100 proposals had been assessed and that 20 had been funded, and were then asked how many of these 20 proposals they would want to be selected by a second independent panel. The objective of this study is to prospectively test shortened proposals and simplified peer review processes for the main funding scheme of the NHMRC of Australia. This involved the parallel assessment of actual proposals submitted to the NHMRC's Project Grant scheme?in 2013. There were 3821 Project Grant proposals and the success rate was 16.9% with a total budget of $A419.6 million. We aimed to identify the agreement between the official process and the two simplified processes, and the peer review cost savings for the simplified processes. Methods Study design This study uses data from simplified and journal peer review panels organised by the research team (figure 1), and the official NHMRC panels for Project Grant proposals. Figure?1 Study design for shortened proposals and simplified peer review processes. The official funding process (shown in grey) was independent of this study. Proposals The target research areas were Basic Science and Public Health. These areas were selected based on the findings from a NHMRC study that identified high (Basic Science) and low (Public Health) correlations between the track record scores from the official panels in 2001 and the corresponding bibliometric measures.21 These two fields were therefore chosen with the aim of examining the widest expected range in agreement. A sample of 72 Project Grant proposals submitted to the NHMRC in March 2013 was voluntarily provided to the team by Australian researchers in response to email invitations sent through our existing contacts from previous studies. We used our contacts rather than a random sample of researchers in order to reduce the administrative costs of running the study. This may impact on our sample's representativeness, although our contacts covered most Australian cities and a wide range of research institutes.