69 Matching Annotations
  1. Nov 2016
    1. P. Azoulay, J. Graff-Zivin, D. Li, B. Sampat, Public R&D investments and private sector patenting: Evidence from NIH funding rules, NBER working paper 20889

      This paper shows a link between grants and private-sector innovations and created a model to quantify the variation in funding for different fields.

      Their results show an increase in private-sector patents by NIH.

    2. R. K. Merton, Science 159, 56–63 (1968).

      In this article, the sociological expression "the rich get richer and the poor get poorer", also called Matthew effect, is presented in the context of scientific publication.

      Scientists who have received grants in the past are more likely to get more grants and produce results.

    3. B. A. Jacob, L. Lefgren, J. Public Econ. 95, 1168–1177 (2011).

      The authors of this paper evaluated the impact of NIH grants on publications. They concluded that researchers who did not get an NIH grant but simultaneously applied for others grants saw one more publication (+7%).

    4. J. Berg, Productivity metrics and peer review scores: NIGMS feedback loop blog (2011)

      This article is a reasonable hypothesis that preliminary data that contribute to receiving an outstanding peer review score likely lead to high visibility publications shortly after the grant is funded.

    5. S. Cole, J. R. Cole, G. A. Simon, Science 214, 881–886 (1981).

      This article is about one negative effect of peer review, that an individual scientist devotes so much time and energy to getting financial support that it takes away from their science.

      Basically, a huge disadvantage of the peer review program is that scientists must spend too much time writing what they intend to research, rather than performing the research.

    6. B. Alberts, M. W. Kirschner, S. Tilghman, H. Varmus, Proc. Natl. Acad. Sci. U.S.A. 111, 5773–5777 (2014).

      Bruce Alberts, Marc W. Kirschner, Shirley Tilghman and Harold Varmus describe the advances in scientific knowledge and human health that have accrued as a result of the long-standing public investment in biomedical research.

    7. peer reviewers are more likely to reward projects with the potential for a very high-impact publication and have considerable ability to discriminate among strong applications

      The authors' findings suggest that peer reviewers are good at identifying innovative and ground-breaking projects.

    8. peer reviewers add value by identifying the strongest research proposals

      The authors show that peer review scores are good predictors of scientific productivity when differences in field of research, year, and applicant qualifications are removed. This suggests that peer reviewers have the necessary expertise to choose good applicants.

    9. These residuals represent the portions of grants’ citations or publications that cannot be explained by applicants’ previous qualifications or by application year or subject are

      The authors removed the influence of the grant applicant's background, demographics, and writing skill in order to look at what effect a reviewer's expertise has.

    10. covariates

      A covariate is a variable that used in a regression analysis. It is a variable that might be responsible for the outcome of a study, or that might be interfering.

      Here, all of the additional variables added in each model were covariates (writing ability, gender, ethnicity, etc.)

    11. the grant with a 1-SD worse score is predicted to have 7.3% fewer future publications and 14.8% fewer future citations

      The authors conclude here that regardless of gender, ethnicity, or institutional prestige, when the peer-review score lowers by one standard deviation, we can observe a corresponding decrease of the number of publications and citations of an author.

    12. Matthew effect

      The Matthew Effect can be summarized as "the rich get richer and the poor get poorer." It describes the idea that benefits are distributed unevenly, and that those who already have the benefits will continue to accumulate them while those without will not have the chance.

      In scientific publication, the Matthew Effect refers to the phenomenon where researchers who are established publish more often simply because they are established (and regardless of the quality of their work).

    13. We also include NIH institute-level fixed effects to control for differences in citation and publication rates by fields

      The authors try to remove the effect of an article's field on its impact. For example, a biochemistry article may appear to have a smaller impact because of the high rate of publication and citation in that field, whereas a physics article's impact may be inflated due to a lower publication and citation rate.

    14. Figure 1

      Figure 1:

      Tab 1 : Axis : The bottom axis is the peer review percentile score that defines how much the committee liked the application (the lower the percentile, the better).

      In the first plot, the y axis is the number of citations an author got after receiving a grant.

      In the second plot, the y axis is the number of publications an author produced after receiving a grant.

      Tab 2 : Description In the first plot, we can see that there are more points in the top left corner than the bottom right corner. This indicates that applications that received better ratings also received more citations.

      In the second plot, we see the same trend as in the first. This indicates that applications that received better ratings produced more publications.

    15. a 1-SD worse score is associated with a 14.6% decrease in grant-supported research publications and a 18.6% decrease in citations to those publications

      Here the authors estimated how much a decrease of one standard deviation on the percentile score affected the number of publications and citations of a grant recipient.

    16. This variation in citations underscores the potential gains from being able to accurately screen grant applications on the basis of their research potential

      The authors found that there is a lot of variation in the research output of projects that receive grants. They conclude that it would be useful to find a way to accurately screen applications to determine their potential.

    17. principal investigator (PI),

      A principal investigator (PI) is the holder of an independent grant administered by a university and the lead researcher for the grant project, usually in the sciences.

      The phrase is also often used as a synonym for "head of the laboratory" or "research group leader."

    18. U.S. Patent and Trademark Office (USPTO)

      The United States Patent and Trademark Office (USPTO or Office) is an agency of the U.S. Department of Commerce which stores, classifies, and disseminates information on patents and gives grant patents for the protection of inventions and to register trademarks.

    19. funding is likely to have direct effect on research productivity

      The authors considered grants which were already funded and competing for renewal. This makes it easier to attribute differences in research productivity to the peer review process, rather than the amount of funding the project has.

    20. NIH is the world’s largest funder of biomedical research (12). With an annual budget of approximately $30 billion, it supports more than 300,000 research personnel at more than 2500 institutions (12, 13). A funding application is assigned by topic to one of approximately 200 peer-review committees (known as study sections).

      Based on an analysis conducted by the authors, biomedical research is valued highly by individuals, governments, foundations, and corporations. Research is seen as a source of more effective treatments and preventive measures and as a route to policy, new commercial products, and economic development.

      As a result, investments in biomedical research are the highest of all sectors.

    21. peer review has high value-added if differences in grants’ scores are predictive of differences in their subsequent research output

      If the evaluation by the peer review committee is correlated with the quality of work put out by the research group, then peer review has high value-added (meaning, it is useful for choosing research groups with the highest potential).

    22. Existing research in this area has focused on understanding whether there is a correlation between good peer-review scores and successful research outcomes and yields mixed results

      In these articles, the authors found out that there was no link between higher ratings from the peer-review committee and the number of citations the article eventually got.

    23. Disagreement about what constitutes important research may introduce randomness into the process

      In this article, the authors showed that getting a research grant depends partially on chance. They reviewed the same proposals with different committees, who each gave different results.

    24. U.S. National Institutes of Health

      The National Institutes of Health (NIH) is an agency of the Department of Health and Human Services and the main agency of the U.S. government responsible for biomedical research and healthcare-related research.

  2. Sep 2016
    1. W. R. Kerr, The ethnic composition of US inventors, Working Paper 08-006, Harvard Business School (2008)

      This study applies an ethnic-name database to individual patent records granted by the United States Patent and Trademark Office to document these trends with greater detail than previously available.

    2. D. F. Horrobin, JAMA 263, 1438–1441 (1990).

      The main goal of peer review in the biomedical sciences is to facilitate the introduction into medicine of improved ways of curing, relieving, and comforting patients. The achievement of this aim requires both quality control and the encouragement of innovation. If an appropriate balance between the two is lost, then peer review will fail to reach its purpose.

    3. Although our findings show that NIH grants are not awarded purely for previous work or elite affiliations and that reviewers contribute valuable insights about the quality of applications, mistakes and biases may still detract from the quality of funding decisions.

      Summing up all results we can say that previous work or elite affiliations do not "close the door" for new ideas in research.

    4. Funds of public organisms such as NIH are given through grants after peer-review applications. Although, some voices raised to protest against this system that might be biased for the known applicants who are supposed to be favored.

      This paper analyses a big set of data concerning NIH applications to determine if this phenomenon is actually observed or not.

    5. National Science Foundation

      The National Science Foundation (NSF) is an independent agency of the U.S. government that is responsible for the development of science and technology. The Foundation carries out it's mission by providing, in general, temporary grants.

    6. Our regression results include separate controls for each type of publication: any authorship position, and first or last author publications

      What the authors mean here is that they made statistical computations that allow them to remove the effect that the position of a name in the authors row can have in a publication

    7. Fig. 3

      Figure 3 :

      Tab 1 : Panes : Each of these plots are made with a different set of data. In the first of the plots is made with the data concerning only the applications that published the most (i.e. the 0,1% best). The two last plots are also made only with the applications that published the least (i.e. 50 and 20% worse).

      Tab 2 : Axis : The bottom axis of these plots is the percentile score of each application, that defines how the committee like the application (the lower the better). The y axis of these plots represents the relative percentile of citations that an application get per percentile score compared to publications near the 10% best. This means that the higher on the y axis it is, the more (or less) cited it has been compared to the number of citations that the publications around the 10% best got.

      Tab 3 : Description : We can see in the 4 first plots that concern the applications that published the most that the more the application got a good rate from the committee, the more likely it will be cited. Although, this phenomenon is more obvious in the top 5 and 10%. In the two last plots, the curve is likely a straight horizontal line. It is especially obvious in the last plot. This means that the least an application publishes after a grant, the less it will be cited regardless of the rate of the committee.

    8. Fig. 2

      Figure 2 :

      Tab 1 : Axis : The bottom axis is for both the plots the percentile score, that defines how the committee like the application (the lower the percentile, the better). For the first plot, the y axis is the residual number of citation an application gets after getting a grant. The residual number of citations is a statistical output that allows to generalise the number of citations by percentile score. For the second plot, the y axis is the residual number of publications a n application gets after getting a grant. The residual number of publications is a statistical output that allows to generalise the number of publications by percentile score.

      Tab 2 : Description : On the first plot, we can see that the slope on the left is really steep. That shows that the committees can tell really effectively if an application has some potential. We can also see that the slope is changing around 60% of the percentile score. The authors tell that this is probably due to the ability of the program officers to detect high potential in applications that had a bad percentile score.

    9. we employ a probabilistic algorithm developed by Kerr to determine applicant gender and ethnicity (Hispanic or Asian)

      The algorithm in question was developped by William Kerr to estimate the contribution of Chinese and Indian scientists to the US Patent and Trademark Office.