Folha: 'The San Francisco Declaration'
Reproduction of Marcelo Viana's column in Folha de S. Paulo.
I am writing this column on my return from Brussels, where I once again participated in the evaluation of research projects for the ERC (European Research Council), the scientific agency of the European Union .
Every year, the ERC opens calls for proposals in four categories. One of them, called "synergy," funds collaborations between up to four researchers, one of whom may be located on another continent. The other three – "starting," "consolidated," and "advanced" – support individual projects carried out in Europe by researchers at different stages of their careers. The grant can reach €2.5 million (R$15 million) per researcher for five years.
I started participating in the "advanced" judging process in 2019, and since 2023 I have been the coordinator of the mathematics committee. I accepted the invitation out of professional curiosity: having been responsible for scientific evaluation at CNPq, Capes, and Faperj, I wanted to know how the main global agencies operate (I have also collaborated with the National Science Foundation of the United States), and understand what we can learn from them. Six years later, it has been well worth it.
The main tension in this area concerns the use of "scientometrics": to what extent can research evaluation be based on quantitative metrics such as the number of citations of scientific articles or the impact factor of the journals in which they are published?
The European Research Commission's (ERC) position is crystal clear: like the NSF, the ERC has formally adhered to the San Francisco Declaration on Research Assessment ( DORA ), which bans the use of such metrics, stating that "the scientific content of an article is far more important than publication metrics or the name of the respective scientific journal." Thus, its judgments are based exclusively on qualitative (subjective!) assessments of the content of each project, carried out by renowned experts. This is how the ERC has built a highly respected standard of excellence, one of the rare points of consensus in the complex European landscape.
More than 3,500 scientific institutions worldwide have already joined DORA, and it's high time that Brazilian scientific agencies did the same (the Serrapilheira Institute is already a signatory). Why does it matter? Judging scientific content is more work than using numerical metrics. But it is fundamental to guaranteeing a high standard of scientific evaluation, which is an indispensable condition for the advancement of science .
Are we heading in the right direction? Years ago, when I coordinated mathematics at CNPq, the president, Professor Erney Camargo, praised the fact that we were the most qualitative area in our evaluations, while, ironically, certain areas of the humanities used purely quantitative criteria. "Mathematicians know very well what numbers cannot do for us," I explained.