January is always welcome: hope for the future. I get that hope this year from a policy speech from the Advisory Board on Science, Technology and Innovation (AWTI), published just before Christmas, about valuing science and scientists.
The House had requested the letter because of concerns about “recognition,” the new way universities try to evaluate their employees. Some scholars criticize this: they think it is too vague, too subjective, too different from what the rest of the world does, and therefore bad for the international standing of Dutch science. And this criticism reached the House of Representatives, hence the question to AWTI: How can the quality of science be objectively determined?
No, the advisory board replied, and by the way that is the wrong question. At least that’s my interpretation, and the council, of course, didn’t say that. They write policy papers there, not columns.
Quality depends on what you consider important
But they made it clear in the letter that the scientific quality does not exist. What constitutes scientific quality is always based on what you consider important. Should science solve social problems? basic knowledge production? train students? make money? This makes a huge difference in what constitutes a meaningful measure of scientific quality – a successful start is a good outcome for a scientist who has to contribute to Dutch earning capacity, but not necessarily for a scientist who wants to generate new knowledge. So AWTI prefers to talk about adjectives and plurals, which also requires several actions.
These metrics can be both quantitative and qualitative. Quantitative metrics have been particularly popular in recent decades: the number of publications, the number of citations, and the amount of subsidy received. In the new recognition and recognition, universities also want to add more qualitative criteria, such as: “ I write columns devotion In doing so, I contribute to the social discussion about how scientists should be evaluated.
Some scholars worry about these qualitative criteria, because they are “anecdotal,” and are subjective. That’s right, as AWTI writes, so are the numbers. Those who just want numbers, for example, are doing a disservice to scientists who conduct research to help policymakers: We don’t have an adequate quantitative measure for contributing to policy. And the bare numbers mask the basic story. Suppose scientist A publishes three articles a year, and scientist B only publishes one, but scientist B spends a lot of time reviewing articles by others and giving advice to colleagues, while scientist A rejects every request, so does scientist A really better seeker?
No measure of quality is completely objective. Too bad, but that’s life. However, we can try to be as objective as possible and do as much justice as possible for all the different roles and circumstances that scientists have. The new Recognition and Evaluation attempts to do this by combining quantitative and qualitative criteria.
A new method of evaluation is inevitable
AWTI sees no reason to assume that this new method of assessment threatens the international standing of Dutch science. On the contrary, it is precisely international developments that make a new assessment of scientists inevitable. The fear that the Netherlands will stand alone internationally, as the Council has shown, is unjustified. However, the Netherlands, along with the United Kingdom, Finland and Norway, is leading the way, providing a good opportunity to determine the exact course of new roads.
the position papersAnd the Toolboxes, Forms and protocols with that course ready. What matters now is whether individual scientists will follow that course. Almost every scientist is also a scholar-resident: as a member of application or promotion committees, as an evaluator for grant applications, as a department chair, as a dissertation approver.
My hope in 2023 is that, every time they evaluate their colleagues, scientists will observe all the qualities that scientists might possess, and then dare to value those qualities based on words and numbers.
