Your current location:

20

2022

-

06

[Paper Express] The concept, connotation and key issues of multi-subject responsible evaluation in the whole process.


Abstract:Responsible evaluation has gradually emerged and received attention in the international scientific and technological community, but the theoretical research on the concept and connotation of responsible evaluation is still blank. By summarizing the relevant explorations of international representative academic organizations (DORA, HEFCE, INORMS) in promoting responsible evaluation, and further based on "objectives-elements-subjects-criteria", the concept of whole-process multi-subject responsible evaluation (REMAP) is put forward, the connotation of responsible evaluation is revealed, and four basic criteria (ATTA) and eight key issues of responsible evaluation are put forward, in order to provide inspiration and reference for China's scientific research management and evaluation practice under the "four only" in the new period.

 

Key words:Responsible Evaluation; Scientific Research Evaluation; Breaking Four Only; REMAP;ATTA

 

Abstract: In early 2020, the release of relevant documents from the Ministry of Science and Technology and the Ministry of Education further strengthened the guidance of breaking the "Siwei ", thus the voice of the scientific and technological community looking forward to building new theories and methods of scientific research evaluation is growing. From the perspective of practice at home and abroad, responsible evaluation is quietly rising in the global academic circles and has received more attention. Many well-known international academic organizations have explored the concept and connotation of responsible evaluation at different levels. However, how to understand the concept and connotation of responsible evaluation has not yet formed a systematic understanding in theory, let alone effective guidance for practical work. In order to enrich the research in this field, based on the idea of "responsible innovation", this paper summarizes the relevant exploration of international representative academic organizations(DORA, HEFCE, INORMS) in promoting responsible evaluation, puts forward the concept of whole process multi-agent responsible evaluation(REMAP) based on "target-element-subject-criterion", and reveals the connotation of responsible evaluation as well as four basic criteria(ATTA) and eight key issues of responsible evaluation. From the target level, REMAP is the extension and expansion of responsible innovation in the field of science and technology evaluation. It advocates the scientific research evaluation oriented by innovation quality, performance and contribution, and opposes the quantitative evaluation of simple machinery; It emphasizes the "responsibility" consciousness of key subjects in scientific and technological innovation activities-in the whole process of promoting and guiding scientific and technological innovation activities, we should consider not only the short-term impact, but also the positive and negative impact on all aspects of economic and social development from the perspective of future development; From the element level, REMAP is not only limited to the organization of evaluation, but also runs through the whole process of scientific research evaluation, including evaluation purpose, method design, expert selection, result use, etc; From the perspective of subject level, responsible evaluation is not only limited to scientific researchers, but also involves the key stakeholders who organize or participate in scientific and technological evaluation.Scientific research institutions, innovation groups, business circles, policy makers and research funders, third-party evaluation institutions, evaluation experts and other groups should establish a sense of responsibility and responsibility, and consciously assume their due responsibilities in scientific and technological innovation activities; From the perspective of criteria, REMAP should meet the four criteria of "Assessable, Testable, Traceable and Accountable"(ATTA). It not only ensures and promotes the implementation of responsible evaluation, but also supervises and feeds back the implementation of responsible evaluation in practice. Therefore, it can be subdivided into eight key issues in the application of responsible evaluation in practice. Specifically, from the perspective of evaluable principle, one should consider what is the purpose of the proposed scientific research evaluation? Is scientific research evaluation the best way to solve problems? From the perspective of testability principle, one should consider how to make good use of quantitative tools and quantitative evaluation methods? How to select experts and make good use of peer review? From the perspective of traceability principle, one should consider whether the responsibility division of each evaluation subject is clear and the boundary is clear? Is the evaluation result based on a complete chain of evidence? From the perspective of the principle of accountability, one should consider who is responsible for the evaluation results and the use of the results? What can be further improved in the whole assessment? Furthermore, combined with the theoretical research on remap, this paper puts forward some policy suggestions on how to implement responsible evaluation in China.

 

Keyword:responsible evaluation; research evaluation; break "Siwei"; REMAP; ATTA;

 

Author: Xu Fang Dai Tao

Author: Institute of Science and Technology Strategy Consulting, Chinese Academy of Sciences

School of Public Policy and Management, University of Chinese Academy of Sciences

The full text has been published in Science of Science and Science and Technology Management, Issue 6, 2021.

Key words:

Partners

See more>