The Review Characteristics of Expert and Anti-assessment Research

doi:10.3969/j.issn.1007-7375.2020.06.017

Abstract

Abstract: The high quality of expert database guarantees project reviews to be fair and just. Supervision and evaluation of experts review quality will benefit building high quality of expert database. Firstly whole network analysis method is used to estimate experts' power within the academic community to reflect their academic level. Secondly Pearson similarity analysis method is used to measure consistency in projects' score and project evaluation index score of experts reflect their credibility of expert. Thirdly through analyzing hit rate of project's support and no support, review professional level of experts is discussed. Finally TOPSIS evaluation method is used to evaluate review expert and verify the theoretical and application value of the research.

Key words: characteristics of expert, review level evaluations, whole network analysis, Pearson correlation, TOPSIS

CLC Number:

G311

ZHANG Yao, LIN Chun. The Review Characteristics of Expert and Anti-assessment Research[J]. Industrial Engineering Journal, 2020, 23(6): 124-130,154.

References

[1] 万猛. 关于科技评审专家的选择及其评审行为的判断方法[J]. 研究与发展管理, 2007, 19(3): 119-122, 129
WAN Meng. The selection of science and technology appraisal expert and judgement method for their appraisal behavior[J]. R&D Management, 2007, 19(3): 119-122, 129
[2] 江笑颜, 李栋亮. 多层次、智能化的项目评审专家管理体系构建研究——基于广东科研项目评审专家管理的优化[J]. 科技管理研究, 2018, 38(16): 212-217
JIANG Xiaoyan, LI Dongliang. A multi-level intelligent project evaluation expert management system: based on the optimization of expert management of Guangdong research projects[J]. Science and Technology Management Research, 2018, 38(16): 212-217
[3] 张志清, 凡艳, 苏顺华. 基于SNA的科研项目评审专家选择与回避策略研究[J]. 武汉理工大学学报 (信息与管理工程版), 2016, 38(3): 367-371
ZHANG Zhiqing, FAN Yan, SU Shunhua. Strategy for experts selection and avoidance of scientific research project evaluation based on social network analysis[J]. Journal of Wuhan University of Technology (Information & Management Engineering), 2016, 38(3): 367-371
[4] 刘丹. 国内同行评议专家库研究综述[J]. 重庆大学学报 (社会科学版), 2012, 18(5): 87-91
LIU Dan. Review of domestic research on the experts database of peer review[J]. Journal of Chongqing University (Social Science Edition), 2012, 18(5): 87-91
[5] 徐林生, 王执铨, 戴跃伟. 评审专家可信度评价模型及应用[J]. 南京理工大学学报 (自然科学版), 2010, 34(1): 30-34
XU Linsheng, WANG Zhiquan, DAI Yuewei. Expert's credibility model and evaluation method[J]. Journal of Nanjing University of Science and Technology (Natural Science), 2010, 34(1): 30-34
[6] 高鑫, 杨如艳, 官晓飞, 等. 葡萄酒评酒专家评价的可信度量化模型及应用[J]. 云南农业大学学报, 2014, 29(2): 235-240
GAO Xin, YANG Ruyan, GUAN Xiaofei, et al. Wine professional sommelier's credibility model and evaluation method[J]. Journal of Yunnan Agricultural University (Natural Science), 2014, 29(2): 235-240
[7] 李旭彦, 宋英华, 杨晓秋. 基于PageRank的评审专家信誉度度量方法[J]. 科研管理, 2016, 37(3): 133-142
LI Xuyan, SONG Yinghua, YANG Xiaoqiu. Reviewer reputation measuring approach based on PageRank algorithm[J]. Science Research Management, 2016, 37(3): 133-142
[8] 赵亚娟. 专家群评价结果可信度分析与检验[J]. 中国科学技术大学学报, 2016, 46(2): 165-172
ZHAO Yajuan. Reliability analysis of group experts and testing of evaluation outcomes[J]. Journal of University of Science and Technology of China, 2016, 46(2): 165-172
[9] 蔡文学, 温旖旎, 郑顺潮, 等. 基于偏差度权重的科技项目评审专家绩效评价方法研究[J]. 科技管理研究, 2012, 32(15): 75-78, 83
CAI Wenxue, WEN Yini, ZHENG Shunchao, et al. Research on performance evaluation method of technology projects appraising experts based on deviation rate[J]. Science and Technology Management Research, 2012, 32(15): 75-78, 83
[10] 李海峰, 党延忠. 专家反评估与项目评价相结合的科技项目评审方法研究[J]. 科学学与科学技术管理, 2010(10): 59-63
LI Haifeng, DANG Yanzhong. Expert anti-assessment and project evaluation combined project review method research[J]. Science of Science and Management of Science & Technology, 2010(10): 59-63
[11] 朱红, 方文, 李青, 等. 基于模糊评价的评标专家考评管理机制研究[J]. 湖北省社会主义学院学报, 2017(1): 92-96
ZHU Hong, FANG Wen, LI Qing, et al. Research on evaluation management mechanism of experts based on fuzzy evaluation[J]. Journal of Hubei Institute of Socialism, 2017(1): 92-96
[12] 徐雯. 基于群决策-熵模型的评标专家可靠性评价[J]. 项目管理技术, 2018, 16(10): 7-10
XU Wen. Reliability evaluation of bid evaluation experts based on entropy model of group decision[J]. Project Management Technology, 2018, 16(10): 7-10
[13] 汪建, 王裴裴, 丁俊. 科技项目专家评审的元评价综合模型研究[J]. 科研管理, 2020, 41(2): 183-192
WANG Jian, WANG Peipei, DING Jun. A comprehensive model on the meta-evaluation of science and technology projects by review experts[J]. Science Research Management, 2020, 41(2): 183-192
[14] FREEMAN L C. Centrality in networks: I. conceptual clarification[J]. Social Networks., 1979, 1(3): 215-239
[15] CHO J, KWON K, PARK Y. Implicit user credibility extraction for reputation rating mechanism in B2C e-commerce[J]. International Journal of Intelligent Information and Database Systems, 2007, 1(3): 247-263
[16] WANG C L, YOON K. Multiple attribute decision making: methods and applications[M]. NewYork: Springer Verlag, 1981.