[Purpose/significance] In order to explore whether the measurement results of Altmetrics indicators are consistent in the evaluation of subject papers, this paper attempts to build a comprehensive literature evaluation model based on Altmetrics through the existing measurement data, and to create new metrics for the evaluation and reuse of data.[Method/process] In this study, the Altmetrics indicators of all articles published in the Civil Engineering disciplines from 2009 to 2016 on the PLoS ALMs are obtained. The 15 original indicators are divided into 4 principal components using correlation analysis and principal component analysis methods, which are applied to specific disciplines. It analyzes the correlation between the single index and the comprehensive evaluation index of Altmetrics, and uses the fairness test method to explore the differences in the relevance of time to the evaluation of the paper.[Result/conclusion] The study finds that after using the fairness test method to eliminate time impact, the relevance of the citation indicators has increased, while the indicators related to social media communication have declined. But the Twitter index has not decreased, and it has always been highly correlated with the F value.
Liu Hongxu
,
Wang Zheng
. Study on Altmetrics Academic Quality Evaluation Method Based on Fairness Test[J]. Library and Information Service, 2018
, 62(16)
: 102
-110
.
DOI: 10.13266/j.issn.0252-3116.2018.16.012
[1] PRIEM J, TARABORELLI D, GROTH P, et al. Altmetrics:a manifesto[EB/OL].[2018-05-21]. http://altmetrics.org/manifesto.
[2] LIU J. Metrics and beyond@SpotOn London 2012[EB/OL].[2017-11-02]. https://www.altmetric.com/blog/metrics-and-beyond-spoton-london-2012/.
[3] NISO.NISO to develop standards and recommended practices for Altmetrics[EB/OL].[2017-11-02].http://www.niso.org/news/pr/view?item_key=72efc1097d4caf7b7b5bdf9c54a165818399ec86.
[4] Proquest. Altmetric data now available in the Summon Discovery Service[EB/OL].[2017-11-02].http://www.proquest.com/about/news/2016/Altmetric-data-now-available-in-the-Summon-Discovery-Service.html.
[5] 余厚强, 邱均平. 论替代计量学在图书馆文献服务中的应用[J]. 情报杂志, 2014, 33(9):163-166.
[6] BRODY T, HARNAD S, CARR L. Earlier Web usage statistics as predictors of later citation impact[J]. Journal of the association for information science & technology, 2006, 57(8):1060-1072.
[7] WAN J K, HUA P H, ROUSSEAU R, et al. The journal download immediacy index (DⅡ):experiences using a Chinese full-text database[J]. Scientometrics, 2010, 82(3):555-566.
[8] YAN K K, GERSTEIN M. The spread of scientific information:insights from the web usage statistics in PLoS article-level metrics[J]. PLoS one, 2011, 6(5):e19917.
[9] WANG X, LIU C, FANG Z, et al. From attention to citation, what and how does altmetrics work?[EB/OL].[2017-11-02]. https://arxiv.org/ftp/arxiv/papers/1409/1409.4269.pdf.
[10] ALHOORI H, FURUTA R. Do altmetrics follow the crowd or does the crowd follow altmetrics?[C]//Proceedings of the 14th ACM/IEEE-CS joint conference on digital libraries. Piscataway:IEEE Press, 2014:375-378.
[11] BORNMANN L. Alternative metrics in scientometrics:a meta-analysis of research into three altmetrics[J]. Scientometrics, 2015, 103(3):1123-1144.
[12] COSTAS R, ZAHEDI Z, WOUTERS P. Do "altmetrics" correlate with citations? Extensive comparison of altmetric indicators with citations from a multidisciplinary perspective[J]. Journal of the Association for Information Science and Technology, 2015, 66(10):2003-2019.
[13] ZUCCALA A A, VERLEYSEN F T, CORNACCHIA R, et al. Altmetrics for the humanities:comparing goodreads reader ratings with citations to history books[J]. Aslib journal of information management, 2015, 67(3):320-336.
[14] CHEN K, TANG M, WANG C, et al. Exploring alternative metrics of scholarly performance in the social sciences and humanities in Taiwan[J]. Scientometrics, 2015, 102(1):97-112.
[15] THELWALL M, HAUSTEIN S, LARIVIÈRE V, et al. Do altmetrics work? Twitter and ten other social web services[J]. PloS ONE, 2013, 8(5):e64841.
[16] PRIEM J, PIWOWAR H A, HEMMINGER B M. Altmetrics in the wild:using social media to explore scholarly impact[EB/OL].[2017-11-02].https://arxiv.org/abs/1203.4745.
[17] LIN J. A case study in anti-gaming mechanisms for altmetrics:PLoS ALMs and DataTrust[EB/OL].[2017-11-02].http://altmetrics.org/altmetrics12/lin/.
[18] 余厚强.替代计量指标的换算[EB/OL].[2017-11-02]. http://blog.sciencenet.cn/blog-441629-893201.html.
[19] 刘春丽. 基于PLOS API的论文影响力选择性计量指标研究[J]. 图书情报工作, 2013, 57(7):89-95.
[20] 顾立平. 论文级别计量研究:应用案例分析[J]. 现代图书情报技术, 2013(11):1-7.
[21] 邱均平, 余厚强. 替代计量学的提出过程与研究进展[J]. 图书情报工作, 2013, 57(19):5-12.
[22] 由庆斌, 汤珊红. 补充计量学及应用前景[J]. 情报理论与实践, 2013, 36(12):6-10.
[23] 刘春丽. 论文层面计量学(Article-Level Metrics):发展过程、特点、指标与应用[J]. 图书馆杂志, 2016, 35(2):63-69.
[24] NEYLON C, WU S. Article-level metrics and the evolution of scientific impact[J]. Plos biology, 2009, 7(11):e1000242.
[25] PLOS. Article-Level Metrics[EB/OL].[2018-05-21]. https://www.plos.org/article-level-metrics.
[26] 宋玲玲. 基于PLoS ALMs的论文影响力评价指标研究——以航空航天医学为例[J]. 情报理论与实践, 2016, 39(9):30-36.
[27] 王睿, 胡文静, 郭玮. 常用Altmetrics工具比较[J]. 现代图书情报技术, 2015, 30(12):18-26.
[28] 刘晓娟, 宋婉姿. 基于PLOS ALM的altmetrics指标可用性分析[J]. 图书情报工作, 2016, 60(4):93-101.
[29] LAPINSKI S, PIWOWAR H, PRIEM J. Riding the crest of the altmetrics wave:how librarians can help prepare faculty for the next generation of research impact metrics[J]. College & research libraries news, 2013, 119(18):3359-3366.
[30] 王雯霞, 刘春丽. 不同学科间论文影响力评价指标模型的差异性研究[J]. 图书情报工作, 2017, 61(13):108-116.
[31] PIWOWAR H, PRIEM J. The power of altmetrics on a CV[J]. Bulletin of the American society for information science & technology, 2013, 39(4):10-13.
[32] RODGERS E, BARBROW S.A look at altmetrics and its growing significance to research libraries[EB/OL].[2017-11-02].https://deepblue.lib.umich.edu/handle/2027.42/99709.
[33] NIEVES G, MARIA-ISABEL D, ANTONIO C. What role do librarians play in altmetrics?[EB/OL].[2017-11-02].http://eprints.rclis.org/25481/1/Altmetrics.pdf.
[34] RADICCHI F, CASTELLANO C. Testing the fairness of citation indicators for comparison across scientific domains:The case of fractional citation counts[J]. Journal of Informetrics, 2012, 6(1):121-130.
[35] LEYDESDORFF L, RADICCHI F, BORNMANN L, et al. Field-normalized impact factors (IFs):a comparison of rescaling and fractionally counted IFs[J]. Journal of the Association for Information Science and Technology, 2013, 64(11):2299-2309.
[36] 陈福佑, 杨立英, 丁洁兰. 不同学科期刊学术影响力比较的方法与实证研究[J]. 图书情报工作, 2013, 57(23):85-89.
[37] 王睿, 胡文静, 郭玮. 高Altmetrics指标科技论文学术影响力研究[J]. 图书情报工作, 2014, 58(21):92-98.
[38] PLOS. Public Library of Science (PLOS) citationInfo[EB/OL].[2018-05-22]. http://www.lagotto.io/plos/#citationInfo.
[39] Altmetric support. How is the altmetric attention score calculated?[EB/OL].[2017-11-02]. https://help.altmetric.com/support/solutions/articles/6000060969-how-is-the-altmetric-score-calculated-.
