[目的/意义]信息搜索是人们常用的信息查询方法，目前搜索系统在查找事实型信息时支持效果较好，但是对人们以学习为目的的搜索功能还缺乏研究。"搜索即学习"（search as learning）是近年来交互式信息检索的研究热点，这类研究中将搜索看作学习的过程，并尝试对用户搜索中的知识学习进行评估，进而提出系统支持用户学习的功能优化建议。本文着重解决如何全面评估用户搜索前后的知识水平，为此类研究提供参考。[方法/过程]采用用户实验法，对用户搜索前和搜索后撰写的知识内容进行评估，提出综合数量与质量维度的用户知识评估方法，对用户在学习型任务搜索前后知识水平进行评估。数据分析阶段采用统计方法来验证用户搜索后与搜索前的知识水平差异。[结果/结论]研究发现，用户在知识数量上的表现随着搜索的完成而变得更加全面和深入，在知识点数量、知识面数量、知识面广度和知识面深度上都有显著的提升。同时，在搜索后产生了专业度较高的知识面。对于搜索前较模糊的某些概念，在搜索后表达得更清晰明确。在质量上，搜索后绝大多数的用户都在知识的相关性、分析程度及用户观点的提出方面有所提升。
[Purpose/significance] Information search extensively influence people's daily lives, and people tend to use search engine to find the information they need. However, the existing search system is better for supporting factual tasks search, and it is not satisfactory to support tasks such as searching in order to learn. In recent years, researchers have begun to pay attention to regarding search as learning, and try to evaluate the knowledge learning in search.[Method/process] For the purpose of mastering users' learning status in search thoroughly, in this paper, 32 students from Peking University were recruited to complete user experiment. User knowledge assessment method considering knowledge quantity and quality comprehensively was established in this paper. Based on this knowledge assessment method, user knowledge level in learning tasks before and after search was evaluated respectively, and the changes of knowledge level were investigated.[Result/conclusion]The research shows that users' performance on knowledge quantity dimension becomes more comprehensive and in-depth with the search completion, and has a significant improvement in the number of knowledge points, the number of knowledge facets, the breadth of knowledge facets and the depth of knowledge facets. At the same time, there are a number of relatively professional knowledge facets after searching. Some vague knowledge concept expressions before searching were expressed more clearly and explicitly after searching. In the dimension of knowledge quality, knowledge relevance, analysis and user opinion, the vast majority of users improve after searching. This paper provides a feasible way for researchers to evaluate user knowledge learning in information search from the perspective of multi-angle evaluation of knowledge status before and after searching.
[1] RIEH S Y, COLLINS-THOMPSON K, HANSEN P, et al. Towards searching as a learning process[J]. Journal of information science, 2016, 42(1):19-34.
[2] GWIZDKA J, HANSEN P, HAUFF C, et al. Search as learning (SAL) workshop 2016[C]//Proceedings of International ACM SIGIR conference on research and development in information retrieval. New York:ACM, 2016:1249-1250.
[3] RIEH S Y, GWIZDKA J, FREUND L, et al. Searching as learning:novel measures for information interaction research[J]. Proceedings of the American Society for Information Science & Technology, 2014, 51(1):1-4.
[4] MARCHIONINI G. Exploratory search:from finding to understanding[J]. Communications of the ACM, 2006, 49(4):41-46.
[5] 张云秋, 安文秀, 于双成. 探索式搜索中用户认知的实验研究[J]. 情报理论与实践, 2013, 36(6):73-77.
[6] ALLAN J, CROFT B, MOFFAT A, et al. Frontiers, challenges, and opportunities for information retrieval:report from SWIRL 2012 the second strategic workshop on information retrieval in Lorne[J]. Comptes rendus des séances de la société de biologie et de ses filiales, 2012, 161(12):2355-2359.
[7] AGOSTI M, FUHR N, TOMS E, et al. Evaluation methodologies in information retrieval dagstuhl seminar 13441[J]. ACM SIGIR forum, 2014, 48(1):36-41.
[8] WHITE R W, ROTH RA. Exploratory search:beyond the queryresponse paradigm[C]//Proceeding of Synthesis lectures on information concepts retrieval and services. Williston:Morgan & Claypool Publishers, 2009:1-98.
[9] 袁红, 李秋. 搜索任务和搜索能力对用户探索式搜索行为的影响研究[J]. 图书情报工作, 2015, 59(15):94-105.
[10] 张云秋, 安文秀, 冯佳. 探索式信息搜索行为研究[J].图书情报工作, 2012, 56(14):67-72.
[11] EICKHOFF C, TEEVAN J, WHITE R, et al. Lessons from the journey:a query log analysis of within-session learning[C]//Proceedings of ACM conference on Web search and data mining. New York:ACM, 2014:223-232.
[12] COLLINS-THOMPSON K, RIEH S Y, HAYNES C C, et al. Assessing learning outcomes in Web search:a comparison of tasks and query strategies[C]//Proceedings of ACM conference on human information interaction and retrieval. New York:ACM, 2016:163-172.
[13] LAZONDER A W, BIEMANS H J A, WOPEREIS I G J H. Differences between novice and experienced users in searching information on the World Wide Web[J]. Journal of the American society for information science & technology, 2000, 51(6):576-581.
[14] SAITO H, MIWA K. A cognitive study of information seeking processes in the WWW:the effects of searcher's knowledge and experience[C]//Proceedings of the Second international conference on Web information systems engineering (WISE'01) IEEE Computer Society. Washington:IEEE Computer Society, 2001:321-327.
[15] WHITE R W, DUMAIS S T, TEEVAN J. Characterizing the influence of domain expertise on Web search behavior[C]//Proceedings of the second ACM international conference on Web search and data mining. New York:ACM, 2009:132-141.
[16] KOROBILI S, MALLIARI A, ZAPOUNIDOU S. Factors that influence information-seeking behavior:the case of greek graduate students[J]. Journal of academic librarianship, 2011, 37(2):155-165.
[17] LIU C, ZHANG L, SONG X. Are self-assessment of search ability and performance reliable?[C]//Proceedings of the 79th ASIS&T annual meeting:creating knowledge, enhancing lives through information & technology. Maryland:American Society for Information Science Silver Springs, 2016:87.
[18] HERSH W R, ELLIOT D L, HICKAM D H, et al. Towards new measures of information retrieval evaluation[C]//International ACM SIGIR conference on research and development in information retrieval. New York:ACM, 1995:164-170.
[19] NELSON L, HELD C, PIROLLI P, et al. With a little help from my friends:examining the impact of social annotations in sensemaking tasks[C]//Proceedings of the SIGCHI conference on human factors in computing systems. New York:ACM, 2009:1795-1798.
[20] KAMMERER Y, NAIRN R, PIROLLI P, et al. Signpost from the masses:learning effects in an exploratory social tag search browser[C]//Proceedings of the SIGCHI conference on human factors in computing systems. New York:ACM, 2009:625-634.
[21] WILLOUGHBY T, ANDERSON S A, WOOD E, et al. Fast searching for information on the Internet to use in a learning context:the impact of domain knowledge[J]. Computers & education, 2009, 52(3):640-648.
[22] WILSON M J, WILSON M L. A comparison of techniques for measuring sensemaking and learning within participant-generated summaries[J]. Journal of the American Society for Information Science & Technology, 2013, 64(2):291-306.
[23] BLOOM B S, ENGELHART M D, FURST E J, et al. Taxonomy of educational objectives:the classification of educational goals. Handbook I:Cognitive domain[M]. London:Longmans,1956:58.
[24] LEE J. Does higher education foster critical and creative learners? An exploration of two universities in South Korea and the USA[J]. Higher education research & development, 2015, 34(1):1-16.
[25] 刘畅, 赵瑜, 杨帆. 信息检索用户实验设计中时间限制和任务次序的影响研究[J]. 图书情报工作, 2015, 59(1):99-105.
[26] LIU J, BELKIN N J, ZHANG X, et al. Examining users' knowledge change in the task completion process[J]. Information processing & management, 2013, 49(5):1058-1074.
