Library and Information Service >
Examination of the Effects of Time Constraint and Task Order in User Experiments
Received date: 2014-11-17
Revised date: 2014-12-20
Online published: 2015-01-05
[Purpose/significance] User experiments are often conducted in order to measure user behavior, and by understanding user behavior, better system performance and better user experience can be achieved. This study examines the effects of time constraint and task order on participants in a user experiment and comes up with practices of suggestions. [Method/process] A user experiment was conducted to examine the effects of time constraints and task order effects on search experience. In the experiment, participants were asked to search four tasks of two types under two different time conditions. [Result/conclusion] The results demonstrated that participants had significantly lower confidence before searching and evaluated their search performance worse when there was time constraint. In addition, task order also influenced participants' search experience; i.e. they had much lower confidence in latter tasks than in previous tasks. The authors suggest that when conducting user experiments, researchers should systematically rotate the search tasks and other contextual factors examined in the experiment, e.g. Latin Square; and they should set strictly time constraint only if the time constraint effect will be examined. These findings could have practical implication for the design of user experiments in interactive information retrieval.
Key words： time constraint; task order; search experience; experimental design; Latin Square
Liu Chang , Zhao Yu , Yang Fan . Examination of the Effects of Time Constraint and Task Order in User Experiments[J]. Library and Information Service, 2015 , 59(1) : 99 -105 . DOI: 10.13266/j.issn.0252-3116.2015.01.013
[1] Liu Chang, Qu Peng, Xu Tao. Research on Web search and user search behaviors in mainland China[M]// Du Jia, Zhu Qinghua, Koronios A. Library and Information Science Research in Asia-Oceania: Theory and Practice. Hershey: Information Science Reference,2014:45-60.
[2] 周文杰. 图书馆学研究中的实验法:回顾与前瞻[J].图书与情报, 2011(2): 91-95.
[3] 李月琳,肖雪,仝晓云. 数字图书馆中人机交互维度与用户交互绩效的关系研究[J].图书情报工作,2014,58(2):38-46.
[4] Borlund P, Ingwersen P. The development of a method for the evaluation of interactive information retrieval systems[J]. Journal of Documentation, 1997, 53(3):225-250.
[5] 李月琳, 肖雪, 胡蝶. 信息检索实验中的任务设计——真实与模拟仿真工作任务的比较研究[J]. 图书情报工作, 2014, 58(16): 5-12.
[6] 张萍, 甘利人, 薛春香. 不同干预下高级信息检索方法的学习行为分析[J]. 现代情报, 2011,31(11):3-8.
[7] 白晨,甘利人. 基于内驱力理论的强化学习模型实验探索[J]. 数字图书馆论坛, 2009(8):20-25.
[8] 甘利人, 史飞,吴鹏. 不同强度干预下检索方法学习中的用户心智模型动态变化研究:以大学生为例[J]. 情报学报, 2012, 31(6): 662-672.
[9] 钱敏, 甘利人, 孙蕾, 等.基于符号表征理论的用户心智模型与网站表现模型研究:以商品分类使用分析为例[J]. 情报学报, 2012,31(10): 1110-1120.
[10] 刘佳, 沈旺, 李贺.信息检索认知模型及认知负荷评价研究[J]. 图书情报工作, 2012,56(24): 133-136.
[11] 周文杰, 数字信息分析中用户焦虑实验研究[J]. 中国图书馆学报, 2011(6):58-66.
[12] 胡秀梅,邓小昭.文献呈现次序对用户相关性判断的影响[J].图书情报工作,2010,54(14):32-35.
[13] 张云秋, 安文秀,冯佳. 探索式信息搜索行为研究[J]. 图书情报工作, 2012,56(14): 67-72.
[14] 邱瑾.用户协同信息检索行为与系统评价研究——以任务类型和协同能力为视角[J].现代图书情报技术,2012(9):62-68.
[15] Li Yuelin, Zhang Xiangmin. Individual differences and the usability of digital libraries[J]. Journal of the China Society for Scientific and Technical Information, 2011, 30(9):980-989.
[16] 潘幼乔,邓小昭.基于Zelen设计的网络用户信息检索行为实验方案与评估[J].图书情报工作, 2006,50(3):76-79.
[17] Savolainen R. Time as a context of information seeking[J]. Library & Information Science Research, 2006,28(1):110-127.
[18] Slone D J. The impact of time constraints on Internet and Web use[J]. Journal of the American Society for Information Science and Technology, 2007,58(4):508-517.
[19] Crescenzi A, Capra R, Arguello J. Time pressure, user satisfaction and task difficulty[EB/OL].[2014-08-08]. http://onlinelibrary.wiley.com/doi/10.1002/meet.14505001121/pdf.
[20] Maule A J, Hockey G R, Bdzola L. Effects of time-pressure on decision-making under uncertainty: Changes in affective state and information processing strategy[J]. Acta Psychologica, 2000,104(3):283-301.
[21] Anderson L W, Krathwohl D A. A taxonomy for learning, teaching and assessing: A revision of Bloom's taxonomy of educational objectives[M]. New York: Longman,2001:67-68.
[22] Wu Wan-Ching, Kelly D, Edwards A, et al. Grannies, Tanning Beds, Tattoos and NASCAR:Evaluation of search tasks with varying levels of cognitive complexity[C]//Proceedings of the 4th Information Interaction In Context Symposium (IIiX'12).New York: ACM, 2012: 254-257.
[23] Weenig M W, Maarleveld M. The impact of time constraint on information search strategies in complex choice tasks[J]. Journal of Economic Psychology, 2002,23(6):689-702.
[24] Kelly D. Methods for evaluating interactive information retrieval systems with users[J]. Foundations and Trends in Information Retrieval, 2009,3(1-2):1-224.
[25] Babbie E. The practice of social research[M]. Twelfth edition. Belmont: Wadsworth,2010:240.
[26] Fujikawa K, Joho H, Nakayama S I. Constraint can affect human perception, behaviour, and performance of search[EB/OL]. [2014-12-08]. http://link.springer.com/chapter/10.1007/978-3-642-34752-8_5.
/
| 〈 |
|
〉 |
