图书情报工作

图书情报工作 >

2018 , Vol. 62 >Issue 11: 23 - 34

DOI: https://doi.org/10.13266/j.issn.0252-3116.2018.11.003

理论研究

基于深度循环神经网络的跨领域文本情感分析

  • 余传明
展开
  • 中南财经政法大学信息与安全工程学院 武汉 430073
余传明(ORCID:0000-0001-7099-0853),副教授,E-mail:yucm@zuel.edu.cn。

收稿日期: 2017-09-06

  修回日期: 2018-03-04

  网络出版日期: 2018-06-05

基金资助

本文系国家自然科学基金面上项目"大数据环境下基于领域知识获取与对齐的观点检索研究"（项目编号：71373286）和国家自然科学基金青年项目"突发公共卫生事件社交媒体信息主题演化与影响力建模"（项目编号：71603189）研究成果之一。

收起

A Cross-domain Text Sentiment Analysis Based on Deep Recurrent Neural Network

  • Yu Chuanming
Expand
  • School of Information and Safety Engineering, Zhongnan University of Economics and Law, Wuhan 430073

Received date: 2017-09-06

  Revised date: 2018-03-04

  Online published: 2018-06-05

Fold

摘要

[目的/意义]通过在标注资源丰富的源领域（Source Domain）中学习，并将目标领域（Target Domain）的文档投影到与源领域相同的特征空间（Feature Space）中去，从而解决目标领域因标注数据量较小而难以获得好的分类模型的问题。[方法/过程]选择亚马逊在书籍、DVD和音乐类目下的中文评论作为实验数据，以跨领域情感分析作为研究任务，提出一种跨领域深度循环神经网络（Cross Domain Deep Recurrent Neural Network，CD-DRNN）模型，实现不同领域环境下的知识迁移。CD-DRNN模型在跨领域环境下的平均分类准确度达到了81.70%，优于传统的栈式长短时记忆网络（Stacked Long Short Term Memory，Stacked-LSTM）模型（79.90%）、双向长短时记忆网络模型（Bidirectional Long Short Term Memory，Bi-LSTM）模型（80.50%）、卷积神经网络长短时记忆网络串联（Convolution Neural Network with Long Short Term Memory，CNN-LSTM）（74.70%）模型以及卷积神经网络长短时记忆网络并联（Merged Convolution Neural Network with Long Short Term Memory，Merged-CNN-LSTM）模型（80.90%）。[结果/结论]源领域和目标领域的知识迁移能够有效解决监督学习在小数据集上难以获得好的分类效果的问题，通过CD-DRNN模型能够从无标注数据中有效地筛选特征，从而大大降低目标领域数据标注相关的工作量。

关键词： 跨领域; 迁移学习; 深度循环神经网络; 情感分析

本文引用格式

余传明 . 基于深度循环神经网络的跨领域文本情感分析[J]. 图书情报工作, 2018 , 62(11) : 23 -34 . DOI: 10.13266/j.issn.0252-3116.2018.11.003

Abstract

[Purpose/significance] In order to solve the problem of classification model in target domain that caused by the lack of data, this study firstly trains the model of source domain that includes rich labeling/tagging data, and then, projects source and target domain documents into the same feature space. [Method/process] The reviews of three product categories, i.e. books, DVD and music, from Amazon, which are written in Chinese, are taken as the experimental data, and the cross-domain text sentiment analysis is considered as the research task. A novel model, i.e. the Cross Domain Deep Recurrent Neural Network (CD-DRNN), is proposed to achieve knowledge transfer among domains. The average accuracy value of CD-DRNN achieves 81.70%,which excels the values of Stacked Long Short Term Memory (79.90%), Bidirectional Long Short Term Memory(80.50%), Convolution Neural Network with Long Short Term Memory (74.70%) and Merged Convolution Neural Network with Long Short Term Memory (80.90%). [Result/conclusion] Knowledge transfer in source domain and target domain could effectively solve the difficulties of achieving good classification performances on small data sets. The proposed method can be leveraged to effectively select features from unlabeled data, thereby greatly reducing the workload related to data annotation in the target domain.

Key words： cross domain; transfer learning; deep recurrent neural network; sentiment analysis

参考文献

[1] WEISS K, KHOSHGOFTAAR T M, WANG D D. A survey of transfer learning[J]. Journal of big data, 2016, 3(1):1-40.
[2] TAHMORESNEZHAD J, HASHEMI S. Visual domain adaptation via transfer feature learning[J]. Knowledge and information systems, 2017, 50(2):1-21.
[3] BIFET A, PECHENIZKIY M, BOUCHACHIA A.A survey on concept drift adaptation[J]. ACM computing surveys, 2014, 46(4):1-44.
[4] BARDDAL J P, GOMES H M, ENEMBRECK F, et al. A survey on feature drift adaptation:definition, benchmark, challenges and future directions[J]. Journal of systems and software, 2016,127:278-294.
[5] VAPNIK V, IZMAILOV R. Knowledge transfer in SVM and neural networks[J]. Annals of mathematics and artificial Intelligence, 2017,80(1-2):3-19.
[6] 徐军, 丁宇新, 王晓龙. 使用机器学习方法进行新闻的情感自动分类[J]. 中文信息学报, 2007, 21(6):95-100.
[7] 吴应良, 黄媛, 王选飞. 在线中文用户评论研究综述:基于情感计算的视角[J]. 情报科学, 2017(6):159-163.
[8] BLITZER J, MCDONALD R, PEREIRA F. Domain adaptation with structural correspondence learning[C]//Conference on empirical methods in natural language processing. Stroudsburg, PA, USA:Association for Computational Linguistics, 2006:120-128.
[9] PAN S J, NI X, SUN J T, et al. Cross-domain sentiment classification via spectral feature alignment[C]//International conference on World Wide Web, WWW 2010, Raleigh, North Carolina, U.S.A., April. DBLP, 2010:751-760.
[10] DAI W, YANG Q, XUE G R, et al. Boosting for transfer learning[C]//International conference on machine learning. New York, NY, USA:ACM, 2007:193-200.
[11] 马凤闸, 吴江宁, 杨光飞. 基于双重选择策略的跨领域情感倾向性分析[J]. 情报学报, 2012, 31(11):1202-1209.
[12] GLOROT X, BORDES A, BENGIO Y. Domain adaptation for large-Scale sentiment classification:a deep learning approach[C]//International conference on machine learning. Madison, WI, USA:Omnipress,2011:1-9.
[13] SUN M, TAN Q, DING R, et al. Cross-domain sentiment classification using deep learning approach[C]//International conference on cloud computing and intelligence systems. New York, NY, USA:IEEE, 2015:60-64.
[14] 吕韶华, 杨亮, 林鸿飞. 基于SimRank的跨领域情感倾向性分析算法研究[J]. 中文信息学报, 2012, 26(6):38-44.
[15] 魏现辉, 张绍武, 杨亮,等. 基于加权SimRank的跨领域文本情感倾向性分析[J]. 模式识别与人工智能, 2013, 26(11):1004-1009.
[16] 张志武. 跨领域迁移学习产品评论情感分析[J]. 现代图书情报技术, 2013(6):49-54.
[17] 黄瑞阳, 康世泽. 一种改进EM算法的跨领域情感分类方法[J]. 计算机应用研究, 2017, 34(9):2696-2699.
[18] XIA R, ZONG C, HU X, et al. Feature ensemble plus sample selection:domain adaptation for sentiment classification[J]. IEEE intelligent systems, 2013, 28(3):10-18.
[19] DESHMUKH J S, TRIPATHY A K. Entropy based classifier for cross-domain opinion mining[J]. Applied computing and informatics, 2017,14(1):55-64.
[20] TANG D, QIN B, LIU T. Deep learning for sentiment analysis:successful approaches and future challenges[J]. Wiley interdisciplinary reviews data mining and knowledge discovery, 2015, 5(6):292-303.
[21] 余传明, 冯博琳, 安璐. 基于深度表示学习的跨领域情感分析[J]. 数据分析与知识发现, 2017(7):73-81.
[22] YU J, JIANG J. Learning sentence embeddings with auxiliary tasks for cross-domain sentiment classification[C]//Conference on empirical methods in natural language processing. Stroudsburg, PA, USA:ACL,2016:236-246.
[23] 吴斐, 张玉红, 胡学钢. 面向评论信息的跨领域词汇情感倾向判别方法[J]. 计算机科学, 2015, 42(6):220-222.
[24] 冯超, 梁循,李亚平,等. 基于词向量的跨领域中文情感词典构建方法[J]. 数据采集与处理, 2017, 32(3):579-587.
[25] DENECKE K. Are SentiWordNet scores suited for multi-domain sentiment classification?[C]//International conference on digital information management. New York, NY, USA:IEEE, 2009:1-6.
[26] GRAVES A. Supervised sequence labelling with recurrent neural networks[M].Berlin, Heidelberg:Springer, 2012.
[27] SUTSKEVER I, VINYALS O, Le Q V. Sequence to sequence learning with neural networks[EB/OL].[2017-06-30]. https://arxiv.org/pdf/1409.3215.
[28] JOZEFOWICZ R, VINYALS O, SCHUSTER M, et al. Exploring the limits of language modeling[EB/OL].[2017-06-30]. https://arxiv.org/pdf/1602.02410.
[29] DAN G, BRUNK C, VINYALS O, et al. Multilingual language processing from bytes[EB/OL].[2017-06-30]. https://arxiv.org/pdf/1512.00103.
[30] VINYALS O, TOSHEV A, BENGIO S, et al. Show and tell:a neural image caption generator[C]//IEEE Conference on Computer Vision and Pattern Recognition. New York, NY, USA:IEEE, 2015:3156-3164.
[31] YAO Y, HUANG Z. Bi-directional LSTM recurrent neural network for Chinese word segmentation[C]//International Conference on Neural Information Processing. Cham, Switzerland:Springer, 2016:345-353.
[32] CROSS J, HUANG L. Incremental parsing with minimal features using Bi-Directional LSTM[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA, USA:ACL, 2016:32-37.
[33] PLANK B, SØGAARD A, GOLDBERG Y. Multilingual part-of-speech tagging with bidirectional long short-term memory models and auxiliary loss[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA, USA:ACL, 2016:1-7.
[34] ABDI H, WILLIAMS L J. Principal component analysis[J]. Wiley interdisciplinary reviews computational statistics, 2010, 2(4):433-459.
[35] ULLRICH C. Support vector classification[M]//Forecasting and hedging in the foreign exchange markets. Berlin Heidelberg:Springer, 2009:345-356.
[36] MENARD S. Applied logistic regression analysis[J]. Technometrics, 2002, 38(2):184-186.
[37] LANDGREBE D. A survey of decision tree classifier methodology[J]. IEEE transactions on systems man and cybernetics, 2002, 21(3):660-674.
[38] BREIMAN L. Random Forest[J]. Machine learning, 2001, 45(1):5-32.
[39] JORDAN S J, VIVIAN A, WOHAR M E. Forecasting market returns:bagging or combining?[J]. International journal of forecasting, 2017, 33(1):102-120.
[40] WANG J, GAO L, ZHANG H, et al. Adaboost with SVM-based classifier for the classification of brain motor imagery tasks[C]//International conference on universal access in human-computer interaction:users diversity. Berlin Heidelberg:Springer-Verlag, 2011:629-634.
[41] LAI S, LIU K, HE S, et al. How to generate a good word embedding[J]. IEEE intelligent systems, 2016, 31(6):5-14.
[42] DYER C, BALLESTEROS M, WANG L, et al. Transition-based dependency parsing with stack long short-term memory[J]. Computer science, 2015, 37(2):321-332.
[43] WANG J, YU L C, LAI K R, et al. Dimensional sentiment analysis using a regional CNN-LSTM model[C]//Meeting of the Association for Computational Linguistics. Stroudsburg, PA, USA:ACL, 2016:225-230.
Options
文章导航

/

〈 〉