Abstract
Image sentiment classification, which aims to predict the polarities of sentiments conveyed by the images, has gained a lot of attention. Most existing methods address this problem by training a general classifier with certain visual features, ignoring the discrepancies across domains. In this paper, we propose a novel weighted co-training method for cross-domain image sentiment classification, which iteratively enlarges the labeled set by introducing new high-confidence classified samples to reduce the gap between the two domains. We train two sentiment classifiers with both the images and the corresponding textual comments separately, and set the similarity between the source domain and the _target domain as the weight of a classifier. We perform extensive experiments on a real Flickr dataset to evaluate the proposed method, and the empirical study reveals that the weighted co-training method significantly outperforms some baseline solutions.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Siersdorfer S, Minack E, Deng F, Hare J. Analyzing and predicting sentiment of images on the social web. In Proc. International Conference on Multimedia, October 2010, pp.715-718.
Machajdik J, Hanbury A. Affective image classification using features inspired by psychology and art theory. In Proc. International Conference on Multimedia, October 2010, pp.83-92.
Borth D, Chen T, Ji R, Chang S F. SentiBank: Large-scale ontology and classifiers for detecting sentiment and emotions in visual content. In Proc. International Conference on Multimedia, October 2013, pp.459-460.
Borth D, Ji R, Chen T, Breuel T, Chang S F. Large-scale visual sentiment ontology and detectors using adjective noun pairs. In Proc. International Conference on Multimedia, October 2013, pp.223-232.
You Q, Luo J, Jin H, Yang J. Robust image sentiment analysis using progressively trained and domain transferred deep networks. arXiv: 1509.06041, 2015. https://arxiv.org/pdf/1509.06041.pdf, May 2017.
Pang B, Lee L, Vaithyanathan S. Thumbs up? Sentiment classification using machine learning techniques. In Proc. Conference on Empirical Methods in Natural Language Processing, July 2002, pp.79-86.
Baccianella S, Esuli A, Sebastiani F. SentiWordNet 3.0: An enhanced lexical resource for sentiment analysis and opinion mining. In Proc. International Conference on Language Resources and Evaluation, Volume 10, May 2010, pp.2200-2204.
Liao C, Feng C, Yang S, Huang H Y. A hybrid method of domain lexicon construction for opinion _targets extraction using syntax and semantics. Journal of Computer Science and Technology, 2016, 31(3): 595-603.
Kim Y. Convolutional neural networks for sentence classification. arXiv:1408.5882, 2014. https://arxiv.org/abs/14-08.5882, May 2017.
Li X, Pang J, Mo B, Rao Y, Wang F L. Deep neural network for short-text sentiment classification. In Proc. International Conference on Database Systems for Advanced Applications, April 2016, pp.168-175.
Ding X, Liu B, Yu P S. A holistic lexicon-based approach to opinion mining. In Proc. International Conference on Web Search and Data Mining, February 2008, pp.231-240.
Jiang F, Liu Y Q, Luan H B, Sun J S, Zhu X, Zhang M, Ma S P. Microblog sentiment analysis with emoticon space model. Journal of Computer Science and Technology, 2015, 30(5): 1120-1129.
Liu B, Zhang L. A survey of opinion mining and sentiment analysis. In Mining Text Data, Aggarwal C C, Zhai C X (eds.), Spinger, 2012, pp.415-463.
Tang D, Qin B, Liu T. Document modeling with gated recurrent neural network for sentiment classification. In Proc. the Conference on Empirical Methods in Natural Language Processing, September 2015, pp.1422-1432.
Campos V, Salvador A, Nieto X, Jou B. Diving deep into sentiment: Understanding fine-tuned CNNs for visual sentiment prediction. In Proc. International Workshop on Affect and Sentiment in Multimedia, October 2015, pp.57-62.
Zhang Y, Shang L, Jia X. Sentiment analysis on microblogging by integrating text and image features. In Proc. Pacific-Asia Conference on Knowledge Discovery and Data Mining, May 2015, pp.52-63.
Tan J, Xu M, Shang L, Jia X. Sentiment analysis for images on microblogging by integrating textual information with multiple kernel learning. In Proc. Pacific Rim International Conference on Artificial Intelligence, August 2016, pp.496-506.
Xu X, Shen F, Yang Y, Shen H T, Li X. Learning discriminative binary codes for large-scale cross-modal retrieval. IEEE Transactions on Image Processing, 2017, 26(5): 2494-2507.
Zhu X, Huang Z, Shen H T, Zhao X. Linear cross-modal hashing for efficient multimedia search. In Proc. International Conference on Multimedia, October 2013, pp.143-152.
Song J, Yang Y, Yang Y, Huang Z, Shen H T. Inter-media hashing for large-scale retrieval from heterogeneous data sources. In Proc. ACM SIGMOD International Conference on Management of Data, June 2013, pp.785-796.
Zhou G, Zhou Y, Guo X, Tu X, He T. Cross-domain sentiment classification via topical correspondence transfer. Neurocomputing, 2015, 159(2): 298-305.
Bollegala D, Mu T, Goulermas J Y. Cross-domain sentiment classification using sentiment sensitive embeddings. Transactions on Knowledge and Data Engineering, 2016, 28(2): 398-410.
Zhang Y, Hu X, Li P, Li L, Wu X. Cross-domain sentiment classification-feature divergence, polarity divergence or both? Pattern Recognition Letters, 2015, 65: 44-50.
Kisilevich S, Rohrdant C, Keim D. “Beautiful picture of an ugly place”. Exploring photo collections using opinion and sentiment analysis of user comments. In Proc. International Multiconference on Computer Science and Information Technology, October 2010, pp.419-428.
Chang C C, Lin C J. LIBSVM: A library for support vector machines. IEEE Transactions on Intelligent Systems and Technology, 2011, 2(3): 27.
Mikolov T, Sutskever I, Chen K, Corrado G S, Dean J. Distributed representations of words and phrases and their compositionality. In Proc. Advances in Neural Information Processing Systems, December 2013, pp.3111-3119.
Cao Y, Xu R, Chen T. Combining convolutional neural network and support vector machine for sentiment classification. In Proc. Social Media Processing, November 2015, pp.144-155.
Chen T, Borth D, Darrell T, Chang S F. DeepSentiBank: Visual sentiment concept classification with deep convolutional neural networks. arXiv:1410.8586, 2014. https://arxiv.org/abs/1410.8586, May 2017.
Krizhevsky A, Sutskever I, Hinton G E. ImageNet classification with deep convolutional neural networks. In Proc. Advances in Neural Information Processing Systems, December 2012, pp.1097-1105.
Shen H T, Jiang S, Tan K L, Huang Z, Zhou X. Speed up interactive image retrieval. The VLDB Journal, 2009, 18(1): 329-343.
Xu X, Liang H, Baldwin T. UNIMELB at SemEval-2016 tasks 4A and 4B: An ensemble of neural networks and a Word2Vec based model for sentiment classification. In Proc. International Workshop on Semantic Evaluation, June 2016, pp.183-189.
Yu Y, Lin H, Meng J, Zhao Z. Visual and textual sentiment analysis of a microblog using deep convolutional neural networks. Algorithms, 2016, 9(2): 41.
Tieleman T, Hinton G. Lecture 6.5-RmsProp: Divide the gradient by a running average of its recent magnitude. COURSERA: Neural Networks for Machine Learning, 2012, 4(2).
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Author information
Authors and Affiliations
Corresponding author
Electronic supplementary material
Below is the link to the electronic supplementary material.
ESM 1
(PDF 170 kb)
Rights and permissions
About this article
Cite this article
Chen, M., Zhang, LL., Yu, X. et al. Weighted Co-Training for Cross-Domain Image Sentiment Classification. J. Comput. Sci. Technol. 32, 714–725 (2017). https://doi.org/10.1007/s11390-017-1753-8
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11390-017-1753-8