International Journal of Computer Applications (0975 – 8887)

Volume 139 – No.11, April 2016

Sentiment Analysis of Twitter Data: A Survey of

Techniques
Vishal A. Kharde S.S. Sonawane
Department of Computer Engg, Department of Computer Engg,
Pune Institute of Computer Technology,Pune Pune Institute of Computer Technology,Pune
University of Pune (India) University of Pune (India)

ABSTRACT their products or services in such a way that it can be offered as

With the advancement of web technology and its growth, there per the user‟s requirements.
is a huge volume of data present in the web for internet users Textual Information retrieval techniques mainly focus on
and a lot of data is generated too. Internet has become a processing, searching or analyzing the factual data present.
platform for online learning, exchanging ideas and sharing Facts have an objective component but,there are some other
opinions. Social networking sites like Twitter, Facebook, textual contents which express subjective characteristics. These
Google+ are rapidly gaining popularity as they allow people contents are mainly opinions, sentiments, appraisals, attitudes,
to share and express their views about topics, have discussion and emotions, which form the core of Sentiment Analysis
with different communities, or post messages across the world. (SA). It offers many challenging opportunities to develop new
There has been lot of work in the field of sentiment analysis of applications, mainly due to the huge growth of available
twitter data. This survey focuses mainly on sentiment analysis information on online sources like blogs and social networks.
of twitter data which is helpful to analyze the information in For example, recommendations of items proposed by a
the tweets where opinions are highly unstructured, recommendation system can be predicted by taking into
heterogeneous and are either positive or negative, or neutral in account considerations such as positive or negative opinions
some cases. In this paper, we provide a survey and a about those items by making use of SA.
comparative analyses of existing techniques for opinion mining
like machine learning and lexicon-based approaches, together 2. SENTIMENT ANALYSIS
with evaluation metrics. Using various machine learning Sentiment analysis can be defined as a process that automates
algorithms like Naive Bayes, Max Entropy, and Support Vector mining of attitudes, opinions, views and emotions from text,
Machine, we provide research on twitter data streams.We have speech, tweets and database sources through Natural Language
also discussed general challenges and applications of Processing (NLP). Sentiment analysis involves classifying
Sentiment Analysis on Twitter. opinions in text into categories like "positive" or "negative" or
"neutral". It's also referred as subjectivity analysis, opinion
Keywords mining, and appraisal extraction.
Twitter, Sentiment analysis (SA), Opinion mining, Machine
learning, Naive Bayes (NB), Maximum Entropy, Support The words opinion, sentiment, view and belief are used
Vector Machine (SVM). interchangeably but there are differences between them.

1. INTRODUCTION  Opinion: A conclusion open to dispute (because different

experts have different opinions )
Nowadays, the age of Internet has changed the way people
express their views, opinions. It is now mainly done through  View: subjective opinion
blog posts, online forums, product review websites, social
media ,etc. Nowadays, millions of people are using social  Belief: deliberate acceptance and intellectual assent
network sites like Facebook, Twitter, Google Plus, etc. to
 Sentiment: opinion representing one„s feelings
express their emotions, opinion and share views about their
daily lives. Through the online communities, we get an An example for terminologies for Sentiment Analysis is as
interactive media where consumers inform and influence others given below,
through forums. Social media is generating a large volume of
sentiment rich data in the form of tweets, status updates, blog <SENTENCE> = The story of the movie was weak and boring
posts, comments, reviews, etc. Moreover, social media <OPINION HOLDER> =<author>
provides an opportunity for businesses by giving a platform to
connect with their customers for advertising. People mostly <OBJECT> = <movie>
depend upon user generated content over online to a great <FEATURE> = <story>
extent for decision making. For e.g. if someone wants to buy a
product or wants to use any service, then they firstly look up its <OPINION >= <weak><boring>
reviews online, discuss about it on social media before taking a
<POLARITY> = <negative>
decision. The amount of content generated by users is too vast
for a normal user to analyze. So there is a need to automate Sentiment Analysis is a term that include many tasks such as
this, various sentiment analysis techniques are widely used. sentiment extraction, sentiment classification, subjectivity
classification, summarization of opinions or opinion spam
Sentiment analysis (SA)tells user whether the information
detection, among others. It aims to analyze people's sentiments,
about the product is satisfactory or not before they buy it.
, attitudes, opinions emotions, etc. towards elements such as,
Marketers and firms use this analysis data to understand about
products, individuals, topics ,organizations, and services.

5
 International Journal of Computer Applications (0975 – 8887)
Volume 139 – No.11, April 2016

Mathematically we can represent an opinion as a quintuple classes. They experimented with models such as: unigram
(o, f, so, h, t), where model, a feature based model and a tree kernel based model.
For tree kernel based model they represented tweets as a
o = object; tree.The feature based model uses 100 features and the unigram
f = feature of the object o; model uses over 10,000 features. They arrived on a conclusion
that features which combine prior polarity of words with their
so= orientation or polarity of the opinion on feature f of object parts-of-speech(pos) tags are most important and plays a major
o; rolein the classification task. The tree kernel based model
h = opinion holder; outperformed the other two models.

t = time when the opinion is expressed. Davidov et al.,(2010) [7] proposed a approach to utilize Twitter
user-defined hastags in tweets as a classification of sentiment
Object:An entity which can be a, person, event, product type using punctuation, single words, n-grams and patterns as
,organization, or topic different feature types, which are then combined into a single
feature vector for sentiment classification. They made use of
Feature:An attribute (or a part) of the object with respect to
K-Nearest Neighbor strategy to assign sentiment labels by
which evaluation is made.
constructing a feature vector for each example in the training
Opinion orientation or polarity:The orientation of an opinion and test set.
on a feature f represent whether the opinion is positive,
Po-Wei Liang et.al.(2014) [8] used Twitter API to collect
negative or neutral .
twitter data. Their training data falls in three different
Opinion holder: The holder of an opinion is the person or categories (camera, movie , mobile). The data is labeled as
organization or an entitythat expresses the opinion . positive, negative and non-opinions. Tweets containing
opinions were filtered. Unigram Naive Bayes model was
In recent years a lot of work has been done in the field of implemented and the Naive Bayes simplifying independence
“Sentiment Analysis on Twitter“ by number of researchers. In assumption was employed. They also eliminated useless
its early stage it was intended for binary classification which features by using the Mutual Information and Chi square
assigns opinions or reviews to bipolar classes such as positive feature extraction method. Finally , the orientation of an tweet
or negative only. is predicted. i.e. positive or negative.
Pak and Paroubek(2010) [1] proposed a model to classify the Pablo et. al. [9] presented variations of Naive Bayes classifiers
tweets as objective, positive and negative. They created a for detecting polarity of English tweets. Two different variants
twitter corpus by collecting tweets using Twitter API and of Naive Bayes classifiers were built namely Baseline (trained
automatically annotating those tweets using emoticons. Using to classify tweets as positive, negative and neutral), and Binary
that corpus, theydeveloped a sentiment classifier based on the (makes use of a polarity lexicon and classifies as positive and
multinomial Naive Bayes method that uses features like N- negative. Neutral tweets neglected). The features considered by
gram and POS-tags. The training set they used was less classifiers were Lemmas (nouns, verbs, adjectives and
efficient since it contains only tweets having emoticons. adverbs), Polarity Lexicons, and Multiword from different
Parikh and Movassate(2009) [2] implemented two models, a sources and Valence Shifters.
Naive Bayes bigram model and a Maximum Entropy model to Turney et al [11] used bag-of-words method for sentiment
classify tweets. They found that the Naive Bayes classifiers analysis in which the relationships between words was not at
worked much better than the Maximum Entropy model. all considered and a document is represented as just a
Go and L.Huang (2009) [3] proposed a solution for sentiment collection of words. To determine the sentiment for the whole
analysis for twitter data by using distant supervision, in which document, sentiments of every word was determined and those
their training data consisted of tweets with emoticons which values are united with some aggregation functions.
served as noisy labels. They build models using Naive Bayes, Kamps et al. [12] used the lexical database WordNet to
MaxEnt and Support Vector Machines (SVM). Their feature determine the emotional content of a word along different
space consisted of unigrams, bigrams and POS. They dimensions. They developed a distance metric on WordNet and
concluded that SVM outperformed other models and that determined semantic polarity of adjectives.
unigram were more effective as features.
Xia et al. [13] used an ensemble framework for Sentiment
Barbosa et al.(2010) [4] designed a two phase automatic Classification which is obtained by combining various feature
sentiment analysis method for classifying tweets. They sets and classification techniques. In thier work, they used two
classified tweets as objective or subjective and then in second types of feature sets (Part-of-speech information and Word-
phase, the subjective tweets were classified as positive or relations) and three base classifiers (Naive Bayes, Maximum
negative. The feature space used included retweets, hashtags, Entropy and Support Vector Machines) . They applied
link, punctuation and exclamation marks in conjunction with ensemble approaches like fixed combination, weighted
features like prior polarity of words and POS. combination and Meta-classifier combination for sentiment
Bifet and Frank(2010) [5] used Twitter streaming data classification and obtained better accuracy.
provided by Firehouse API , which gave all messages from Luoet. al. [14] highlighted the challenges and an efficient
every user which are publicly available in real-time. They techniques to mine opinions from Twitter tweets. Spam and
experimented multinomial naive Bayes, stochastic gradient wildly varying language makes opinion retrieval within Twitter
descent, and the Hoeffding tree. They arrived at a conclusion challenging task.
that SGD-based model, when used with an appropriate learning
rate was the better than the rest used. A General model for sentiment analysis is as follows,
Agarwal et al. (2011)[6] developed a 3-way model for
classifying sentiment into positive, negative and neutral

6
 International Journal of Computer Applications (0975 – 8887)
Volume 139 – No.11, April 2016

Sample Tweets http://goo.gl/UQv 667 tweets

dx
Stanford Movie http://ai.stanford. 50000 movie
dataset Reviews edu/~amaas/data/ reviews
sentiment/
Stanford Tweets http://cs.stanford. 4 million
edu/people/alecm tweets
go/trainingandtest categorized as
data.zip positive and
negative
Spam Spam http://myleott.co 400 deceptive
dataset Reviews m/op_spam and 400
truthful
reviews in
positive and
negative
category.
Soe dataset Sarcasm http://nlds.soe.uc 1,000
Fig.1. Sentiment Analysis Architecture and nasty sc.edu/iac discussions,
reviews ~390,000
Following are the phases required for sentiment analysis of posts, and
twitter data, some ~
73,000,000
words
2.1 Pre-processing of the datasets
A tweet contains a lot of opinions about the data which are
expressed in different ways by different users .The twitter 2.2 Feature Extraction
dataset used in this survey work is already labeled into two The preprocessed dataset has many distinctive properties. In
classes viz. negative and positive polarity and thus the the feature extraction method, we extract the aspects from the
sentiment analysis of the data becomes easy to observe the processed dataset. Later this aspect are used to compute the
effect of various features. The raw data having polarity is positive and negative polarity in a sentence which is useful for
highly susceptible to inconsistency and redundancy. determining the opinion of the individuals using models like
Preprocessing of tweet include following points, unigram, bigram [18].

 Remove all URLs (e.g. www.xyz.com), hash tags (e.g. Machine learning techniques require representing the key
#topic), targets (@username) features of text or documents for processing. These key
features are c o n s i d e r e d as feature vectors which are used
 Correct the spellings; sequence of repeated characters is to for the classification task..Some examples features that have
be handled been reported in literature are:
 Replace all the emoticons with their sentiment. 1. Words And Their Frequencies:
 Remove all punctuations ,symbols, numbers Unigrams, bigrams and n-gram models with their frequency
counts are considered as features. There has been more
 Remove Stop Words research on using word presence rather than frequencies to
better describe this feature. Panget al. [23] showed better
 Expand Acronyms(we can use a acronym dictionary) results by using presence instead of frequencies.
 Remove Non-English Tweets
2. Parts Of Speech Tags
Table 1. Publicly Available Datasets For Twitter Parts of speech like adjectives, adverbs and somegroups of
verbs and nouns are good indicators of subjectivity and
HASH Tweets http://demeter.inf 31,861 Pos sentiment. We can generate syntactic dependency patterns by
.ed.ac.uk tweets 64,850 parsing or dependency trees.
Neg tweets,
125,859 Neu
tweets
3. Opinion Words And Phrases
Apart from specific words, some phrases and idioms which
EMOT Tweets http://twittersenti 230,811 Pos&
convey sentiments can be used as features.
and ment.appspot.co 150,570 Neg
e.g. cost someone an arm and leg.
Emoticons m tweets
ISIEVE Tweets www.i-sieve.com 1,520 Pos
4. Position Of Terms
tweets,200 Neg
The position of a term with in a text can affect on how much
tweets, 2,295
the term makes difference in overall sentiment of the text.
Neu tweets
Columbia Tweets Email: 11,875 tweets
univ.dataset [email protected] 5. Negation
Negation is an important but difficult feature to interpret. The
bia.edu
presence of a negation usually changes the polarity of the
Patient Opinions http://patientopin 2000 patient
opinion..
dataset ion.org.uk opinions

7
 International Journal of Computer Applications (0975 – 8887)
Volume 139 – No.11, April 2016

e.g., I am not happy. margin reduces indecisive decisions. SVM also supports
classification and regression which are useful for statistical
6. Syntax learning theory and it also helps recognizing the factors
Syntactic patterns like collocations are used as features to precisely, that needs to be taken into account, to understand it
learn subjectivity patterns by many of the researchers. successfully.

2.3 Training 3. APPROACHES FOR SENTIMENT

Supervised learning is an important technique for solving ANALYSIS
classification problems. Training the classifier makes it easier There are mainly two techniques for sentiment analysis for the
for future predictions for unknown data. twitter data:

2.4 Classification 3.1 Machine Learning Approaches

Machine learning based approach uses classification technique
2.4.1 Naive Bayes: to classify text into classes. There are mainly two types of
It is a probabilistic classifier and can learn the pattern of machine learning techniques
examining a set of documents that has been categorized [9]. It
compares the contents with the list of words to classify the
3.1.1. Unsupervised learning:
documents to their right category or class. Let d be the tweet
It does not consist of a category and they do not provide with
and c* be a class that is assigned to d, where
the correct targets at all and therefore rely on clustering.
C   arg macc PNB (c | d )
3.1.2. Supervised learning:
m It is based on labeled dataset and thus the labels are provided to
( P(c)) p( f | c)
ni ( d )
the model during the process. These labeled dataset are trained
PNB (c | d )  i 1 to get meaningful outputs when encountered during decision-
making.
P(d )
The success of both this learning methods is mainly depends on
From the above equation, „f‟ is a „feature‟, count of feature (fi) the selection and extraction of the specific set of features used
is denoted with ni(d) and is present in d which represents a to detect sentiment.
tweet. Here, m denotes no. of features.
The machine learning approach applicable to sentiment
Parameters P(c) and P(f|c) are computed through maximum analysis mainly belongs to supervised classification. In a
likelihood estimates, and smoothing is utilized for unseen machine learning techniques, two sets of data are needed:
features. To train and classify using Naïve Bayes Machine
Learning technique ,we can use the Python NLTK library . 1. Training Set
2. Test Set.

2.4.2 Maximum Entropy A number of machine learning techniques have been

In Maximum Entropy Classifier, no assumptions are taken formulated to classify the tweets into classes. Machine learning
regarding the relationship in between the features extracted techniques like Naive Bayes (NB), maximum entropy (ME),
from dataset. This classifier always tries to maximize the and support vector machines (SVM) have achieved great
entropy of the system by estimating the conditional distribution success in sentiment analysis.
of the class label. Machine learning starts with collecting training dataset. Nextly
Maximum entropy even handles overlap feature and is same as we train a classifier on the training data. Once a supervised
logistic regression method which finds the distribution over classification technique is selected, an important decision to
classes. The conditional distribution is defined as MaxEnt make is to select feature. They can tell us how documents are
makes no independence assumptions for its features, unlike represented.
Naive Bayes. The most commonly used features in sentiment classification
The model is represented by the following: are

exp[ i fi (c, d )]
Term presence and their frequency

PME (c | d ,  )  i Part of speech information

 c
exp[ i fi (c, d )]  Negations
i  Opinion words and phrases
Where c is the class,d is the tweet and λiis the weight
vector.The weight vectors decide the importance of a feature in
classification.

2.4.3 Support Vector Machine:

Support vector machine analyzes the data, define the decision
boundaries and uses the kernels for computation which are
performed in input space[15]. The input data are two sets of
vectors of size m each. Then every data which represented as a
vector is classified into a class. Nextly we find a margin
between the two classes that is far from any document. The
distance defines the margin of the classifier, maximizing the

8
 International Journal of Computer Applications (0975 – 8887)
Volume 139 – No.11, April 2016

3.2.2. Corpus-Based:
The corpus-based approach have objective of providing
dictionaries related to a specific domain. These dictionaries are
generated from a set of seed opinion terms that grows through
the search of related words by means of the use of either
statistical or semantic techniques.
 Methods based on statistics: Latent Semantic
Analysis (LSA).
 Methods based on semantic such as the use of
synonyms and antonyms or relationships from
thesaurus like WordNet may also represent an
interesting solution.
According to the performance measures like precision and
recall, we provide a comparative study of existing techniques
for opinion mining, including machine learning, lexicon-based
Fig.2 Sentiment Classification Based On Emoticons approaches, cross domain and cross-lingual approaches, etc., as
shown in Table 2.
With respect to supervised techniques, support vector machines
(SVM), Naive Bayes, Maximum Entropy are some of the most
Table 2. Performance Comparison Of Sentiment Analysis
common techniques used.
Methods
Whereas semi-supervised and unsupervised techniques are
Method Data Set Acc. Author
proposed when it is not possible to have an initial set of labeled
documents/opinions to classify the rest of items
Machine SVM Movie 86.40% Pang,
3.2 Lexicon-Based Approaches Learning reviews Lee[23]
Lexicon based method [20] uses sentiment dictionary with
opinion words and match them with the data to determine CoTraining Twitter 82.52% Liu[14]
polarity. They assigns sentiment scores to the opinion SVM
words describing how Positive, Negative and Objective the Stanford
words contained in the dictionary are. Deep Sentimen 80.70% Richard[18]
learning t
Lexicon-based approaches mainly rely on a sentiment Treebank
lexicon, i.e., a collection of known and precompiled sentiment Lexical Product Turkey
terms, phrases and even idioms, developed for traditional based Corpus reviews 74.00%
genres of communication, such as the Opinion Finder lexicon;
Amazon‟ --- Taboada[20]
Dictionary s
Mechani
cal Turk
Ensemble Amazon 81.00% Wan,X[16]
Cross-
lingual Co-Train Amazon, 81.30% Wan,X.[16]
ITI68
IMDb >90% Abbasi,A.
EWGA movie
review
CLMM MPQA,N 83.02% Mengi
TCIR,ISI
Active Book, 80% Li, S
Cross- Learning DVD, (avg)
domain Electroni Bollegala[22
Fig 3.Lexicon-Based Model cs,
Thesaurus ]
There are Two sub classifications for this approach: SFA Kitchen Pan S J[15]
3.2.1.Dictionary-based:
It is based on the usage of terms (seeds) that are usually 4. SENTIMENT ANALYSIS TASKS
collected and annotated manually. This set grows by searching Sentiment analysis is a challenging interdisciplinary task
the synonyms and antonyms of a dictionary. An example of which includes natural language processing, web mining and
that dictionary is WordNet, which is used to develop a machine learning. It is a complex task and can be decomposed
thesaurus called SentiWordNet. into following tasks, viz:
Drawback : Can‟t deal with domain and context specific
orientations.  Subjectivity Classification

 Sentiment Classification

9
 International Journal of Computer Applications (0975 – 8887)
Volume 139 – No.11, April 2016

 Complimentary Tasks
5.1 Document level
o ObjectHolderExtraction It deals with tagging individual documents with their
o Object/Feature Extraction sentiment. In Document level the whole document is classify
eitherinto positive or negative class.
General Approach:
Find the sentiment polarities of individual sentences or words
and combine them together to find the polarity of the
document.
Other approaches:
Complex linguistic phenomena like co-reference resolution,
pragmatics, etc.
Various Tasks involved in this are:

 Task: Sentiment Classification of whole document

Fig.4 Sentiment Analysis Tasks

 Classes: Positive, negative and neutral

A . Subjectivity classification  Assumption :Each Document focuses on a single object

(not true in discussion posts, blogs, etc.) and contain
Subjectivity classification is the task of classifying sentences as opinion from a single opinion holder
opinionated or not opinionated.
5.2 Sentence or phrase level
Let S = {s1, . . . , sn} be a set of sentences in document D. The Sentence-level Sentiment Analysis deals with tagging
problem of subjectivity classification is to identify sentences individual sentences with their respective sentiment polarities.
used to represent opinions and other forms of Sentence level sentiment classification classifies sentence into
subjectivity(subjective sentences set Ss) from sentences used to positive, negative or neutral class.
objectively present factual information (objective sentences set
So), where SsUSo=S. General approach:
find the sentiment orientation of individual words in the
B. Sentiment Classification sentence/phrase and then to combine them to determine the
Once the task of finding whether a sentence is opinionated is sentiment of the whole sentence or phrase.
d o n e , we have to find the polarity of the sentence i.e.,
whether it expresses a positive or negative opinion. Sentiment Other approaches:
classification can be a binary classification (positive or consider discourse structure of the text
negative),multi-class classification(extremely negative, Various Tasks involved in this are:
negative, neutral, positive or extremely positive),regression or
ranking .
 Task 1: Identifying Subjective/ Objective Sentences
Depending upon the application of sentiment analysis, subtasks Classes: Objective and Subjective
of opinion holder extraction and object feature extraction c a n  Task 2: Sentiment Classification of Sentences
b e t r e a t e d a s optional.
Classes: positive and negative
C. Complimentary Tasks Assumption: A sentence contains only one
 OpinionHolder Extraction Opinion which may not always be true

It is the discovery of opinion holders or sources. Detection

of opinion holder is to recognize direct or indirect sources
5.3 Aspect level or Feature level
It deals with labeling each word with their sentiment and also
of opinion.
identifying the entity towards which the sentiment is directed.
 Object /Feature Extraction Aspect or Feature level sentiment classification concerns with
identifying and extracting product features from the source
It is the discovery of the target entity. data. Techniques like dependency parser and discourse
structures are used in this.
5. LEVELS OF SENTIMENT ANALYSIS
Tasks described in the previous section can be done at several Various Tasks involved in this are:
levels of granularity.
 Task1: Identify and extract object features that have been
commented on by an opinion holder (eg. A reviewer)

 Task2: Determining whether the opinions on features are

negative, positive or neutral

 Task 3: Find feature synonyms

Fig.5 Levels Of Sentiment Analysis

10
 International Journal of Computer Applications (0975 – 8887)
Volume 139 – No.11, April 2016

5.4 Word Level 500 0.612375089222

Most recentworks have used the prior polarity of words and 1000 0.652301927195
phrases for sentiment classification at sentence and document 5000 0.697403640257
levels Word sentiment classification use mostly adjectives as 10000 0.712928265525
features but adverbs, 0.717389364739
15000
The two methods of automatically annotating sentiment at the
20000 0.722764989293
word level are:
(1) Dictionary-Based Approaches 25000 0.729478943612
(2) Corpus-Based Approaches. 30000 0.729122055675
35000 0.73244557459
6. EVALUATION OF SENTIMENT 40000 0.733226266952
CLASSIFICATION 45000 0.736549785867
The performance of sentiment classification can be evaluated
by using four indexes calculated as the following equations: Following are the details on most informative features after the
classifier is executed on train data.
Accuracy = (TP+TN)/(TP+TN+FP+FN)
sad = True neg : pos = 37.6 : 1.0
Precision = TP/(TP+FP)
worst. = True neg :pos = 32.4 : 1.0
Recall = TP/(TP+FN)
crying = True neg : pos = 24.7 : 1.0
F1 = (2×Precision×Recall)/(Precision+Recall)
fml = True neg : pos = 24.1 : 1.0
In which TP, FN, FP and TN refer respectively to the number
of true positive instances, the number of false hurts = True neg : pos = 21.2 : 1.0
negativeinstances, the number of false positive instances and awful = True neg : pos = 21.1 : 1.0
the number of true negative instances, as defined in the table 1.
ugh. = True neg :pos = 20.4 : 1.0
Table 3. Confusion Matrix
terrible = True neg : pos = 20.4 : 1.0
Predicted Predicted boo. = True neg :pos = 19.2 : 1.0
Positives Negatives cancelled = True neg : pos = 19.2 : 1.0
Actual Positive TP FN
Actual Negative FP TN B. Naïve Bayes Algorithm:

7. RESULTS AND DISCUSSION Effect of Stopwords

We used the twitter dataset publicly made available by WhenNaiveBayes(Baseline)wasrun,itgaveanaccuracyof73.65pe
Stanford university. Analyses was done on this labeled datasets rcent,whichisconsideredasthebaselineresult.Thenextthingusedw
using various feature extraction technique. We used the asr e m o v a l of
framework where the preprocessor is applied to the raw stopword.WhenstopwordswereremovedandNaiveBayeswasrun,i
sentences which make it more appropriate to understand. tgaveanaccuracyof74.56percent.Following table shows the
Further, the different machine learning techniques trains the accuracy obtained at different sizes for the Naïve Bayes with
dataset with feature vectors and then the semantic analysis stopwords removed and using preprocessed data and based on
offers a large set of synonyms and similarity which provides unigram model.
the polarity of the content. Table 5. Accuracy of Naïve Bayes Algorithm (Stopword
removal+unigram)
Dataset Description:
Dataset Accuracy
Train Data 45000 10 0.522305496074
Negative 23514 50 0.583333333333
Positive 21486 100 0.593839221984
500 0.649134546752
Test Data 44832 1000 0.673536759458
Negative 22606 5000 0.7005710207
Positive 22226 10000 0.717300142755
15000 0.725486259814
A. Baseline Algorithm:
20000 0.731441827266
The baseline algorithm used is Naïve Bayeswithout
preprocessed data and unigram model. Following table shows 25000 0.734653818701
the accuracy obtained at different sizes for the baseline 30000 0.738891862955
algorithm. 35000 0.740743219129
40000 0.742148465382
Table 4. Accuracy of Baseline Algorithm 45000 0.745605817273
Dataset Accuracy
Most Informative FeaturesFor Naïve Bayes with stopwords
10 0.46475731620
removed and unigram model are,
50 0.533324411135
100 0.54744379015 bummed = True neg : pos = 34.8 : 1.0

11
 International Journal of Computer Applications (0975 – 8887)
Volume 139 – No.11, April 2016

disappointed = True neg : pos = 28.8 : 1.0 Effect of using Trigram:.

sad = True neg : pos = 27.6 : 1.0 Running Naïve Bayes using Trigrams, bigrams and unigrams
awful = True neg : pos = 20.3 : 1.0 together gave an accuracy of 75.41 percent which is less than
ugh = True neg : pos = 19.3 : 1.0 the accuracy obtained when Bigrams were used as a feature.
poor = True neg : pos = 19.3 : 1.0 Also this feature combination bloats up the feature space
sucks = True neg : pos = 18.7 : 1.0 exponentially and the execution becomes extremely slow.
upset = True neg : pos = 18.0 : 1.0 Hence for further analysis, the trigrams are not considered as
argh = True neg : pos = 17.3 : 1.0 they do not have a notice able impact on the accuracy.
battery = True neg : pos = 16.6 : 1.0 Following table shows the accuracy obtained at different sizes
for the Naïve Bayes algorithm with Trigram model.
The results are slightly different; this was the case even with
Linear SVC. This shows that stopwords really affect the Table 7. Accuracy of Naïve Bayes Algorithm (Stopword
predictions. An intuition to this can be obtained from the fact removal+Trigram)
that given the short length of tweets, people generally use
stopwords such as and, while, before, after and so on. Thus Dataset Accuracy
removal of stopwords makes a lot of difference to the accuracy. 10 0.486995895789
50 0.528484118487
Effect of Bigram: 100 0.581571199143
Bigram uses a combination of two words as a feature. Bigram 500 0.634346002855
effectively captures some features in the data that unigram fails 1000 0.654331727338
to capture. For example, words like ‟not sad‟, ‟not good‟ 5000 0.703403818701
clearly say that the sentiment is negative. This effect can be 10000 0.721002855103
clearly seen from the increase in accuracy from 15000 0.731352605282
74.56(Unigram) to 76.44 percent which is almost a 2%
20000 0.737419700214
increase. Following table shows the accuracy obtained at
25000 0.742148465382
different sizes for the Naïve Bayes algorithm with bigram
30000 0.74823786581
model.
35000 0.748773197716
Table 6. Accuracy of Naïve Bayes Algorithm (Stopword 40000 0.753234296931
removal+Bigram) 45000 0.754171127766
Dataset Accuracy
The most informative features for Naive Bayes with Trigrams
10 0.544990185582
as features.
50 0.593593861527
100 0.591407922912 ('so', 'sad') = True neg :pos = 59.1 : 1.0
500 0.654956281228 ('lost', 'my') = True neg :pos = 38.9 : 1.0
1000 0.67193076374 ('i', 'miss', 'my') = True neg :pos = 36.9 : 1.0
5000 0.718214668094 ('going', 'to', 'miss') = True neg :pos = 28.5 : 1.0
10000 0.730973411849
0.740609386153 ('miss', 'him') = True neg :pos = 25.4 : 1.0
15000
20000 0.746431120628 ('happy', "mother's", 'day') = True pos :neg = 25.0 : 1.0
25000 0.75073608137 ("can't", 'sleep') = True neg :pos = 21.5 : 1.0
30000 0.755041042113 ('sad', 'that') = True neg :pos = 21.5 : 1.0
35000 0.758453783012
('miss', 'my') = True neg :pos = 21.4 : 1.0
40000 0.762892576731
45000 0.764476266952 ('i', 'lost') = True neg :pos = 20.9 : 1.0
Following graph shows the summary of the results obtained by
The most informative features for Naive Bayes with Bigrams using different features and variation in the naïve bayes
as features. algorithm.
('so', 'sad') = True neg :pos = 55.2 : 1.0
sad. = True neg :pos = 44.2 : 1.0
bummed = True neg : pos = 33.8 : 1.0
horrible = True neg : pos = 32.0 : 1.0
('USERNAME', 'welcome') = True pos :neg = 29.5 : 1.0
('welcome', 'to') = True pos :neg = 28.1 : 1.0
sad = True neg : pos = 27.5 : 1.0
('i', 'lost') = True neg :pos = 24.7 : 1.0
died = True neg : pos = 24.3 : 1.0
('miss', 'him') = True neg :pos = 24.1 : 1.0

Fig.6 Graph Representing Different results obtained for

Naïve Bayes Algorithm.

12
 International Journal of Computer Applications (0975 – 8887)
Volume 139 – No.11, April 2016

Table 8. Accuracy of Naïve Bayes Algorithm accuracy consistently in-between NaiveBayes and SVM.
Also it runs iteratively and takes a large amount of time to
Algorithm Accuracy run. Hence MaxEnt was not used for all the feature
Naïve Bayes (unigram) 74.56 combinations.
Naïve Bayes (bigram) 76.44
Naïve Bayes (trigram) 75.41 Table 11. Summary Of Results For Unigram
Method Accuracy(Unigram)
C. Support Vector Machine (SVM): Baseline 73.65
Effect using unigram
Naïve Bayes 74.56
Following table shows the accuracy obtained at different sizes
SVM 76.68
for the SVM algorithm with unigram model.
Maximum Entropy 74.93
Table 9. Accuracy of SVM Algorithm (Unigram)
As the table shows, when the processing, analysis was done on
Dataset Accuracy the bigger dataset, the accuracy scaled upto a great extent.
10 0.525450571021 NaiveBayes baseline scaled upto 76.44 and SVM scaled upto
50 0.550521948608 77.73percent.The best result tested thus far, was obtained
100 0.569726980728 when SVM was used on a feature set of a combination of
500 0.6261375803 Unigram, Bigram with stopwords removal, gave an accuracy of
1000 0.660421127766 77.73. MaxEnt also performed well and gave an accuracy of
5000 0.726222341185 74.93when stopwords was removed.
10000 0.739806388294
15000 0.748973947181
20000 0.75426034975
25000 0.758096895075
30000 0.76130888651
35000 0.762847965739
40000 0.76556923626
45000 0.766862955032

Effect using Bigram

Following table shows the accuracy obtained at different sizes
for the SVM algorithm with Bigram model.
Table 10. Accuracy of SVM Algorithm (Bigram)
Dataset Accuracy
10 0.500223054961 Fig.7 Graph Representing Different results obtained for
0.574232690935 Naïve Bayes Algorithm And Linear SVC (SVM).
50
100 0.56437366167 8. CHALLENGES IN SENTIMENT
500 0.632293897216
1000 0.657989828694 ANALYSIS
5000 0.725486259814 Sentiment Analysis is a very challenging task. Following are
some of the challenges[13] faced in Sentiment Analysis of
10000 0.746609564597
Twitter.
15000 0.756468593862
20000 0.761487330478 1. Identifying subjective parts of text:
25000 0.767375981442 Subjective parts represent sentiment-bearing content. The same
30000 0.771011777302 word can be treated as subjective in one case, or an objective in
35000 0.77210474661 some other. This makes it difficult to identify the subjective
40000 0.775941291934 portions of text.
45000 0.777324232691
For example:
Table 10. Summary for Accuracy of SVM Algorithm 1. The language of the Mr.Dennis was very crude.
2. Crude oil is obtained by extraction from the sea beds.
Algorithm Accuracy The word „crude‟ is used as an opinion in first example, while
SVM with unigram 76.68 it is completely objective inthe second example.
SVM with bigram 77.73
2. Domain dependence[24]:
D. Maximum Entropy The same sentence or phrase can have different meanings in
In Maximum Entropy Classifier, no assumptions are taken different domains. For Example, the word „unpredictable‟ is
regarding the relationship between features.we obtained an positive in the domain of movies, dramas ,etc, but if the same
accuracy of 74.93 percent with unigram model word is used in the context of a vehicle's steering, then it has a
negative opinion.
With all the features considered, the results show that SVM
outperforms NaiveBayes and maximum entropy as well in all 3. Sarcasm Detection:
cases .In particular, the feature combination of Slang Sarcastic sentences express negative opinion about a target
stopwords removal and Bigram gives the maximum accuracy using positive words in unique way..
of 77.73 with SVM. Maximum Entropy model gives an Example:

13
 International Journal of Computer Applications (0975 – 8887)
Volume 139 – No.11, April 2016

“Nice perfume. You must shower in it.” 9. APPLICATIONS OF SENTIMENT

The sentence contains only positive words but actually it
expresses a negative sentiment. ANALYSIS
Sentiment Analysis has many applications in various Fields.
4. Thwarted expressions: 1.Applications that use Reviewsfrom Websites:
There are some sentences in which only some part of text
determines the overall polarity of the document. Today Internet has a large collection of reviews and feedbacks
on almost everything. This includes product reviews, feedbacks
Example: on political issues, comments about services, etc. Thus there is
“This Movie should be amazing. It sounds like a great plot, the a need for a sentiment analysis system that can extract
popular actors , and the supporting cast is talented as well. “ sentiments about a particular product or services. It will help us
to automate in provision of feedback or rating for the given
In this case,a simple bag-of-words approaches will term it as product, item, etc. This would serve the needs of both the users
positive sentiment, but the ultimate sentiment is negative. and the vendors.
5. Explicit Negation of sentiment: 2. Applications as a Sub-component Technology
Sentiment can be negated in many ways as opposed to using
simple no, not, never, etc. It is difficult to identify such A sentiment predictor system can be helpful in recommender
negations . systems as well. The recommender system will not recommend
items that receive a lot of negative feedback or fewer ratings.
Example:
“It avoids all suspense and predictability found in Hollywood In online communication, we come across abusive language
movies.” and other negative elements. These can be detected simply by
identifying a highly negative sentiment and correspondingly
Here the words suspense and predictable bear a negative taking action against it.
sentiment, the usage of „avoids‟ negatestheir respective
sentiments. 3. Applications in Business Intelligence

6. Order dependence: It has been observed that people nowadays tend to look upon
Discourse Structure analysis is essential for Sentiment reviews of products which are available online before they buy
Analysis/Opinion Mining. them. And for many businesses, the online opinion decides the
Example: success or failure of their product. Thus, Sentiment Analysis
A is better than B, conveys the exact opposite opinion from, B plays an important role in businesses. Businesses also wish to
is better than A. extract sentiment from the online reviews in order to improve
their products and in turn their reputation and help in customer
7. Entity Recognition: satisfaction .
There is a need to separate out the text about a specific entity 4. Applications across Domains:
and then analyze sentiment towards it.
Recentresearches in sociology and other fields like medical,
Example: sports have also been benefitted by Sentiment Analysis that
“I hate Microsoft, but I like Linux”. show trends in human emotions especially on social media.

A simple bag-of-words approach will label it as neutral, 5. Applications In Smart Homes

however, it carries a specific sentiment for both the entities Smart homes are supposed to be the technology of the future.
present in the statement. In future entire homes would be networked and people would
8. Building a classifier for subjective vs. objective tweets. be able to control any part of the home using a tablet device.
Recently there has been lot of research going on Internet of
Current research work focuses mostly on classifying positive
Things(IoT). Sentiment Analysis would also find its way in
vs. negative correctly. There is need to look at classifying
IoT. Like for example, based on the current sentiment or
tweets with sentiment vs. no sentiment closely.
emotion of the user, the home could alter its ambiance to create
9. Handling comparisons. a soothing and peaceful environment.
Bag of words model doesn't handle comparisons very well. Sentiment Analysis can also be used in trend prediction. By
Example: tracking public views, important data regarding sales trends and
"IIT‟s are better than most of the private colleges", the tweet customer satisfaction can be extracted.
would be considered positive for both IIT‟s and private
colleges using bag of words model because it doesn't take into 10. CONCLUSION
account the relation towards "better". In this paper, we provide a survey and comparative study of
existing techniques for opinion mining including machine
10. Applying sentiment analysis to Facebook messages. learning and lexicon-based approaches, together with cross
There has been less work on sentiment analysis on Facebook domain and cross-lingual methods and some evaluation
data mainly due to various restrictions by Facebook graph api metrics. Research results show that machine learning methods,
and security policies in accessing data. such as SVM and naive Bayes have the highest accuracy and
can be regarded as the baseline learning methods, while
11. Internationalization [16,17].
lexicon-based methods are very effective in some cases, which
Current Research work focus mainly on English content, but require few effort in human-labeled document .We also studied
Twitter has many varied users from across. the effects of various features on classifier. We can conclude
that more the cleaner data, more accurate results can be
obtained. Use of bigram model provides better sentiment

14
 International Journal of Computer Applications (0975 – 8887)
Volume 139 – No.11, April 2016

accuracy as compared to other models. We can focus on the [12] J. Kamps, M. Marx, R. J. Mokken, and M. De Rijke,
study of combining machine learning method into opinion “Using wordnet to measure semantic orientations of
lexicon method in order to improve the accuracy of sentiment adjectives,” 2004.
classification and adaptive capacity to variety of domains and
different languages. [13] R. Xia, C. Zong, and S. Li, “Ensemble of feature sets and
classification algorithms for sentiment classification,”
11. REFERENCES Information Sciences: an International Journal, vol. 181,
[1] A.Pak and P. Paroubek. „Twitter as a Corpus for Sentiment no. 6, pp. 1138–1152, 2011.
Analysis and Opinion Mining". In Proceedings of the [14] ZhunchenLuo, Miles Osborne, TingWang, An effective
Seventh Conference on International Language Resources approachto tweets opinion retrieval", Springer Journal
and Evaluation, 2010, pp.1320-1326 onWorldWideWeb,Dec 2013, DOI: 10.1007/s11280-013-
[2] R. Parikh and M. Movassate, “Sentiment Analysis of 0268-7.
User- GeneratedTwitter Updates using Various [15] Liu, S., Li, F., Li, F., Cheng, X., &Shen, H.. Adaptive co-
Classi_cation Techniques",CS224N Final Report, 2009 training SVM for sentiment classification on tweets. In
[3] Go, R. Bhayani, L.Huang. “Twitter Sentiment Proceedings of the 22nd ACMinternational conference on
ClassificationUsing Distant Supervision". Stanford Conference on information & knowledgemanagement (pp.
University, Technical Paper,2009 2079-2088). ACM,2013.

[4] L. Barbosa, J. Feng. “Robust Sentiment Detection on [16] Pan S J, Ni X, Sun J T, et al. “Cross-domain sentiment
Twitterfrom Biased and Noisy Data". COLING 2010: classification viaspectral feature alignment”. Proceedings
Poster Volume,pp. 36-44. of the 19th internationalconference on World wide web.
ACM, 2010: 751-760.
[5] Bifet and E. Frank, "Sentiment Knowledge Discovery
inTwitter Streaming Data", In Proceedings of the 13th [17] Wan, X..“A Comparative Study of Cross-Lingual
InternationalConference on Discovery Science, Berlin, SentimentClassification”. In Proceedings of the The 2012
Germany: Springer,2010, pp. 1-15. IEEE/WIC/ACMInternational Joint Conferences on Web
Intelligence and IntelligentAgent Technology-Volume 01
[6] Agarwal, B. Xie, I. Vovsha, O. Rambow, R. Passonneau, (pp. 24-31).IEEE Computer Society.2012
“Sentiment Analysis of Twitter Data", In Proceedings of
the ACL 2011Workshop on Languages in Social [18] Socher, Richard, et al. "Recursive deep models for
Media,2011 , pp. 30-38 semanticcompositionality over a sentiment Treebank."
Proceedings of theConference on Empirical Methods in
[7] Dmitry Davidov, Ari Rappoport." Enhanced Sentiment Natural Language Processing(EMNLP). 2013.
Learning Using Twitter Hashtags and Smileys". Coling
2010: Poster Volumepages 241{249, Beijing, August [19] Meng, Xinfan, et al. "Cross-lingual mixture model for
2010 sentimentclassification." Proceedings of the 50th Annual
Meeting of theAssociation for Computational Linguistics
[8] Po-Wei Liang, Bi-Ru Dai, “Opinion Mining on Social Volume 1,2012
MediaData", IEEE 14th International Conference on
Mobile Data Management,Milan, Italy, June 3 - 6, 2013, [20] Taboada, M., Brooke, J., Tofiloski, M., Voll, K., &Stede,
pp 91-96, ISBN: 978-1-494673-6068-5, M..“Lexicon basedmethods for sentiment analysis”.
http://doi.ieeecomputersociety.org/10.1109/MDM.2013. Computational linguistics, 2011:37(2), 267-307.

[9] Pablo Gamallo, Marcos Garcia, “Citius: A Naive-Bayes [21] Li, S., Xue, Y., Wang, Z., & Zhou, G..“Active learning for
Strategyfor Sentiment Analysis on English Tweets", 8th cross-domainsentiment classification”. In Proceedings of
InternationalWorkshop on Semantic Evaluation (SemEval the Twenty-Thirdinternational joint conference on
2014), Dublin, Ireland,Aug 23-24 2014, pp 171-175. Artificial Intelligence (pp. 2127-2133).AAAI Press,2013

[10] Neethu M,S and Rajashree R,” Sentiment Analysis in [22] Bollegala, D., Weir, D., & Carroll, J.. Cross-Domain
Twitter using Machine Learning Techniques” 4th SentimentClassification using a Sentiment Sensitive
ICCCNT 2013,at Tiruchengode, India. IEEE – 31661 Thesaurus. Knowledge andData Engineering, IEEE
Transactions on, 25(8), 1719-1731,2013
[11] P. D. Turney, “Thumbs up or thumbs down?: semantic
orientation applied to unsupervised classification of [23] Pang, B.and Lee, L. “A sentimental education: Sentiment
reviews,” in Proceedings of the 40th annual meeting on analysis using subjectivity summarization based on
association for computational linguistics, pp. 417–424, minimum cuts”. 42nd Meeting of the Association for
Association for Computational Linguistics, 2002. Computational Linguistics[C] (ACL-04). 2004, 271-278.
[24] V. M. K. Peddinti and P. Chintalapoodi, “Domain
adaptation in sentiment analysis of twitter,” in Analyzing
Microtext Workshop, AAAI, 2011.

IJCATM : www.ijcaonline.org 15