Sentiment analysis

Sentiment analysis, also known as opinion mining, is used to analyze sentiment in text to identify and extract subjective information. This is used to help businesses understand social sentiment around a company's brand, product, or service while monitoring online conversations. With advances in deep learning, natural language processing, and computational linguistics, sentiment analysis tools have been able to move beyond basic sentiment analysis and count-based metrics to develop in-depth research. These new techniques can allow users to classify conversations based on key aspects of a brand's product or service that customers care about and users' underlying intentions and reactions around those aspects.

Sentiment analysis allows businesses to understand the sentiment of their customers towards the brand through automatic scoring of sentiment behind social media conversations, reviews, and can help businesses make more customer-focused decisions. Sentiment analysis can also help businesses understand employees and their sentiment towards the company's brand and particular subjects within the company. Sentiment analysis can also help businesses gain insight into the effectiveness of call center agents and customer support representatives and to gauge the overall opinion around a businesses products or services.

With social media, a viral review, especially a negative viral review, can have an outsized impact on a brand. However, a positive review, as shown by research done by Bain & Co, is capable of growing 4 to 8 percent revenue over the competition, increase the customer lifecycle 6 to 14 times, and improve customer retention by up to 55 percent. Automated sentiment analysis tools can drive this growth through the analysis of tweets, online reviews, and news articles. And they can give business analysts useful insights into customer opinion on brands, products, and services. While they can provide customer support directors and social media managers insight into trending issues before they can cause a brand more damage by going viral.

The cost of replacing an employee averages 20 to 30 percent of their salary, and yet 20 percent of workers voluntary leave their jobs each year, and another 17 percent are fired or let go. However, human resources teams can use data analytics and sentiment analysis to help reduce turnover and improve performance. This can, in turn, cut employee churn by understanding how employees feel and what they are discussing. This can include analytics from employee surveys, Slack messages, emails, and other communications, which can help HR teams address pain points and improve morale.

The other benefits include allowing businesses to sort data at scale in an automated and cost-effective way, and making that data available to decision makers. It offers real-time analysis which can help a company identify critical issues and take action. And it offers a consistent criteria to determine the sentiment of a particular text.

Sentiment analysis works through the use of "sentiment weights" given to descriptive words, which a basic sentiment analysis system will draw on. These sentiment weights lead to a sentiment library, which is what the sentiment analysis system uses to understand sentiment-bearing phrases it encounters. The libraries are generally large collections of adjectives and phrases that have been scored by human coders. However, this manual process can be a difficult process, as there has to be agreement on the strength or weakness of a specific adjective or phrase. And in the case of a multilingual sentiment analysis system, there is a requirement to maintain unique libraries for each language.

Once these libraries are prepared, software engineers are responsible for writing rules to help the computer evaluate the sentiment expressed and its nearness to known positive or negative words. Queries into this sentiment analysis system will then return a "hit count" to represent the number of times a word appears near a different adjective, and these are combined using a mathematical operation called a "log odds ratio," which generates a numerical sentiment score.

The drawback of these rules-based sentiment analysis systems are their simplicity, which requires a rule for every word combination in a given sentiment library. Creating and maintaining these rules requires a lot of labor, and these rules are unable to keep pace with the evolution of human language. Especially as messaging changes the traditional rules of grammar, and as rule sets cannot account for abbreviations, acronyms, double-meanings, and misspellings that appear in a given context and could change a given sentiment. In addition, some rules-based systems can fail to consider negators and intensifiers and can be inherently naïve. This can lead to false conclusions about sentiment. Further, in the case where a phrase or sentiment appears that is not accounted for, a proper score cannot be assigned.

When used with sentiment analysis systems, machine learning can improve and automate the low-level text analytics functions sentiment analysis relies on, including part of speech tagging. Data scientists can train a machine learning model to identify nouns through training of data sets with pre-tagged examples, and through supervised and unsupervised learning techniques, the machine learning model can learn what a noun "looks like." Once this model is ready, it can be applied to building new models to identify other parts of speech, and the result can be quick and reliable part of speech tagging that helps larger text analytics systems identify sentiment-bearing phrases more effectively.

Machine learning can also help analysts solve problems caused by the evolution of language. For example, a phrase with different meanings cannot necessarily have a rule set capable of accounting for every potential meaning, but if fed into a machine learning model with a few thousand pre-tagged examples, it can learn to understand what the phrase means in context, such as a phrase used in a video game context versus the same phrase used in a healthcare context and the variation in the possible meanings given the context.

As suggested by the name, a hybrid sentiment analysis system combines machine learning with traditional rules to improve on the deficiencies of either approach. Most hybrid sentiment analysis systems combine machine learning with software rules across the entire text analytics function stack, from low-level tokenization and syntax analysis, to the highest-levels of sentiment analysis.

Through tracking sentiment, organizations can see which customers are more opinionated than others. For instance, many believe 80 percent of customer issues come from 20 percent of buyers. But through the use of sentiment analysis this statistic can be found to either be true or untrue, and the groups can be segmented based on the qualities of the group and either fix common issues or avoid those buyers.

Through the analysis of customer opinions and sentiment, businesses can understand what customers like or dislike about a given product or service, and in some cases even offer what they would prefer to see instead. This can allow businesses to update software products or improve the design of physical goods while drawing on customer sentiment, or even offer new products or services based on customer sentiment.

While customer sentiment is often negative, negative feedback is often useful as it can identify customer service or other issues and allow businesses to improve overall customer service processes to benefit customers and increase customer satisfaction.

Sentiment analysis is a metric worth checking in short periods of time, and it is an important metric to continuously monitor. This can track changes in opinions as processes and products are improved or changed, and this can allow businesses to see these changes and better navigate through customers sentiment changes. Further, it can allow businesses to see whether the changes or improvements, if based on previous sentiment analysis, correlate to an improvement in sentiment around the given product or service.