Sentiment Analysis for Financial News

Friday, 5 May 2017 Yige Zhao

What Is Sentiment Analysis

Sentiment Analysis is the means of applying natural language processing methods and determining subjective information in the source text.

In text analysis, the sentiment is the attitude or opinion expressed towards something. Sentiment can be positive, negative or neutral.

Why Would We Want to Do This

Emotion and psychology influence trading and investment decisions, causing people to behave in an unpredictable or irrational way.

 

Meanwhile, the glut of data makes reading everything an impossible task.

So we need sentiment analysis to:

  • Extract more information
  • Automate the analysis of unstructured content
  • Speed up the understanding
  • Limit the noise

Process of Real-time Sentiment Analysis

The process of real-time sentiment analysis can be roughly divided into the following four steps:

1.Topic Classification
2.NER
3.Sentiment Score Process
4.Visualization

1.Topic Classification on Apache Spark

We consider five different news topics: Economics, Legal, Politics, Security, and Non.

Non-topic consists of all other topics, such as Health, Technology, and Sports. Naive Bayes Algorithm from Apache Spark’s MLlib is used to train and predict news topic.

The step outlines:

  • Extract articles titles and contents
  • Tokenize the texts and remove non-alphabet characters and stop words
  • Split the articles into training and test set
  • Calculate tf-idf matrix on training set
  • Train Naive Bayes Algorithm with training set
  • Classify the test article and measure the result performance

2. Named Entity Recognition (NER)

Normally, a reader needs to know the following two questions from a piece of news:

What’s objective that the news is talking about? For example, Apple or Facebook?

In general, is it bad or good?

The technology of Named Entity Recognition (NER) is for answering the first question: What’s objective that the news is talking about? For example, Apple or Facebook?

More specifically, quickly determining which item in the text maps to proper names, such as people or places.

For InfoTrie, we need to go further to determine which company is involved in the news.
We decouple the task into two parts:
Use the popular community package like nltk and Stanford NER to narrow down the searching space.
Search for the company name using our own company synonym database.

After NER process, the news will be documented under the identified company name for delivery or further analysis.

Sometimes, one news mentioned several companies. In this scenario, relevance measure is conducted. The relevance measure considers the location of a term in the text. For example, intuitively, one news may be more relevant to a company when the name of the company occurs in the title.

 

3. Sentiment Score Process

Ordinary method:
To know whether a news is bad or good to a company, a common way is to search for the emotional states such as “angry,” “sad,” and “happy.” and count on the occurrence of these states.

Our method: In our case, we first collect a library of these emotional states specialized in

In our case, we first collect a library of these emotional states specialized in the financial community.
Next, we count on all the words that both in the library and text.
Then normalize the counting result for both positive and negative words to [0, 10], where score 0 means that all words are negative and score 10 means that all are positive.

Advantages of our method:
These scores can be treated as a quantitative measure of sentiment that can be used to compare between companies and time.

4. Visualization

Finally, both NER and sentiment scoring process is completed on the distributed computational clusters so that the analyzing result can be delivered and documented in real-time.

Practical Application: Real-time Analytics in Trading Business

Let’s see a practical application: Real-time analytics in trading business.

 

Data Feed Engine

Traders usually need to make a mass of trading decisions based on multiple dimensions of information like news, financial analysis reports, real-time quotes and so on.

Real-time Analytics Engine

With the help of the real-time analytics, the latency of the pre-decision process can be largely improved to the range of milliseconds to a few seconds once the business event has occurred.

Portal

Last but not least, an alert will send to the trader and wait for his or her final trigger. Traders become the strategy creators and decision makers instead of data collectors and processors.

 

Professional Product for Sentiment Analysis

Since Sentiment Analysis is so important, is there any professional product which has following features to do it?
Advanced Technology
A Large Number of Users
Beautiful Interface
Ultra High Processing Speed

FinSentS is a cutting edge Sentiment Analysis and News Analytics engine.

FinSentS web Dashboard indexes in real-time, in a way similar to what Google or Bing does for business news, blogs and social media feed. It scans thousands of websites, blogs, and business news publications in real-time.

KEY FEATURES of FinSentS

A Financial Google

Find everything you need. FinSentS scans millions of sources in real-time: websites, blogs, social media your private data and processes premium such as Bloomberg, Reuters or Dow Jones. It monitors 50,000 + stocks, topics, people and other assets.

Accurately Gauged Sentiment

You don’t have to read everything anymore. FinSentS’s real-time sentiment analysis can help you quickly understand insights based on sentiment. Latency is now fully real time: results come out immediately if any news arrives.

Intelligent Real-time Alert

Stop getting left behind. FinSentS can detect signals that lie hidden in vast amounts data and alert you in real time when the sentiment or the volume of news for the topics, stocks or assets you care about are abnormal.

Feed Your Own Data Sources

You have your own data subscriptions? You already have access to internal research, analyst reports? Want to analyze voice or TV? We can help you untangle a large amount of unstructured data and transform it into actionable signals.

Predict Trends of Market

Are you looking into preemptive opportunities? Our sentiment technical indicators help you detect market changes and trends, anticipate volatility, and take actions in advance that lead to desired business outcomes.

Multi-language Support

Stop wasting time translating Chinese, Japanese, Korean, French, German, Spanish, Arab, Bahasa… FinSentS’s multi-language support allows you easy access to different sources in various languages.

Take two minutes to register, save two hours every day! Get started!

Leave a Reply

Your email address will not be published. Required fields are marked *