Harnessing Twitter Sentiment for Predicting Stock Market Trends Using Machine Learning

The study investigates the use of sentiment analysis, combining lexicon-based and machine learning methods, to predict stock market behavior based on financial-related tweets in Turkish. Support Vector Machine and Multilayer Perceptron models performed best, highlighting the potential of social media sentiment in forecasting stock trends.

A comprehensive study by Handan Cam from the Department of Management Information Systems, Alper Veli Cam from the Department of Health Care Management, Ugur Demirel from Irfan Can Kose Vocational School, all at Gumushane University in Turkey, and Sana Ahmed from Henley Business School, University of Reading in the UK, investigates the application of sentiment analysis to predict stock market behavior. The study focuses on financial-related tweets in Turkish, an area with limited existing research. The researchers collected 17,189 tweets posted between November 7 and November 15, 2022, related to the Borsa Istanbul (BIST30) stock index. The tweets were collected using hashtags like #Borsaistanbul, #Bist, #Bist30, and #Bist100, and the aim was to analyze public sentiment as an indicator for stock market trends. The study employs a hybrid sentiment analysis model, combining lexicon-based methods with machine learning classifiers to categorize the tweets into positive, neutral, and negative sentiments. This combination allowed the researchers to explore the effectiveness of sentiment analysis in predicting stock market behavior, a topic that has not been deeply investigated in the Turkish financial context.

Utilizing Lexicon-Based and Machine Learning Classifiers

The researchers used MAXQDA 2020, a qualitative data analysis software, to import the tweets and Orange, an open-source machine learning and data mining tool, to label the sentiment of the tweets. Orange’s multilingual sentiment analysis tool was particularly valuable as it provided sentiment scores in Turkish, making the process efficient for this study. After the preprocessing steps, the lexicon-based method categorized the tweets into positive, neutral, or negative sentiments. In total, 9,076 tweets were labeled as either positive or negative, excluding the neutral tweets for further machine learning analysis. The next step involved applying six different machine learning classifiers to conduct sentiment analysis. These classifiers included Naive Bayes, Logistic Regression, Support Vector Machine (SVM), K-Nearest Neighbor, Decision Trees, and Multilayer Perceptron (MLP), all implemented using Python’s sklearn library. The dataset was split using the 80-20 rule, with 80% of the data used for training and 20% for testing and validation. The machine learning models were trained on this dataset to predict the sentiment of the remaining tweets.

Support Vector Machine and MLP Show Superior Performance

The results showed that the Support Vector Machine and Multilayer Perceptron classifiers performed the best, with accuracy rates of 0.89 and 0.88, respectively, and AUC values of 0.8729 and 0.8647. These models significantly outperformed the other classifiers, which achieved an average accuracy of around 78.5%. The SVM and MLP models demonstrated higher precision in predicting sentiment, which is critical in understanding public mood in response to stock market trends. The study found that 34.81% of the tweets were classified as positive, 47.20% as neutral, and 17.99% as negative. This indicates that public sentiment towards the stock market during the time period studied was generally positive, though neutral opinions dominated the data. The data collection and analysis process also revealed some interesting insights regarding specific companies’ shares. For instance, tweets about Sasa Polyester Sanayi AS (SASA), Hektas Ticaret AS (HEKTS), and Eregli Demir ve Celik Fabrikalari TAS (EREGL) were particularly prevalent. These companies saw higher volumes of tweets, and their stock prices reflected the overall public sentiment expressed in the tweets. For example, SASA was mentioned in over 10,000 tweets, and the analysis suggested that public attention correlates with stock price stability or volatility.

Predicting Stock Market Behavior through Social Sentiment

One of the key contributions of the paper is its exploration of the potential for sentiment analysis to predict stock market behavior. The findings suggest that tweets expressing positive sentiments towards certain stocks can be indicators of price increases, while negative sentiments might predict a decline. However, the researchers also caution that this relationship is complex, as external factors, such as global market trends or economic policies, can influence stock prices. The authors acknowledge that the accuracy of sentiment analysis could be further improved by refining the machine learning models and expanding the dataset. They suggest that future studies could apply deep learning approaches, which have shown promise in other domains of sentiment analysis. Moreover, integrating other types of data, such as multimedia content from tweets or information from additional social media platforms, could provide a more holistic view of public sentiment.

Understanding Limitations and Complex Market Dynamics

Despite the promising results, the authors point out several limitations of the study. For instance, the dataset used in this research was limited to a specific time frame and only included tweets in Turkish. This may have excluded other relevant data that could have influenced the stock market during that period. Additionally, while the study focuses on the relationship between tweet sentiment and stock prices, it does not account for company-specific events, such as earnings reports or product launches, which could also affect stock performance. The researchers suggest that future studies could address these limitations by including data from different periods and incorporating more diverse sources of information. Overall, the paper contributes valuable insights into the use of sentiment analysis in financial contexts and highlights the potential for future research in this area.

Future Directions for Sentiment Analysis in Finance

The paper concludes that sentiment analysis, especially when applied to financial tweets, can provide meaningful insights into public opinion on stock market trends. It calls for future research to explore deeper machine learning and natural language processing methods to capture the nuances of financial sentiment more accurately. The researchers believe that expanding datasets and incorporating other social media platforms could enhance the accuracy and relevance of the predictions. They also recommend using multimodal data, such as images and links in tweets, to further improve sentiment analysis. Addressing these areas could provide a more comprehensive understanding of how public sentiment on social media influences stock market behavior, laying the groundwork for more accurate and reliable financial forecasting models.