Does High Frequency Social Media Data Improve Forecasts of Low Frequency Consumer Confidence Measures? Steven F. Lehrer Tian Xie Tao Zeng NBER Working Paper No. 26505

Issued in November 2019

NBER Program(s):Asset Pricing

Social media data presents challenges for forecasters since one must convert text into data and deal with issues related to these measures being collected at different frequencies and volumes than traditional financial data. In this paper, we use a deep learning algorithm to measure sentiment within Twitter messages on an hourly basis and introduce a new method to undertake MIDAS that allows for a weaker discounting of historical data that is well-suited for this new data source. To evaluate the performance of approach relative to alternative MIDAS strategies, we conduct an out of sample forecasting exercise for the consumer confidence index with both traditional econometric strategies and machine learning algorithms. Irrespective of the estimator used to conduct forecasts, our results show that (i) including consumer sentiment measures from Twitter greatly improves forecast accuracy, and (ii) there are substantial gains from our proposed MIDAS procedure relative to common alternatives. You may purchase this paper on-line in .pdf format from SSRN.com ($5) for electronic delivery. Access to NBER Papers You are eligible for a free download if you are a subscriber, a corporate associate of the NBER, a journalist, an employee of the U.S. federal government with a ".GOV" domain name, or a resident of nearly any developing country or transition economy. If you usually get free papers at work/university but do not at home, you can either connect to your work VPN or proxy (if any) or elect to have a link to the paper emailed to your work email address below. The email address must be connected to a subscribing college, university, or other subscribing institution. Gmail and other free email addresses will not have access. E-mail:

Supplementary materials for this paper:

online appendix Acknowledgments Machine-readable bibliographic record - MARC, RIS, BibTeX Document Object Identifier (DOI): 10.3386/w26505