Introduction: What Bloomberg Terminal’s News Trends Feature (Doesn’t) Show

On a nondescript commute in late July, In the aftermath of the Brexit vote in England and the rise of the hugely popular game called Pokémon GO, I was listening to Bloomberg radio on my way to work. Bloomberg had taken a brief detour from Brexit and the daily musings of the market to instead focus on technology stocks and how to use a Bloomberg Terminal feature called NT (News Trends) to illustrate the information of both news trends and stock prices, aggregating topic mentions across many news sources as a time series. From the description that is all the feature of the $20,000 a month service does. Admittedly I haven’t been in front of a terminal since grad school and I do not remember the NT feature.

The awkward radio broadcast covered a game the “kids” were playing called Pokémon GO and its relationship with Nintendo’s stock. As the conversation wore on detailing how to use NT within a terminal and thereby illustrate the service’s value proposition, Hilary Clark begrudgingly stated that “the stock ticks up for Nintendo were driven… By the mentions of Pokémon GO” and later added, “the value out of this function [NT] is just to see what is driving up the stock price.”

Now, as a text miner, I take particular note of word choice. In communication theory, a message’s meaning lies with the destination not the sender or channel. This means the word choice, tone and medium have an impact on the audience’s comprehension and ascribed attitudes towards the message. You probably already know this inherently, but it is important to note: choosing your words, tone and channel correctly affects the message meaning, anyone who is married knows this firsthand :).

I am guessing you, a DataCamp blog reader, already identified a flaw in Clark’s word choice: as an authority (sender) speaking on an authoritative radio (channel) broadcast, she’s uses words like “driven” (message) in the NT service, which includes a correlation calculation. An unsophisticated consumer of this message may interpret a causal affect from her words and may even attempt to trade stocks using the NT service. However, as the sophisticated audience (destination) that we are, we own the meaning and therefore get to recreate what we think the NT service is and determines its value for ourselves. If it’s so valuable to be part of a $20,000 a month service, then we should be able to gain some novel insight using Google News Trends and Yahoo’s stock service.

Let’s make our own News Trends feature for free and see how awesome a poor man’s Bloomberg Terminal’s NT service could be!

Create Your Own R News Trends Feature

Load the Libraries

To start load the libraries we will need to make the visuals. The quantmod library is a popular R package for importing stock market data. Next, gridExtra and grid are needed to arrange the two resulting visuals. Both ggplot2 and ggthemes are used to construct the time series. ggplot2 is a great grammar of graphics library, while ggthemes provides premade palettes for easy implementation. Lastly, the gtrendsR package provides an interface to get Google Trends data.

It could be that you need to install packages if you don’t have them installed yet. In that case, use for example the install.packages("gtrendsR") command to import the gtrendsR package.

eyJsYW5ndWFnZSI6InIiLCJzYW1wbGUiOiIjIEltcG9ydCBxdWFudG1vZFxubGlicmFyeShxdWFudG1vZClcblxuIyBJbXBvcnQgZ3JpZEV4dHJhXG5saWJyYXJ5KGdyaWRFeHRyYSlcblxuIyBJbXBvcnQgZ3JpZFxubGlicmFyeShncmlkKVxuXG4jIEltcG9ydCBnZ3Bsb3QyXG5saWJyYXJ5KGdncGxvdDIpXG5cbiMgSW1wb3J0IGdndGhlbWVzXG5saWJyYXJ5KGdndGhlbWVzKVxuXG4jIEltcG9ydCBndHJlbmRzUlxubGlicmFyeShndHJlbmRzUikifQ==

Assemble the Google Trends Data

The next step is to assemble the data that will be plotted. In the code below, change the usr string to your own Gmail account. Then change psw to your password. These two objects are passed to gconnect() so that your R console can programmatically connect to the Google Trends service. Next create poke utilizing gtrends() with a search pattern, and dates. After a few moments a list is returned containing the trends information among other data. In the list each week receives an indexed score between 0-100. You can call plot() directly on this list to create the figure.

Now check out the code, and make sure to set up your Gmail variables in order to plot the graph!

eyJsYW5ndWFnZSI6InIiLCJwcmVfZXhlcmNpc2VfY29kZSI6ImxpYnJhcnkoZ3RyZW5kc1IpIiwic2FtcGxlIjoiIyBTZXQgdXAgeW91ciBHbWFpbCB2YXJpYWJsZXNcbnVzciA8LSBcInF3ZXJ0eXdAZ21haWwuY29tXCIgIFxucHN3IDwtIFwicEBzc3dvcmRcIlxuXG4jIENvbm5lY3QgdG8geW91ciBHb29nbGUgYWNjb3VudFxuZ2Nvbm5lY3QodXNyLCBwc3cpXG5cbiMgUXVlcnkgR29vZ2xlIFRyZW5kc1xucG9rZSA8LSBndHJlbmRzKCdQb2tlbW9uIEdvJywgc3RhcnRfZGF0ZSA9IFwiMjAxNi0wMS0wMVwiLCBlbmRfZGF0ZSA9IFwiMjAxNi0wOC0wMVwiKVxuXG4jIFBsb3QgdGhlIHF1ZXJ5IHJlc3VsdHNcbnBsb3QocG9rZSkifQ==

Note that gconnect() can sometimes fail for account security reasons or due to 2-step verification of your account. This will not, however, prevent you to finish the tutorial. Keep on going!

Assemble the Nintendo Stock Price Data

Next, you need to assemble the Nintendo stock price. Nintendo is traded on the Japanese stock market and is referenced by number, not name. Using setSymbolLookup() , pass in the stock index for Nintendo. Don’t forget to also add yahooj , because the data is collected from Yahoo Japan. In other words, you should specify that the stock data from YJ7974.T or Nintendo should be downloaded from Yahoo Japan. The setSymbolLookup() function is used to create a reference table of one or more ticker symbols.

The next function call getSymbols() actually retrieves the information from Yahoo Japan. The returned object is an extensible time series ( xts ) class. An xts object is similar to a data frame, but row names are dates not strings. In this case, the object contains dates from 2007 to present along with opening, closing, high, low and adjusted prices for the day. Once again, you can plot() the returned object to create this second graph:

# Specify which stock data to download and from where setSymbolLookup(YJ7974.T='yahooj') # Load YJ7974.T from Yahoo Japan getSymbols('YJ7974.T') # Plot the Yahoo data plot(YJ7974.T)

Since the raw data has a temporal mismatch, you need to first subset to dates in this year and then adjust the xts daily numbers to reconcile with the weekly trends data. Adjusting an xts object is slightly different than a data frame. Placing 2016-01/2016-8 within the index brackets subsets the time series to dates between January 2016 and August 2016. The xts class has unique grouping functions. In this case, apply.weekly() is passed the xts object along with the function, mean , to be applied to each group.

# Subset the time series nintendo.xts <- YJ7974.T['2016-01/2016-8'] # Get the weekly average of the time series nintendo.xts <- apply.weekly(nintendo.xts, mean)

If needed, the code below changes the xts object to a data frame so you can use more common functions like subset() .

# Change the `xts` object to a data frame nintendo <- data.frame(date=index(nintendo.xts), coredata(nintendo.xts)) # Subset the data frame to only include data from August 2016 nintendo.aug.16 <- subset(nintendo, format.Date(date, "%y")=="16" & format.Date(date, "%m")=="08")

Using str on the poke object, you can identify the list element containing the trend information. The poke$trend element is a traditional data frame. Within the data frame, the weekly trends vector is called pokemon.go . This vector is extracted in a new object poke.trends .

eyJsYW5ndWFnZSI6InIiLCJwcmVfZXhlcmNpc2VfY29kZSI6InBva2VUcmVuZDwtIHJlYWQuY3N2KHVybChcImh0dHA6Ly9zMy5hbWF6b25hd3MuY29tL2Fzc2V0cy5kYXRhY2FtcC5jb20vYmxvZ19hc3NldHMvcG9rZV90cmVuZHNfYXVnLmNzdlwiKSwgaGVhZGVyID0gVFJVRSlcbnBva2UgPC1saXN0KHRyZW5kPXBva2VUcmVuZCkiLCJzYW1wbGUiOiIjIFRha2UgYSBsb29rIGF0IHRoZSB0cmVuZHNcbnN0cihwb2tlJHRyZW5kKVxuXG4jIEV4dHJhY3QgdGhlIHdlZWtseSBQb2tcdTAwZTltb24gdHJlbmRzXG5wb2tlLnRyZW5kczwtcG9rZSR0cmVuZCRwb2tlbW9uLmdvIn0=

Make the Data Frame for the Visuals

Now you can assemble the data into a succinct data frame to be used in the visuals. The poorman.nt object is a data frame with nintendo.close , poke.trends and week.2016 vectors. The first references the weekly average closing price. The next contains the weekly Pokémon GO score. Lastly, week.2016 comprises a sequence of numbers between 1 and the number of rows in the nintendo object. You could append week start dates, but a simple sequence makes for a more appealing visual.

eyJsYW5ndWFnZSI6InIiLCJwcmVfZXhlcmNpc2VfY29kZSI6ImxpYnJhcnkoeHRzKVxuI0xvYWQgaW4gbmludGVuZG8gc3RvY2sgZGF0YSBcbm5pbnRlbmRvIDwtIHJlYWQuY3N2KHVybChcImh0dHA6Ly9zMy5hbWF6b25hd3MuY29tL2Fzc2V0cy5kYXRhY2FtcC5jb20vYmxvZ19hc3NldHMvbmludGVuZG9fd2Vla2x5X3N0b2NrX2F1Zy5jc3ZcIiksIGhlYWRlcj1UUlVFKVxuI0xvYWQgaW4gdGhlIHBva2UgZGF0YVxucG9rZVRyZW5kPC0gcmVhZC5jc3YodXJsKFwiaHR0cDovL3MzLmFtYXpvbmF3cy5jb20vYXNzZXRzLmRhdGFjYW1wLmNvbS9ibG9nX2Fzc2V0cy9wb2tlX3RyZW5kc19hdWcuY3N2XCIpLCBoZWFkZXIgPSBUUlVFKVxucG9rZSA8LWxpc3QodHJlbmQ9cG9rZVRyZW5kKVxucG9rZS50cmVuZHMgPC0gcG9rZSR0cmVuZCRwb2tlbW9uLmdvLiIsInNhbXBsZSI6InBvb3JtYW4ubnQ8LWRhdGEuZnJhbWUobmludGVuZG8uY2xvc2U9bmludGVuZG8kWUo3OTc0LlQuQ2xvc2UsXG4gICAgICAgICAgICAgICAgICAgICAgIHBva2UudHJlbmRzPXBva2UudHJlbmRzLFxuICAgICAgICAgICAgICAgICAgICAgICB3ZWVrLjIwMTY9c2VxKDE6bnJvdyhuaW50ZW5kbykpKSJ9

Compute the Correlation

The radio broadcast mentioned a correlation of .95. Using the code below, we use cor() to compute the correlation between nintendo.close and poke.trends , and we get pretty close!

eyJsYW5ndWFnZSI6InIiLCJwcmVfZXhlcmNpc2VfY29kZSI6ImxpYnJhcnkoeHRzKVxuI0xvYWQgaW4gbmludGVuZG8gc3RvY2sgZGF0YSBcbm5pbnRlbmRvIDwtIHJlYWQuY3N2KHVybChcImh0dHA6Ly9zMy5hbWF6b25hd3MuY29tL2Fzc2V0cy5kYXRhY2FtcC5jb20vYmxvZ19hc3NldHMvbmludGVuZG9fd2Vla2x5X3N0b2NrX2F1Zy5jc3ZcIiksIGhlYWRlciA9IFRSVUUpXG4jTG9hZCBpbiB0aGUgcG9rZSBkYXRhXG5wb2tlVHJlbmQ8LSByZWFkLmNzdih1cmwoXCJodHRwOi8vczMuYW1hem9uYXdzLmNvbS9hc3NldHMuZGF0YWNhbXAuY29tL2Jsb2dfYXNzZXRzL3Bva2VfdHJlbmRzX2F1Zy5jc3ZcIiksIGhlYWRlciA9IFRSVUUpXG5wb2tlIDwtbGlzdCh0cmVuZD1wb2tlVHJlbmQpXG5wb2tlLnRyZW5kcyA8LSBwb2tlJHRyZW5kJHBva2Vtb24uZ28uXG4jQnVpbGQgcG9vcm1hbiBOVCBmZWF0dXJlXG5wb29ybWFuLm50PC1kYXRhLmZyYW1lKG5pbnRlbmRvLmNsb3NlPW5pbnRlbmRvJFlKNzk3NC5ULkNsb3NlLFxuICAgICAgICAgICAgICAgICAgICAgICBwb2tlLnRyZW5kcz1wb2tlLnRyZW5kcyxcbiAgICAgICAgICAgICAgICAgICAgICAgd2Vlay4yMDE2PXNlcSgxOm5yb3cobmludGVuZG8pKSkiLCJzYW1wbGUiOiJjb3IocG9vcm1hbi5udCRuaW50ZW5kby5jbG9zZSxcbiAgICBwb29ybWFuLm50JHBva2UudHJlbmRzKSAifQ==

Make Charts

The final step! Let’s make two line charts from the poorman.nt data. Using ggplot pass in the data, specify the X and Y axes along with a group of 1 representing individual weeks. The next layer is the geometric line, geom_line , with some aesthetics. The next layer applies the Google Docs theme. In order to align the charts, another theme call is added to remove aspects of the Y axis. Finally, a title is added using ggtitle .

eyJsYW5ndWFnZSI6InIiLCJwcmVfZXhlcmNpc2VfY29kZSI6IiMgV2UgdXNlIGdncGxvdCBhbmQgZ2d0aGVtZXNcbmxpYnJhcnkoZ2dwbG90MilcbmxpYnJhcnkoZ2d0aGVtZXMpXG5saWJyYXJ5KHh0cylcbiMgTG9hZCBpbiBuaW50ZW5kbyBzdG9jayBkYXRhIFxubmludGVuZG8gPC0gcmVhZC5jc3YodXJsKFwiaHR0cDovL3MzLmFtYXpvbmF3cy5jb20vYXNzZXRzLmRhdGFjYW1wLmNvbS9ibG9nX2Fzc2V0cy9uaW50ZW5kb193ZWVrbHlfc3RvY2tfYXVnLmNzdlwiKSwgaGVhZGVyID0gVFJVRSwgZmlsZUVuY29kaW5nPVwiVVRGLTE2TEVcIilcbiNMb2FkIGluIHRoZSBwb2tlIGRhdGFcbnBva2VUcmVuZDwtIHJlYWQuY3N2KHVybChcImh0dHA6Ly9zMy5hbWF6b25hd3MuY29tL2Fzc2V0cy5kYXRhY2FtcC5jb20vYmxvZ19hc3NldHMvcG9rZV90cmVuZHNfYXVnLmNzdlwiKSwgaGVhZGVyID0gVFJVRSlcbnBva2UgPC1saXN0KHRyZW5kPXBva2VUcmVuZClcbnBva2UudHJlbmRzIDwtIHBva2UkdHJlbmQkcG9rZW1vbi5nby5cbiNQb29ybWFuIE5UIGZlYXR1cmVcbnBvb3JtYW4ubnQ8LWRhdGEuZnJhbWUobmludGVuZG8uY2xvc2U9bmludGVuZG8kWUo3OTc0LlQuQ2xvc2UsXG4gICAgICAgICAgICAgICAgICAgICAgIHBva2UudHJlbmRzPXBva2UudHJlbmRzLFxuICAgICAgICAgICAgICAgICAgICAgICB3ZWVrLjIwMTY9c2VxKDE6bnJvdyhuaW50ZW5kbykpKSIsInNhbXBsZSI6Im5pbnRlbmRvLnByaWNlPC1nZ3Bsb3QocG9vcm1hbi5udCwgYWVzKHg9d2Vlay4yMDE2LCB5PW5pbnRlbmRvLmNsb3NlLGdyb3VwID0gMSkpICsgZ2VvbV9saW5lKGNvbG9yPSdkYXJrcmVkJywgc2l6ZT0xKSsgdGhlbWVfZ2RvY3MoKSArIFxuICB0aGVtZShsZWdlbmQucG9zaXRpb249XCJub25lXCIsXG4gICAgICAgIGF4aXMudGl0bGUueT1lbGVtZW50X2JsYW5rKCksXG4gICAgICAgIGF4aXMudGV4dC55PWVsZW1lbnRfYmxhbmsoKSxcbiAgICAgICAgYXhpcy50aWNrcy55PWVsZW1lbnRfYmxhbmsoKSkgK1xuICBnZ3RpdGxlKFwiTmludGVuZG8gU3RvY2sgUHJpY2VcIilcbm5pbnRlbmRvLnByaWNlIn0=

Use ggplot to Make the Line Charts

The same process is repeated for the trends data. The Y axis is changed to poke.trends along with the line’s color and title.

eyJsYW5ndWFnZSI6InIiLCJwcmVfZXhlcmNpc2VfY29kZSI6ImxpYnJhcnkoZ2dwbG90MilcbmxpYnJhcnkoZ2d0aGVtZXMpXG5saWJyYXJ5KHh0cylcbm5pbnRlbmRvIDwtIHJlYWQuY3N2KHVybChcImh0dHA6Ly9zMy5hbWF6b25hd3MuY29tL2Fzc2V0cy5kYXRhY2FtcC5jb20vYmxvZ19hc3NldHMvbmludGVuZG9fd2Vla2x5X3N0b2NrX2F1Zy5jc3ZcIiksIGhlYWRlciA9IFRSVUUpXG5wb2tlVHJlbmQ8LSByZWFkLmNzdih1cmwoXCJodHRwOi8vczMuYW1hem9uYXdzLmNvbS9hc3NldHMuZGF0YWNhbXAuY29tL2Jsb2dfYXNzZXRzL3Bva2VfdHJlbmRzX2F1Zy5jc3ZcIiksIGhlYWRlciA9IFRSVUUpXG5wb2tlIDwtbGlzdCh0cmVuZD1wb2tlVHJlbmQpXG5wb2tlLnRyZW5kcyA8LSBwb2tlJHRyZW5kJHBva2Vtb24uZ28uXG5wb29ybWFuLm50PC1kYXRhLmZyYW1lKG5pbnRlbmRvLmNsb3NlPW5pbnRlbmRvJFlKNzk3NC5ULkNsb3NlLFxuICAgICAgICAgICAgICAgICAgICAgICBwb2tlLnRyZW5kcz1wb2tlLnRyZW5kcyxcbiAgICAgICAgICAgICAgICAgICAgICAgd2Vlay4yMDE2PXNlcSgxOm5yb3cobmludGVuZG8pKSkiLCJzYW1wbGUiOiJwb2tlbW9uLnRyZW5kPC1nZ3Bsb3QocG9vcm1hbi5udCwgYWVzKHg9d2Vlay4yMDE2LCB5PXBva2UudHJlbmRzLGdyb3VwID0gMSkpICsgZ2VvbV9saW5lKGNvbG9yPSdkYXJrYmx1ZScsIHNpemU9MSkrIHRoZW1lX2dkb2NzKCkgKyBcbiAgdGhlbWUobGVnZW5kLnBvc2l0aW9uPVwibm9uZVwiLFxuICAgICAgICBheGlzLnRpdGxlLnk9ZWxlbWVudF9ibGFuaygpLFxuICAgICAgICBheGlzLnRleHQueT1lbGVtZW50X2JsYW5rKCksXG4gICAgICAgIGF4aXMudGlja3MueT1lbGVtZW50X2JsYW5rKCkpICtcbiAgZ2d0aXRsZShcIlBva2Vtb24gR08gTmV3cyBUcmVuZHNcIilcbnBva2Vtb24udHJlbmQifQ==

Stack the Visuals

Lastly, pass each ggplot object into grid.arrange to stack the visuals. This makes it easier for your audience to consume the figure below.

eyJsYW5ndWFnZSI6InIiLCJwcmVfZXhlcmNpc2VfY29kZSI6ImxpYnJhcnkoZ3JpZEV4dHJhKVxubGlicmFyeShnZ3Bsb3QyKVxubGlicmFyeShnZ3RoZW1lcylcbmxpYnJhcnkoeHRzKVxubmludGVuZG8gPC0gcmVhZC5jc3YodXJsKFwiaHR0cDovL3MzLmFtYXpvbmF3cy5jb20vYXNzZXRzLmRhdGFjYW1wLmNvbS9ibG9nX2Fzc2V0cy9uaW50ZW5kb193ZWVrbHlfc3RvY2tfYXVnLmNzdlwiKSwgaGVhZGVyID0gVFJVRSlcbnBva2VUcmVuZDwtIHJlYWQuY3N2KHVybChcImh0dHA6Ly9zMy5hbWF6b25hd3MuY29tL2Fzc2V0cy5kYXRhY2FtcC5jb20vYmxvZ19hc3NldHMvcG9rZV90cmVuZHNfYXVnLmNzdlwiKSwgaGVhZGVyID0gVFJVRSlcbnBva2UgPC1saXN0KHRyZW5kPXBva2VUcmVuZClcbnBva2UudHJlbmRzIDwtIHBva2UkdHJlbmQkcG9rZW1vbi5nby5cbnBvb3JtYW4ubnQ8LWRhdGEuZnJhbWUobmludGVuZG8uY2xvc2U9bmludGVuZG8kWUo3OTc0LlQuQ2xvc2UsXG4gICAgICAgICAgICAgICAgICAgICAgIHBva2UudHJlbmRzPXBva2UudHJlbmRzLFxuICAgICAgICAgICAgICAgICAgICAgICB3ZWVrLjIwMTY9c2VxKDE6bnJvdyhuaW50ZW5kbykpKVxucG9rZW1vbi50cmVuZDwtZ2dwbG90KHBvb3JtYW4ubnQsIGFlcyh4PXdlZWsuMjAxNiwgeT1wb2tlLnRyZW5kcyxncm91cCA9IDEpKSArIGdlb21fbGluZShjb2xvcj0nZGFya2JsdWUnLCBzaXplPTEpKyB0aGVtZV9nZG9jcygpICsgXG4gIHRoZW1lKGxlZ2VuZC5wb3NpdGlvbj1cIm5vbmVcIixcbiAgICAgICAgYXhpcy50aXRsZS55PWVsZW1lbnRfYmxhbmsoKSxcbiAgICAgICAgYXhpcy50ZXh0Lnk9ZWxlbWVudF9ibGFuaygpLFxuICAgICAgICBheGlzLnRpY2tzLnk9ZWxlbWVudF9ibGFuaygpKSArXG4gIGdndGl0bGUoXCJQb2tlbW9uIEdPIE5ld3MgVHJlbmRzXCIpXG5uaW50ZW5kby5wcmljZTwtZ2dwbG90KHBvb3JtYW4ubnQsIGFlcyh4PXdlZWsuMjAxNiwgeT1uaW50ZW5kby5jbG9zZSxncm91cCA9IDEpKSArIGdlb21fbGluZShjb2xvcj0nZGFya3JlZCcsIHNpemU9MSkrIHRoZW1lX2dkb2NzKCkgKyBcbiAgdGhlbWUobGVnZW5kLnBvc2l0aW9uPVwibm9uZVwiLFxuICAgICAgICBheGlzLnRpdGxlLnk9ZWxlbWVudF9ibGFuaygpLFxuICAgICAgICBheGlzLnRleHQueT1lbGVtZW50X2JsYW5rKCksXG4gICAgICAgIGF4aXMudGlja3MueT1lbGVtZW50X2JsYW5rKCkpICtcbiAgZ2d0aXRsZShcIk5pbnRlbmRvIFN0b2NrIFByaWNlXCIpIiwic2FtcGxlIjoiZ3JpZC5hcnJhbmdlKG5pbnRlbmRvLnByaWNlLCBwb2tlbW9uLnRyZW5kKSAifQ==

It’s clear that the data is correlated. Admittedly, you can even see the news peak happening one week ahead of the stock price peak. Still, market timing based on this type of analysis is fraught with challenges. The periodicity of this visual (never got to see Bloomberg NT for real) is in weeks but the market moves more dynamically. Additionally, the lines do not behave the same after the peak. It is subtle but the lines diverge. So to trade on this information, you time the market peak of the news trend line.