Exploring the limitations of the word cloud as a data visualization.

Author: Shelby Temple; Made with Tableau

Word clouds have recently become a staple of data visualization. They are especially popular when analyzing text. According to Google Trends, it seems that the rise in popularity started around 2009 with search term interest currently just under bar charts.

3 Month Moving Average; Source: Google Trends, Author: Shelby Temple, Made with Tableau

Unlike the bar chart, the word cloud has noticeable limitations as a data visualization. Apparently, I am not alone in thinking this. The data viz catalogue (datavizcatalogue.com) mentions that word clouds are not great for analytical accuracy. Daniel McNichol, in a post published in Towards Data Science, called word clouds the pie chart of text data.

My main problem is that this visualization is usually uninteresting and provides little insight. You typically get a mixture of obvious words and common words.

So you’re telling me, the word “Harry” appeared a lot in Harry Potter. Shocking!

After it really clicked in my head that word clouds are lame, I started to ask myself more questions:

Why is it that whenever there is a new trendy Twitter topic or text to ‘analyze’, people feel the need to post word clouds all over the internet?

What exactly are the limitations of the word cloud?

Is there ever a right time to use word clouds?

The Story Behind Word Clouds

I found the origin story of word clouds in a book called, Introduction to Text Visualization so credits go to Nan Cao and Weiwei Cui.

In this book, they attribute word clouds to a 1976 visualization created by Stanley Milgram, a social psychologist. He asked people to name landmarks in Paris. Then he drew a map with the names of the landmarks as text making the font size bigger if the landmark got more responses.

However, this is not exactly how the word cloud became popular. For that, we can thank the photo sharing website, Flickr. Around 2006, they implemented the word cloud visual as a way to explore their website through popular tags. They called their implementation of word clouds, tag clouds. The UX community embraced this design and it began to appear all over the internet. Due to this, you might come across people still calling word clouds, tag clouds.

Flicker has hilariously, apologized for starting the word cloud craze in the UX community.

3 Month Moving Average; Source: Google Trends; Author: Shelby Temple; Made with Tableau

So what happened around 2010, to make the term word cloud trend up to search interest levels similar to bar charts?

I don’t know of a single person or visualization to attribute this to — but it seems like that the current popularity of the word cloud is related to the rising popularity in mining text for insight.

A primary application is compiling a bunch of reviews and mining them for insight. For example, a website like Amazon could take the reviews of a product, along with the rating (usually a scale of 5/5 stars) and figure out patterns on the product’s strength and weaknesses. Perhaps when filtering on low reviews the phrase, “Doesn’t fit” appears frequently.

For a company with lots of locations, common complaints can be addressed. If one Taco Bell location is constantly getting bad reviews about management… maybe management is bad there?

Unfortunately, word clouds rarely answer these questions or solve for these applications but people still make them anyway!

Aside from commercial applications, anyone who has a basic understanding of R or Python can pull tweets and make a word cloud. The fact that they are now easier to make obviously contributes to their popularity.

In Summary:

In 1976, Stanley Milgram made the first word cloud-ish visualization as a map of landmarks in Paris Flickr started the internet word cloud wild fire in 2006 —they said they are sorry Around 2010, data science and text mining enthusiasts picked up the word cloud torch and have made them more popular than ever!

Exploring the Limitations of the Word Cloud

Low-Information:

When is the last time you made a decision or had an important takeaway based on a word cloud? For me, the answer is never and I imagine most people are on that same ship. The word cloud provides a low-level of information, usually just the frequency a word appeared. A word appearing frequently usually doesn’t mean much without additional context. Who said it, why did they say it, when did they say it, and to whom did they say it to?

I’ve seen usage where color is used to represent a second level of information. Maybe the size of the word is frequency, then the color is the category of the word. I have mixed feelings toward this, because it goes against what people are used to seeing in word clouds. In a way, it takes away one of the only advantages of using a word cloud — that people are extremely familiar with them as a method to visualize word frequency.

Context in terms of sentence structure:

Word clouds typically only look at one word at a time. This is flawed in that it can create a misleading end product. What if a bunch of Tweets to your company twitter said,”Not Cool!” The word cloud would separate out “not” and “cool.” This could mislead people into thinking their company Twitter is cool when it is actually not cool.

Ranking:

Word clouds aren’t even the best visualization for what they are intended to do. Show you the most frequent/popular words. Sure, sometimes you can tell which is the most popular word — but then how about the second, third, and tenth most popular word? Not so easy. A sorted bar chart is much more functional in achieving this.

What is the tenth most popular word?

Author: Shelby Temple; Made with Tableau

How about now?

Author: Shelby Temple; Made with Tableau

Other Limitations:

Some other issues include that the word cloud emphasizes long words over short words. The placement and ordering of the words can be disorienting as they usually just randomly appear in different spots. Also, it is often unclear if common/boring words such as — the, as, is, and or — were filtered out. Finally, if you use a scale other than popularity/frequency people might be caught off guard.

Why Are Word Clouds Still Popular?

As slightly touched on before, text mining is just another surfer on the data science wave. As data science, big data, and A.I. trend up in popularity — natural language processing (NLP) and text mining will trend up as well.

In a weird way, I am now starting to connect the word cloud as the, hello world of NLP. Much like how most coder’s first program they ever made was a program that simply prints, “hello world!” — mining tweets from Twitter and visualizing as a word cloud has pretty much become the intro assignment for text mining. One subtle difference is that a word cloud is much more fun to look at than text saying, “hello world.” A second difference is that it is a little bit more involved to process unstructured text data into a word cloud. Due to this, I think people are much more inclined to share their first word cloud. It serves as some sort of weird flex in data science.

*Proudly posts first word cloud on the internet* Everyone else: “Yup, I remember my first word cloud.“

Final Thoughts

Is there ever a good scenario to create a word cloud?

I think so. Even after pointing out the limitations, I think word clouds have a few things going for them.

They are often colorful and pretty to look at Like a pie chart, almost no description is needed for someone to understand — it’s a schema people are familiar with It makes for a great intro to text mining assignment There aren’t many other visualizations that are easy to make and specialized for unstructured text analysis

But, how do we solve for word clouds being overused? I think there is a great opportunity for visualization and NLP experts to come up with new packages and visualization techniques for text mining. I think if someone built a new Python or R package that easily digests text structures and visualizes them in exciting ways people would obviously take advantage of it.

Until then, we all need to brace ourselves every time there is a new exciting Twitter topic.

Author: Unknown

Need inspiration? Here were some better text visualizations I’ve seen. Notice the lack of word clouds:

Friends Through the Transcripts — Lorna Eden

An Analysis of the Beatles — Adam McCann

Spells of Harry Potter — Skylar Johnson

The Words of Christmas Movies — Zak Geis

Words Used in Sacred Texts — Ken Flerlage

South Park Season One Words Analysis — Rody Zakovich