Last Friday, my roommate Greg had the great idea of doing sentiment analysis (the process of determining/categorizing emotions, attitudes, and opinions in text) on my diary. I've kept a personal diary since October 12, 2012, and have written almost every day since- as of the end of September 2017, my diary's word count is a little over 500,000. That makes for a lot of interesting personal data, so I spent most of my weekend and some of my week building a python program that uses NLTK and TextBlob to do some analysis on my diary. This post shares some of the code that I wrote and some of the more interesting insights that I got.

I spent a while digging around for the right library, and ended up using TextBlob (which is built on top of NLTK) because I didn't want to spend the time to train a classifier. TextBlob's classifier is trained on a big dataset of movie reviews from nltk.corpus, which was good enough for me, although I may revisit this in the future.

What is the average sentiment of my diary?

The first thing I wanted to do was to just read in the diary and get the average sentiment. The sentiment property in TextBlob returns a tuple of the form (polarity, subjectivity). Polarity is a float within the range [-1.0, 1.0], where -1.0 is very negative and 1.0 is very positive. Subjectivity is a float within the range [0.0, 1.0], where 0.0 is very objective and 1.0 is very subjective.

My diary is unfortunately written in an encrypted word doc, so I had to copy and paste it :-( to another file. Once I had the file opened though, making a TextBlob object out of a string is ridiculously easy.