Since I was in high school, I would hear my friends in Model UN or those that were really interested in politics talking about how “The Republicans did this” or “The Democrats did that”. With what seems to be one of the most politicized elections in history about a year behind us, I wanted to answer the question “Is Congress more politically polarized than ever before?”

I set out to do that by some simple visualizations of data that is freely available from the Senate and House websites (both of whom have party division pages). The code for my analysis is available in its entirety on my GitHub page.

The data was scraped from the pages using two different methods. The Senate data had to be scraped with urllib and BeautifulSoup due to the structure of the data on the page. The House data was in a table which allowed me to use the read_html() method of the pandas library. (The cleaning is described in detail in the Jupyter Notebook for the project.)

I explored the Senate data a bit more than the House data (although based on the graphs, the conclusions look the same). The first thing I plotted with the Senate data was the percentage of the seats held by the “Majority Party” versus the “Minority Party”. The “Majority Party” is colored in kelly green while the “Minority Party” is colored in crimson.

An obvious feature of this graph is that the “Majority Party” is always a greater percentage of the Senate than the “Minority Party”. It doesn’t tell us anything about which parties were in power at the time, but it does give some insight. It appears that the percentage difference of the two parties decreases over time. If we define a new feature called Partisan Difference and say it’s the difference between the percent of the majority party and the percent of the minority party, we can more directly measure that perceived decrease. Defining Partisan Difference in this way guarantees a positive value, as the majority has to be larger or equal to the minority. (The color of this chart is the difference of the RGB values of kelly green and crimson.)

As anticipated based on the last plot, the Partisan Difference does decrease over time. This leads to the conclusion that, at least from a party perspective, Congress is less politically polarized than in the past. However, there is the another way of interpreting the question: “Have the policies and the ideologies of the parties become more polar?” This question cannot be answered by this data, but is something that can be explored at a later time.