Congressional Partisanship: A Visualization
Since I was in high school, I would hear my friends in Model UN or those that were really interested in politics talking about how “The Republicans did this” or “The Democrats did that”. With what seems to be one of the most politicized elections in history about a year behind us, I wanted to answer the question “Is Congress more politically polarized than ever before?”
I set out to do that by some simple visualizations of data that is freely available from the Senate and House websites (both of whom have party division pages). The code for my analysis is available in its entirety on my GitHub page.
The data was scraped from the pages using two different methods. The Senate data had to be scraped with urllib andBeautifulSoup due to the structure of the data on the page. The House data was in a table which allowed me to use the read_html() method of the pandas library. (The cleaning is described in detail in the Jupyter Notebook for the project.)
I explored the Senate data a bit more than the House data (although based on the graphs, the conclusions look the same). The first thing I plotted with the Senate data was the percentage of the seats held by the “Majority Party” versus the “Minority Party”. The “Majority Party” is colored in kelly green while the “Minority Party” is colored in crimson.
An obvious feature of this graph is that the “Majority Party” is always a greater percentage of the Senate than the “Minority Party”. It doesn’t tell us anything about which parties were in power at the time, but it does give some insight. It appears that the percentage difference of the two parties decreases over time. If we define a new feature called Partisan Difference and say it’s the difference between the percent of the majority party and the percent of the minority party, we can more directly measure that perceived decrease. Defining Partisan Difference in this way guarantees a positive value, as the majority has to be larger or equal to the minority. (The color of this chart is the difference of the RGB values of kelly green and crimson.)
As anticipated based on the last plot, the Partisan Difference does decrease over time. This leads to the conclusion that, at least from a party perspective, Congress is less politically polarized than in the past. However, there is the another way of interpreting the question: “Have the policies and the ideologies of the parties become more polar?” This question cannot be answered by this data, but is something that can be explored at a later time.
Visualizing the Senate by Party
So we’ve seen that the Partisan Difference decreases over time, but who makes up that “Majority” and “Minority” party? After cleaning the data a bit further and making it more explicit, I plotted it using Python’s seaborn library.
A big problem with this plot is that, due to the large number of parties, the colors that differentiate the parties are too close together to easily identify them. Using matplotlib to plot each party with a different color and a different marker yields this plot:
That’s a lot better. It’s not the prettiest chart, but it conveys the information in a readable format which is the primary goal. One thing we can glean from these charts is that the two main parties in American politics have been the Republicans and Democrats as far back as the 1850s. This means that the ten other parties represented on the graph are only major players in the first 60 or so years of the country’s 241 year history.
Visualizing the House by Party
Let’s do the same thing for the House of Representatives. Using seaborn we get a graph that is similar to the Senate one above.
The seaborn graph runs into the same problems as before. With limited colors that are visually distinct, it’s hard to get a lot of information quickly by reading this graph. To fix it, we’ll use matplotlib with different markers for each party as we did above.
This is much better. Again, it’s not the most beautiful graph but it conveys information quickly and accurately. Just as in the Senate graph, we see that the Republicans and Democrats were the dominant parties beginning in the 1850s.
Where to go from here
These two visualizations are nice to have but they would be even better in an interactive format. The interactive format would allow for easy exploration of the data and make the presentation much more uniform. Another aspect that is missing from the individual graphs is the inability to look at both the House and Senate simultaneously. To address these issues, I made a visualization of the data using Tableau Public.
Final Word: I standardized the names of the parties between the two data sources to make the interactive chart much more interesting to explore. The most notable (read as annoying) difference is the House data using Democrats and Republicans, while the Senate data uses Democrat and Republican. I only made the modification to the data for the Tableau plot because I wanted to leave the data as “original” as possible for this exercise in visualization.
Have any suggestions? I’d love to hear what you think and anyway I could improve my analysis or visualizations. Please leave a comment!
Remember, I included the data I used and full code in my GitHub Repo for the project. The cleaned Senate data can be downloaded from here and the cleaned House data is available here, both in .csv format. If you enjoyed this analysis please share it with your friends and feel free to check out my other projects at decotiismauro.ml.
