Sentiment Analysis of US Presidential Inaugural Addresses

In this blog post, I will take a look at how we can use Azure Cognitive Services Text Analytics API to analyze speeches in terms of their sentiment. This is continuation of my previous explorations of the Text Analytics API, I made an e-book sentiment analyzer, which I encourage you to read to get more details on how to use the Text Analytics API. I will be analyzing all US Presidential inaugural addresses from George Washington up through Donald J. Trump. Keep reading, the results may surprise you.

I am not trying to make any political statements with this analysis. It is simply an example of using Cognitive Services to look at a collection of speeches.  

The source of the data used in this analysis can be found on the American Presidency Project website. I will illustrate with some code how one can extract the actual text of each inaugural address. The source code is available on GitHub, and the most important pieces will be reproduced in the blog post.

TL;DR - Just Give Me The Results

I know that some readers will just be here for the results and will not care much about the implementation, so I will spare them the agony of reading through the entire post. Here are the results:

Summary statistics:

The 10 most positive (in terms of sentiment) inaugural addresses:

The 10 most negative (in terms of sentiment) inaugural addresses:

One observation is that Donald J. Trumps recent "American carnage" inaugural address is not on either top 10 list. It has a score of 0.72, which is a bit below average, but not much so. This is also in agreement with other analyses of that address, that have found it to be relatively positive. My analysis shows it to be a bit below average for inaugural addresses, but in the broader context of speeches, 0.72 is still relatively positive. You can find it on the complete list of scores at the end of this blog. Barack Obama's first inaugural address and John F. Kennedy's inaugural address, are both in the top 10 of most negative sentiment, which may surprise some. However, if you read those addresses, I think you may agree that while they may be great speeches (this blogger does not presume to have an opinion about that), they also talk about some serious problems and challenges, which of course will influence the language. It just shows that a speech can be inspirational (or aspirational) without necessarily having a positive sentiment. Obama's second address was much more positive than the first.

Another observation is that there are more modern presidents on the negative top 10 list, which is also visible if we plot the sentiment as function of year:

It would appear that there is a slight downward trend in sentiment, which may just be a reflection of language changes over the years. We would need some more data from other types of speeches to dig into that. I will leave you to figure out when the addresses will start to be real downers if the current trend continues.

I will also leave the political analysis to others. The purpose of this is just to illustrate that one can get interesting trends and information with relatively little work using Cognitive Services.

Implementation - The Gory Details

So if you would like to do this yourself, it is actually relatively easy. I have written a utility C# library that use can use to analyze speeches or other text documents. The library is called "SpielInsights", since calling it SpeechInsights might confuse it with something that analysis spoken words (as in audio). The library will give you the sentiment of each paragraph of text and also the key phrases in each paragraph. It will also provide you with summary information for the entire document. In this blog post, I am really just reporting the overall sentiment of the document. This overall sentiment is calculated as a weighted average (by the relative length of each paragraph in characters) of the sentiment of all paragraphs in the text. You can find the source code for this utility library here.

In order to use the library, you will need to load your text into a "Spiel" object:

     public class Spiel
    {

        public Spiel()
        {
            Paragraphs = new List<string>();
        }

        public string SourceURI { get; set; }

        public string Speaker { get; set; }

        public string Category { get; set; }

        public DateTime Date { get; set; }

        public List<string> Paragraphs { get; set; }
    }

This contains a bit of information about the speaker and the time when the speech was given and a list of text paragraphs. This object can be passed to the AnalyzeSpiel function:

        static public SpielAnalytics AnalyzeSpiel(Spiel spiel, string apiKey) 
       { 
          ... 
       }

Which will make as many calls as needed (based on length of text) to the Text Analytics API and then return some analytics:

     public class SpielParagraphAnalytics
    {
        public SpielParagraphAnalytics()
        {
            KeyPhrases = new HashSet<string>();
            Sentiment = 0;
            Words = 0;
            Characters = 0;
        }

        public long Words { get; set; }

        public long Characters { get; set; }

        public double Sentiment { get; set; }

        public HashSet<string> KeyPhrases { get; set; }
    }

    public class SpielAnalytics
    {
        public SpielAnalytics()
        {
            SummaryAnalytics = new SpielParagraphAnalytics();
            ParaGraphAnalytics = new List<SpielParagraphAnalytics>();
        }

        public SpielParagraphAnalytics SummaryAnalytics { get; set; }
        public List<SpielParagraphAnalytics> ParaGraphAnalytics { get; set; }
    }

Now that we have the basic structures and functions laid out, we need the data. I will just spend a bit of time here to show how I did the retrieval and cleaning of the data. As mentioned, the inaugural addresses can be found on the American Presidency Project web site, however, there is no database of the raw text that I could find, so I have made a list of each of the speeches with the links to their specific pages. You can find that list here. An excerpt from the list looks like this:

 George Washington;https://www.presidency.ucsb.edu/ws/index.php?pid=25800;April 30, 1789
George Washington;https://www.presidency.ucsb.edu/ws/index.php?pid=25801;March 4, 1793
John Adams;https://www.presidency.ucsb.edu/ws/index.php?pid=25802;March 4, 1797
Thomas Jefferson;https://www.presidency.ucsb.edu/ws/index.php?pid=25803;March 4, 1801
Thomas Jefferson;https://www.presidency.ucsb.edu/ws/index.php?pid=25804;March 4, 1805
James Madison;https://www.presidency.ucsb.edu/ws/index.php?pid=25805;March 4, 1809
James Madison;https://www.presidency.ucsb.edu/ws/index.php?pid=25806;March 4, 1813
James Monroe;https://www.presidency.ucsb.edu/ws/index.php?pid=25807;March 4, 1817
James Monroe;https://www.presidency.ucsb.edu/ws/index.php?pid=25808;March 4, 1821

    ...

I have then written a small C# (.NET CORE) routine that runs through this list, finds the part of the HTML documents where the speech text is and puts each paragraph into the "Spiel" structure before sending it to the analysis routine. You can find the source code of this routine here, and since it is pretty short, I have reproduced it here:

 using System;
using System.IO;
using System.Net.Http;
using System.Threading.Tasks;
using HtmlAgilityPack;
using SpielInsights;
using System.Collections.Generic;
using System.Text;

namespace Inaugurals
{
    class Program
    {
        static void Main(string[] args)
        {
            string inputList = args[0];
            string apiKey = args[1];
            string outputFileName = args[2];

            var client = new HttpClient();

            //Output file
            System.IO.StreamWriter outputFile = new System.IO.StreamWriter(outputFileName);

            Task.Run(async () =>
            {
                using (StreamReader reader = new StreamReader(inputList))
                {
                    string line;
                    while ((line = reader.ReadLine()) != null)
                    {
                        Spiel spiel = new Spiel();

                        string[] components = line.Split(";");

                        spiel.Speaker = components[0];
                        spiel.SourceURI = components[1];
                        spiel.Date = System.Convert.ToDateTime(components[2]);

                        Console.WriteLine("Processing Inaugural: " + components[1]);
                        var response = await client.GetAsync(components[1]);
                        var content = await response.Content.ReadAsStringAsync();

                        HtmlDocument htmlDocument = new HtmlDocument();
                        htmlDocument.LoadHtml(content);

                        foreach (HtmlNode node in 
                                 htmlDocument.DocumentNode.SelectNodes("//span[@class='displaytext']"))
                        {
                            HtmlDocument innerHtmlDocument = new HtmlDocument();
                            innerHtmlDocument.LoadHtml(node.InnerHtml);

                            foreach (HtmlNode pnode in innerHtmlDocument.DocumentNode.SelectNodes("//text()"))
                            {
                                string paragraphText = pnode.InnerText.Trim();
                                spiel.Paragraphs.Add(paragraphText);                  
                            }

                            SpielAnalytics analytics = SpielInsights.SpielInsights.AnalyzeSpiel(spiel, apiKey);


                            //Build semicolon separated output records
                            StringBuilder osb = new StringBuilder();
                            osb.Append(components[0] + ";"); //Speaker
                            osb.Append(components[2] + ";"); //Date
                            osb.Append(analytics.SummaryAnalytics.Sentiment);

                            outputFile.WriteLine(osb.ToString());
                            Console.WriteLine(osb.ToString());
                        }
                    }
                }
            }).GetAwaiter().GetResult();

            outputFile.Close();
        }
    }
}

This routine simply loops through all the lines in the list of inaugural addresses, uses an HttpClient to receive each one and then cuts the right <span> in the HTML document that contains the actual speech text and loops through each section of it. The sections/paragraphs are added to the "Spiel" and then we retrieve analytics.

As you can see there is a lot more information in the analytics structures than I have presented here, but I will leave it to you to play more with it and see what interesting stuff you find. I may find some time to post it online in a way that can be searched and browsed. If I do, I will write about it on my blog, so please subscribe.

That's it. Have fun analyzing speeches or playing with the data presented here. Please rate the blog and post any comments you may have.

Complete Results

For completeness, I am adding all the sentiment scores for all inaugural addresses in chronological order.