saisenberg

Comparing Song, Album, and Artist Metrics with Spotify’s API

2019-05-18T00:00:00+00:00

Click here to go straight to the interactive dashboard!

In a continuation of my recent music-based projects, I decided to dive into the Spotify API to collect and compare data of different songs, albums, and artists. Using the Dash visualization tool, I put together an interactive dashboard that lets users compare nearly 700 unique artists and over 110,000 songs on a variety of metrics. In addition, the application allows for the quick and efficient downloading of any artist’s raw data – regardless of whether that artist is among the initial seven hundred.

Each of the dashboard’s seven metrics ranges from 0 to 1. As found within Spotify’s API documentation, the metrics consist of the following. More information about each metric can be found at the preceding link.

Acousticness: Whether the track is acoustic
Danceability: How suitable a track is for dancing, based on a combination of musical elements including tempo, rhythm stability, beat strength, and overall regularity
Energy: A measure of intensity and activity, based on perceptual features including dynamic range, perceived loudness, timbre, onset rate, and general entropy
Instrumentalness: Whether the track contains no vocals
Liveness: Whether the track was performed live
Speechiness: Detects the presence of spoken words in a track
Valence: A measure of musical positivity conveyed by a track

One note about metric aggregation (i.e., combining song metrics into album or artist metrics): to mitigate the potentially undue influence of an interlude, album introduction, or very short track, every aggregated value has been weighted based on song length. Long songs, therefore, are given slightly more importance than short ones in the context of a full album or artist discography. In addition, albums whose metadata is missing a release month and day are assumed to have been released on June 15th of the labeled year.

Hope you guys enjoy! All data and dashboard code can be found on my GitHub, and the dashboard itself can be found here.

Rap Network Analysis: Which Rappers Work Together?

2019-04-09T00:00:00+00:00

I’ve written my fair share of rap-related posts on this website. Today, though, I’ll be focusing on song titles, not lyrics. Using the geniusr R library, I scraped thousands of song titles from Genius and collected a list of all rapper features.

The A$AP Rocky song 1Train, for instance, features six different rappers: Kendrick Lamar, Action Bronson, Joey Bada$$, Yelawolf, Danny Brown, and Big K.R.I.T. Six different items, then – one for each featured artist – are added to the features list, with each one noting A$AP Rocky as the original song artist. (Imagine this process repeated nearly 70,000 times.)

Now, using the Python networkx package, we can look at an entire mapping of which rappers work most closely together.

First, though, some important notes about the project:

The relative locations of every artist are approximate and should not be taken literally.
An artist must have at least 25 scraped song titles for any connections involving that artist to appear.
The resulting map makes no distinction between rappers who featured on the song, and rappers “credited” for the song. (That is, J. Cole appearing on a 21 Savage song is treated the same as 21 Savage featuring on a J. Cole song.)
No connection is made between two artists who featured on the same song, if neither were credited for the song.
Songs must be credited to only one artist on Genius. For instance, the song Sunflower is credited to the artist “Post Malone & Swae Lee”, rather than just one of the two artists.

Let’s take a look at the resulting map. Because the image is so large, you’ll likely have to right-click the image and view it in a new tab.

The network map does an excellent job of visualizing different “groups” of rappers who most commonly feature on each other’s songs. For example, the members of Odd Future, Pro Era, and the Wu-Tang Clan, among other groups or collectives, appear to have their own distinct respective “sections” of the map.

We can even see different styles – or even eras – of rap appear in different areas on the network. Much of the right-hand section of the map is dedicated to rap from the 1990s or early 2000s, whereas many “SoundCloud rappers” appear farther towards the left. In addition, nearly all the most “mainstream” rappers appear towards the center of the map, which makes sense; artists like DJ Khaled, Rick Ross, and Lil Wayne, by virtue of their popularity and longevity, have had the chance to work with a particularly widespread set of rappers. Also, the relative sizes of the red dots represent the relative “between centralities” of each rapper. The more often a certain rapper appears along the shortest “path” connecting two other artists, the larger their between centrality (and corresponding red point) will be. It’s evident that, along with the trio of rappers mentioned above, Juicy J, Snoop Dogg, and Busta Rhymes all have relatively large between centralities.

All data and code are posted on my GitHub. Hope you enjoyed!

These Baseball Players Do Not Exist

2019-04-04T00:00:00+00:00

Most of my projects thus far have been focused mainly on prediction - especially involving professional sports. Lately, though, I’ve tried to place a heavier emphasis on content generation. The most notable project in this category to date has been the SongwriterBot, which uses Markov chains to randomly generate its own lyrics. This post, though, will take more of a deep learning approach to content generation, and rather than generating sequences of words, we’ll be generating images of non-existent – but (hopefully) realistic! – baseball players.

Up front, I should note that I learned the majority of code for this project from an excellent guide on Medium. As such, I won’t be posting my code on GitHub, but would instead recommend that everyone interested in replicating this sort of project check out the above link.

In order to generate pictures, we’ll be using a deep learning architecture called Generative Adversarial Networks, or GANs. Popularized by a 2014 Ian Goodfellow paper, GANs are comprised of a pair of neural networks which “compete” against one another. The first model, a generator, creates series of fake images in attempts to confuse the discriminator model into being unable to distinguish real images from artificially generated ones. Data scientists have used this architecture to create some incredible-looking images – so incredible, in fact, that even humans can’t always tell the difference between real and fake pictures. My artificially created baseball players probably won’t reach quite that level of intricacy, but we’ll hopefully be able to create some realistic images.

Before we generate anything fake, let’s first look through our dataset of real images. I wrote a small Python scraper to iterate through individual player pages on baseball-reference, saving each player image to my desktop as we go. For the sake of time and convenience, I’ve only collected images of players who made their major league debuts during or after the 2000 season. Because most images involve a relatively similar “setup” (i.e. the ballplayer smiling, directly facing the camera, and wearing his cap and uniform), our generator should theoretically be able to “learn” and recreate this setup relatively quickly. Whether it’ll be able to trick the discriminator (and, in turn, humans) into believing its generations… that’s another story.

It’s important to note that the generator – at least how I’ve set it up – expects every image in our dataset to be equally sized. After scraping each image, then, I use the PIL library to reshape every image into a standard size. For reference, here are a few examples of the images our model hopes to replicate; you’ll see that they’re all equally sized and adhere to the setup described above. Feel free to reach out if you’re interested in the scraper code or the pictures themselves.

Our generator will accept a series of a hundred random numbers as an input, and will use a series of convolutional layers (along with upsampling, batch normalization, and dropout at each layer) to generate an array of size 64x64x3 as an output. Each of our output images will be 64 pixels wide and 64 pixels high, and, because we’re working with colorful images (in an RGB scale), we use 3 as our third dimension. Every number in our resulting array corresponds with an RGB value, which can range between zero and 255. Note that if we were generating our images in grayscale, our output would be in a 64x64x1 array, rather than 64x64x3.

The input to our discriminator model is a 64x64x3 array, which represents an entire player image (which can be fake or real). The components of the image go through another series of convolutional layers, but the output of the discriminator is only one number, ranging between zero and one. The closer the output is to one, the more confident the discriminator is that it’s identified a real image, rather than an image created by our generator.

Once we set a certain number of epochs and a batch size, we can keep training our generator to – at least in theory – create increasingly realistic images. During every epoch, we’ll train our discriminator on a number of fake images (with this number being equal to our batch size), and an identical number of real images, which we respectively label as fake and real. Then, we generate a new set of fake images, but try to trick the discriminator into mislabeling our generated images as real. After every hundred epochs, we’ll save a set of fake images to the desktop so that we can observe the generator’s learning progress over time.

Let’s see how our generated images change over the first thousand epochs, starting with the very first set:

After epoch #1, our generator essentially knows nothing at all – its output is entirely static. By the hundredth epoch, the model has already learned enough to create a very fuzzy set of head-shaped objects, which become progressively more well-defined as we iterate through the epochs. By epoch #1000, the images are still fuzzy, but the generator has clearly learned much more about what “constitutes” a face.

Let’s see how the generator improves through the twelve thousandth epoch. We’ll keep observing changes in hundred-epoch intervals, but we’ll move a bit faster now:

It’s a bit harder to follow the generator progress through so many images, but we can see the model creating more and more detailed pictures. Actual facial features, facial hair, and even ballcap logos begin to appear in our generated images, and the images have become sharper and more well-defined.

Finally, we’ll see how the generated images change up to the sixteen thousandth epoch (which is as long as I’ve run the model so far):

There’s clearly still more work to be done, but we’ve generated images that legitimately resemble real baseball players! While these pictures might not fool anyone as they are (I should mention that by the 16,000th epoch, the discriminator was still able to distinguish real from fake), re-scaling the pictures to their original aspect ratios might make the task slightly more challenging. Just for fun, I’ve compiled some of the best-looking generations below:

By continuing to fine-tune the model’s parameters, as well as running the model for more epochs, I hope to eventually create generated images that are truly indistinguishable from real ones. Parameters to tune include learning rate, number of convolutional layers, and number of nodes at each layer, among many others. Again, many thanks to the AI Insider, whose article was instrumental in preparing this project. Hope you found it interesting!

Lyrics, Pt. 3: Rap Song Clustering with Doc2Vec

2019-02-27T00:00:00+00:00

In the first installment of my lyrics project, we scraped thousands of song lyrics from Genius, and created a set of classifiers which identify song genres from their lyrical content. Part two of the project introduced the world to SongwriterBot, a fully functional web application that uses Markov chaining to randomly generate entire sets of song lyrics. (Special thanks to r/hiphopheads for the love!) Now, in the final installment of the Great Lyrics Trilogy, I’ll once again focus on the contents of the lyrics themselves to determine artist-to-artist similarity, genre-to-genre similarity, and inner/intra-artist song topics.

We’ll start by examining a pair of topic modeling techniques, both of which we’ll use to extract sets of key terms from the lyrics of our selected artist(s). Each set of terms will comprise a “topic” for that artist (or group of artists). Generally, 20 newsgroups is the most popular dataset for topic modeling, with the terms of different extracted topics corresponding with various news categories, such as politics, sports, or technology. We’ll follow a similar approach with our dataset of song lyrics.

If you’re interested in following along, I’ll be sharing snippets of code as I describe each step in the process. Full code, along with a sample set of lyrics, are available on my GitHub.

Let’s initialize a LyricsAnalyzer object, which we’ll use to store and process our set of lyrics. For additional details on the LyricsAnalyzer class, I recommend checking out the .py file of the same name. Our object will include lyrics from the late Lil Peep, an “emo rap” pioneer who was best known for mixing themes from pop-punk and rap, as well as somber lyrics about heartbreak, death, and various narcotics.

First, let’s enter the params.py file and update our artist dictionary accordingly. (Make sure to also change your path parameters as appropriate.)

artist_dict = {
  'country':[],
  'metal_rock':[],
  'pop':[],
  'rap':['lil_peep'],
  'soul':[]
}

In run.py, we can now easily prepare our Lil Peep data frame and initialize our LyricsAnalyzer object:

# Prepare data frame with lyrics from specified artists
lyrics_df = prepareLyricsFrame(artist_dict = artist_dict, path = corpus_path)

# Create LyricsAnalyzer object
la = LyricsAnalyzer(lyrics_df)

Upon initialization of our LyricsAnalyzer object, there are a few important behind-the-scenes methods - TF-IDF and Count vectorization - that automatically act on our set of lyrics. TF-IDF collects a list of every term that appears in any of our selected songs (note that “terms” are not necessarily limited to one word), and counts how often each term appears in each song. It then re-weights every frequency value according to the number of documents in which the associated term appears. Words that appear in virtually every song (“the”, “and”, “or”, etc.) will be down-weighted, and vice-versa.

While not shown in the above code, note that there are optional parameters, min_df and max_df, which are available in our LyricsAnalyzer initialization. Both parameters affect the TF-IDF vectorization that occurs upon initialization; the parameters’ defaults, which are set to 0.03 and 0.60, respectively, ensure that the resulting matrix will ignore terms which appear in less than 3% or greater than 60% of songs. This speeds up processing time for larger sets of lyrics, but also helps prevent topics from being “overtaken” by words which only appear in a couple songs. I recommend playing around with min_ and max_df values for yourself, as various sets of values might work better for certain artist combinations.

The LyricsAnalyzer object also initializes with a CountVectorized matrix, which, aside from the re-weighting described above, is identical to TF-IDF matrix. We will use the TF-IDF matrix for NMF (or non-negative matrix factorization), and the CountVectorized matrix for LDA (or latent Dirichlet allocation), respectively.

By calling the get_nmf_topics and get_lda_topics methods, we’re able to extract the top words for as many topics as we request. It’s important to remember that for both techniques, we’re the ones who choose the number of topics to extract - and our choices will have a significant impact on the extracted topics. Picking too few topics might result in each topic being overly vague and unfocused, while choosing too many will stratify our topics too narrowly. I’ve generally had the most success with somewhere between roughly seven and ten topics, but I recommend experimenting for yourself. Here, I’ll be using eight topics for NMF, and seven for LDA.

# Collect the top N topics (LDA)
lda_topics = la.get_lda_topics(n_topics=8)

# Collect the top N topics (NMF)
nmf_topics = la.get_nmf_topics(n_topics=8)

Let’s take a look at Lil Peep’s topics:

NMF’s first two topics are clearly both about the rapper’s romantic partners, with the first topic appearing far more romantic than the second. Topics 3 and 6, on the other hand, seem to deal with death and drugs, respectively, and Topic 5 appears vaguely associated with heartbreak. The topics for LDA are a bit more ambiguous than NMF’s, but we can still see similarly themed collections of terms appear in each topic.

Let’s repeat this exercise with some other rappers. After adding the discographies of 21 Savage and Tyler, The Creator - both of whom have relatively unique styles - we’ll now collect only three NMF topics, rather than the eight used above. Theoretically, we’d expect NMF to recognize and distinguish each rapper’s style from one another, and treat each one as its own topic. In order to more fully understand each topic, I’ve also elected to showcase the top twenty terms per topic.

Just as we hoped it would, NMF has perfectly identified each artist as his own topic. In order, the three topics above clearly represent Lil Peep, 21 Savage, and Tyler, The Creator. Of course, note that NMF does not label any topics for us - it’s up to us to interpret the underlying “meaning” of each one.

Finally, I’ve also built a Doc2Vec method into the LyricsAnalyzer object, with which we’ll create document “embeddings” for every set of lyrics in our dataset. Doc2Vec is an extension of Word2Vec, which, from a broad standpoint, uses a neural network to turn words into n-length vectors based on the words which most frequently surround them. Doc2Vec, on the other hand, outputs n-length vectors for entire documents, rather than for individual words.

Running Doc2Vec on our set of lyrics is very simple. I’ll be using the default parameters for vector length and window, among other parameters, but you can adjust these as you see fit. The default vector length is three-hundred - on the outskirts of the common range of one- to three-hundred. The method will return to us a series of vectors, each one of length three-hundred, with every vector corresponding to a different song.

la.doc2vec()

For purposes of more efficient song-to-song comparison, we’ll use t-SNE to condense our three-hundred column dataset into one with just two features. This is not an exact exercise, and post-t-SNE results are meant to be taken only as an approximation. While distances between groups of points are meaningful, they should not be taken literally. Further, we should not draw any conclusions from a point’s placement on the x- or y- axis; song positions are only meaningful when examined in relation to other songs on the same plot.

la.tsne_from_d2v(return_df=False)

After writing the resulting image to a .csv file, we can hop into R to visualize our results. I’ve colored the data points by artist and manually annotated some of the more interesting song placements. Let’s see what Doc2Vec made of our lyrics collection:

Look at how precisely Doc2Vec segregated each artist! While a few songs fall across the lyrical “boundaries” of other artists, each rapper in our dataset has a distinct section of the plot. Keep in mind that Doc2Vec had absolutely no knowledge that our dataset had multiple artists; in fact, we never passed any artist data into the model at all! The model simply recognized that certain words occurred most often in the presence of others, and created a series of resulting vectors that so happened to perfectly segment our data across artists. It’s also noteworthy which songs have lyrics that resemble songs from other artists. For instance, 21 Savage’s “A Lot” apparently bears a stronger resemblance to Lil Peep lyrics than to the rest of 21 Savage’s discography.

Repeating this process with select groups of artists can give us some pretty interesting results. Let’s see, for example, if songs from the individual members of mid-2010s rap collective Odd Future separate out as cleanly as the above example:

Aside from Tyler, The Creator and Earl Sweatshirt - both of whose clusters are, at best, ill-formed - there’s not much evidence of significant lyrical differences among Odd Future members. Each rapper’s respective data points are thoroughly scattered throughout the plot, suggesting that while Tyler’s lyrics might heavily diverge from those of Lil Peep and 21 Savage, they remain relatively similar to his Odd Future compatriots.

I’m compiled some more fun clustering examples below. Even better, I haven’t limited my search to just rappers, either - you’ll see comparisons involving pop artists, rock bands, and more! When appropriate, I’ve labeled particularly noteworthy data points in the same manner as above.

See all images here!

In general, it looks like rap songs tend to cluster by artist better than songs from other genres. My hunch is that this phenomenon, at least in part, is due to many rappers having distinct catchphrases or adlibs that they reuse from song to song. Country, pop, and rock artists generally don’t follow this trend, and certain genres (I’m looking at you, country) are often chastisted for what is perceived as redundant lyrical content between different artists.

I had a great time putting this project together, and I hope you enjoyed it too! All code is available on my GitHub.

Pitcher Similarities, By Way of Their Pitch Mixes

2019-01-11T00:00:00+00:00

To directly access the interactive web application, click here!

In this project, I’ll be creating a web application which computes similarities between different pitchers, based on the metrics of their respective pitch mixes. These metrics consist of pitch velocities, spin rates, and the actual pitch mixes themselves (i.e. which pitch types each pitcher throws). Teams might use this tool to identify underrated pitchers with high chances of big-league success. If a specific pitcher’s pitches are very similar to those of perennial All-Stars, for instance, it might suggest that the pitcher himself could have the same potential.

Of course, a major caveat involved in this analysis is that the similarity metric does not consider pitcher command – clearly a crucial component of pitcher success. While the pitch mixes of many relievers contain impressive velocities and spin rates, their struggles with control ultimately limit their respective ceilings. Conversely, some starting pitchers – Dallas Keuchel being a notable example – have sufficient command to compensate for a lack of top-notch “stuff.”

The data used for this project was collected directly from BaseballSavant. Only fastball (four-seam, two-seam, and cutter), changeup, curveball, slider, sinker, and splitter data was collected from the past three seasons. I used different cutoffs – by number of each pitch type thrown by each pitcher – for every pitch, in order to exclude pitchers without much big-league playing time, as well as pitchers with mislabeled pitch types. For instance, only the slider metrics of pitchers who have thrown at least two hundred cumulative sliders over the last three seasons are included in the dataset.

Depending on the pitch mix a user inputs into the application, behind-the-scenes functions collect the data of every pitcher in the dataset with that pitch mix. On the Select by pitcher tab, users can also enter a specific pitcher, rather than a list of pitch types, and the functions will use the pitcher’s n most commonly thrown pitches as the pitch mix.

Before calculating the distances between different pitchers’ respective mixes, however, it is crucial to scale the data. Revolutions per minute and miles per hour – the units for spin rate and velocity, respectively – operate on entirely different scales. A ten-mile per hour difference between two pitchers’ fastballs suggests a significant overall difference between the pitchers, while a ten-RPM difference between fastballs is next to negligible. Leaving the data unscaled would place a disproportionate emphasis on spin rate, as every pitcher would be considered relatively “similar” in terms of velocity. For this reason, the web application scales each set of pitch metrics separately before running any sort of distance algorithm.

The nearest-neighbor calculations are performed using Euclidean distance. I considered reducing the dimensions of the pitch mix datasets prior to running any distance algorithm, since pitchers’ velocities between different pitch types are likely correlated. However, I ultimately determined that maintaining a hundred percent of the variance between pitch types was too important to forego, and refrained from dimension reduction. Additionally, I also considered assigning different “importances” to each pitch type, commensurate to how often every pitcher throws each pitch. In the end, though, I decided this was unnecessary, as pitchers can easily change the relative frequencies of each of their pitches (but cannot always do the same with their pitches’ velocities and spin rates). Instead, however, I added a feature that allows users to place an overall emphasis on velocity or spin rate. By placing the slider at -1, distance calculations will consider pitch velocities twice as heavily as spin rates, and with the slider at 1, the opposite will occur.

As a user selects a specific pitch mix or pitcher, a technique called Multi-Dimensional Scaling, (or MDS) approximates distances between players on a two-dimensional plot. Because it is impossible for us to visualize any more than three dimensions at a time, MDS attempts to maintain distances between players as accurately as possible in as many dimensions as one wishes. Unlike t-SNE, which only maintains local structures of distances (i.e. Player A should be close to Player B), MDS works to maintain distances throughout the entire dataset. You may notice that pitcher neighbors, highlighted in orange on the Select by pitcher tab, occasionally stray from the data point that represents the selected pitcher. This is simply because MDS only approximates the distances between each point, as only so much variance can be captured in just two dimensions. Also, keep in mind that the x- and y-axes do not represent any specific metrics, and that the data tables that appear below the scatterplot are more accurate than the scatterplot itself in terms of player distances.

The full web application can be found here, and all code is available on my GitHub. Thanks for reading!

Lyrics, Pt. 1: Genre Classification

2018-11-03T00:00:00+00:00

In today’s day and age, we’re seeing more crossover than ever between musical artists of different genres. Rappers regularly feature in pop songs heard on the radio nationwide, and soul singers often perform the choruses of rap songs. Heck, even renowned rock musician Paul McCartney teamed up with Kanye West and Rihanna for an unexpected 2015 collaboration.

In the first installment of this project, I’ll be building a model that predicts a song’s genre based solely on its lyrical content.

As per usual, my first step in the modeling process was data collection and cleaning. Aggregating a full, tidy set of song lyrics isn’t the easiest task; many websites discourage web scrapers from crawling their pages – I may have gotten my IP banned from azlyrics last year for exactly this infraction – and some lyrics sites are rife with typos and errors.

First, though, I wrote a few small web scrapers to collect the names of artists in six different musical genres – country, metal, pop, rap, rock, and soul – from Billboard, Ranker, and TheTopTens.

Next, I used Ewen Henderson’s geniusR package to scrape lyrics straight from Genius (formerly known as RapGenius) and create a separate corpus of lyrics for each genre. Genius hosts lyrics for a relatively wide array of musicians, and I’ve found that the site, for the most part, keeps their lyrics relatively tidy.

I wrote a collectGenre function which, given a list of genre artists and a few additional parameters, will randomly scrape the lyrics of artists belonging to that genre. To ensure a reasonable variety of artists within each corpus, I scraped a maximum of 75 songs per artist. Finally, I discarded any result with a title that includes the phrases “album art”, “tracklist”, “script”, or “interview”, since Genius’s lyrics occasionally include transcripts of additional, non-song artist materials.

The below picture demonstrates this function in use; here, collectGenre has been set to scrape 1,500 rap songs of character lengths between 1,750 and 4,500:

To prepare each genre corpus for predictive modeling, I brought the scraped lyrics into Python and performed the following steps:

Remove misclassified artists: Some artists – particularly country and soul musicians – do not yet have their lyrics listed on Genius. Consequently, the function will erroneously return lyrics from the first-listed artist in the search results, whether or not that artist actually exists within the appropriate genre. Nearly 75 Tyler, The Creator songs, for instance, mistakenly appeared in the dataset of country songs and had to be manually removed.
Remove dual-genre artists: I found that the model was being confused by a few specific artists who, while initially classified as pop, could just as easily be classified as another genre. Chris Brown, for instance, could fit as a pop, rap, and soul artist, and Shania Twain is just as much a pop artist as she is a country singer.
Clean all text: This includes (a) removing everything between hard brackets, such as “[Verse 1]”; (b) changing certain types of slang, such as “walkin’” to “walking”; (c) removing all shorthand for repetition of a certain line, such as “x4”; (d) standardizing different forms of apostrophe; (e) elongating all contractions; and (f) removing all new lines (“\n”) and punctuation.
Stem words: Stemming words refers to transforming every word to its base form. This way, there will be no distinction between the words “operation” and “operative”, since both will be converted to “oper”.
Combine metal & rock: I found there not to be enough of a distinction between metal and rock lyrics to justify splitting the two. All models performed the best after having combined the genres into one category.

I started with a Multinomial Naïve Bayes model, which classifies a song based on the conditional probabilities of every specific word in a set of lyrics appearing in a song of a given genre. The model had a cross-validation score of 71%, which, while fair, can definitely be improved upon. In particular, the model had the most trouble classifying pop, country, and soul songs, which had test F1-scores of just 0.56, 0.63, and 0.64, respectively. Rap songs, on the other hand, were extremely accurate, with an F1-score of 0.90.

Next, I moved on to a support vector machine (or SVM) model. For the SVM, I created TF-IDF vectors from the full set of lyrics. For every song, every word’s frequency in that song is weighted by the number of other documents in which that word occurs. Word frequencies are also weighted by the length of each song’s lyrics, so that words in short songs are not given disproportionately high weights. Words which appear very often, such as “the” or “and”, will therefore receive very low weights, as while they may occur often in a given song, they’ll appear in nearly every other song as well. Therefore, TF-IDF vectorization essentially takes care of stopwords on its own.

After much trial and error with SVM parameters, I eventually settled on an alpha of 0.0001 and an ngram range of two. This means that consecutive two-word pairs (or “2-grams”) in a song’s lyrics are considered “vocabulary”, rather than just each individual word. These parameters resulted in a cross-validation score of 74%, a moderate but solid improvement over Naïve Bayes.

Finally, I used XGBoost to run a gradient boosting classification model, which - again, after trial and error with parameters - resulted in a score of 77%, another strong improvement over Naïve Bayes and SVM. Rap was again the easiest genre for the model, with a test F1-score of 0.96, and while pop, soul, and country were still the most difficult to classify, the F1-score for pop jumped from 0.55 to 0.67. Full parameters of the model can be found at my GitHub.

The gradient boosting model most commonly misclassified songs as metal/rock – likely a result of there being approximately twice as many metal/rock songs as songs of any other individual genre. Only one metal/rock song, however, was misclassified as rap, and fittingly enough, that song is by longtime rap-rock band Papa Roach. Additionally, there was only a single rap song – Lauryn Hill’s To Zion – incorrectly labeled as soul (although, if you listen to the song, you’ll see that it essentially is a soul song). Interestingly, Chance the Rapper’s Sunday Candy was one of only four rap songs mislabeled as country, and The Weeknd, labeled in my dataset as a pop artist, had six test songs classified as rap. Further, a few other pop songs were labeled as rap, likely because of the artists featured on those tracks; see Madonna and Nas’s 2015 collaboration, Veni Vidi Vici, as an example.

Just for fun, we can also test out the model on a few previously unseen sets of lyrics. I’ve found the XGBoost model much more reliable when deployed on full sets of lyrics rather than verse-long snippets, so, for the sake of space, these examples will instead utilize the SVM model. svmLyricClassifier is a small function I wrote which preprocesses a given set of text and uses the SVM model to predict a song genre for that text. As follows are the model’s results for songs of every genre:

To view full-size image, right-click and select "Open image in new tab."

As always, thank you for reading! All code is available on my GitHub. Part Two coming soon…

Lyrics, Pt. 2: Automated Songwriting

2018-11-03T00:00:00+00:00

For quick access to SongwriterBot, click here!

In Part 1 of my lyrics project, I scraped thousands of song lyrics from Genius and developed a model which classifies songs into genres based solely on their lyrical content. Now, in the second and final [okay, maybe not final] installment of the project, I’ll provide a brief explanation of how I developed SongwriterBot, which writes its own randomly generated songs.

This project utilizes Jeremy Singer-Vine’s amazing markovify package to create separate Markov models from the scraped songs of each genre. If you read Part 1, you’ll know that I scraped 1,500 songs per genre; here, I scraped approximately ten thousand country, metal, pop, rap, rock, and soul songs. For each genre, every song was split line-by-line and combined into a Markov model, which can randomly generate sentences from the text.

The following picture does a good job of demonstrating how the Markov models generate sentences. In this example, the Markov model includes just two sentences: “Mary had a little lamb” and “Mary had a giant crab.”

At the beginning of the sentence, the model only knows one possible word: Mary. Directly after Mary, the model has only seen one word: had. After had comes only a. After a, though, the model has seen two possibilities, each an equal number of times: little and giant. It randomly picks one of the two options, and continues as normal. It is this exact process – albeit, with far more possibilities – that occurs hundreds of times during the creation of one SongwriterBot song.

While back-to-back lines aren’t always thematically consistent, the content of the lines themselves tends to imitate each genre very well. Here is an example stanza for each genre:

Country: but i don’t understand / it would take my hand / and i won’t let me get my wheels / the screaming wheels and blackjack deals

Metal: look at the seventh veil / too blind to see the white whale / we’re on our crooked scale / divine - night in jail

Pop: you hate the word and we can come on and dance / don’t want to see the private jets to france / he’s giving me the vodka skip the criss / we are what they say that you would never miss

Rap: i smoke on some g s**t / international bring back piper’s pit / i scratch off on they dream / so when i’m with that guillotine killer team

Soul: don’t take your love from the start / love will never be apart / whenever you want to stick / that we have come here quick

Even better, we can also create “genres” that don’t actually exist. I added the entirety of the Harry Potter and A Song of Ice and Fire (more commonly known as Game of Thrones) book series as separate Markov models. markovify also allows users to combine multiple models – here are a couple amusing stanzas of a Harry Potter & rap song:

Harry Potter / Rap: i shouldn’t say s**t and they fall / the room at the yule ball / i let you have your mother’s eyes / but we’ve got to be cooked in pies / you give them to the ministry instead / grab the chrome to ya head / any form of cedric stood there / i walk in the cliff-top garden and into the air

If you’re interested in generating your own songs with SongwriterBot, click here. Because Heroku limits free web applications to 500MB, I removed rock, Harry Potter, and A Song of Ice and Fire models from the available options, as well as the ability to combine genres. Note that the app still occasionally runs into memory bugs and timeouts, which I am working to fix.

Once you generate a song, you can save it – which I recommend, seeing as the program will never create the same song twice – or try out your new lyrics against an instrumental track for any genre. Hope you enjoy!

MLB’s Biggest All-Star Injustices

2018-10-22T00:00:00+00:00

The Major League Baseball All-Star Game occurs a little more than halfway through every season. All-Star rosters consist of 32 players on each side, made up of twenty position players and twelve pitchers, and each team’s starting lineup is determined by a fan vote that takes place from May to July. Reserves are voted in by a combination of fans, players, and the Commissioner’s Office, and every MLB team is ensured at least one All-Star on their league’s roster.

In this project, I will create a model which predicts whether or not a given player will make his league’s All-Star team. This model will focus only on position players, although I eventually intend to produce a pitcher-specific model.

Using a small web scraper I wrote in Python, I quickly extracted the last thirty years of first-half player data from FanGraphs, and filtered my query to include only players with at least two hundred plate appearances at the time of the All-Star break. Additionally, I utilized the 2017 version of the Lahman Database for All-Star Game roster data, as well as regular-season player appearance data (used to determine the defensive position that each player most often played during a given season).

Ultimately, I compiled the following features. Note that not every feature is included in the final model.

Player statistics (batting average, on-base percentage, slugging percentage, home runs, Def, WAR, etc.)
Defensive position (the position at which each player most often appeared during the season, with any first-place ties broken at random)
League rank in each statistic at appropriate position (for example: In 2003, Barry Bonds led all National League outfielders in home runs at the All-Star break, and will therefore receive a league rank of 1 in home runs)
Team rank in each statistic (for example: In 2017, Bryce Harper led the Washington Nationals in home runs at the All-Star break, and will therefore receive a team rank of 1 in home runs)
Separate dummy variables for whether the player was a member of the Yankees, Red Sox, Dodgers, and Cubs, as well as a flag for whether the player was on any of the four teams
Approximate age at start of the season
Flags for whether the player’s team won or lost the prior-year World Series (inspired by the Royals’ 2015 All-Star turnout)

The final dataset consists of roughly 6,800 rows, each one representing a player’s first-half numbers for a particular season. Of these rows, approximately 1,200 (or 18%) are players who made their league’s All-Star roster. Additionally, the dataset includes 65 possible features. As evidenced by the dark red and blue tiles below, many of these features are strongly correlated with one another:

Not every feature is listed on the axes above, but AVG, BABIP, OBP, SLG, WAR, wOBA, and wRC+ all share correlations of at least 0.60. This stands to reason, as each statistic is a reflection of a batter’s offensive output. Similarly, team and league ranks for these metrics are highly correlated, and there is a strong negative correlation between offensive statistics and player ranks (since a player’s home run rank, for example, decreases as his home run total rises).

I found that a strong model could be produced without the vast majority of these features, and included in the model only a player’s position, AVG, HR, K%, SB, SLG, WAR, and Def, as well as whether his team won or lost the prior-year World Series and whether the player is a member of the Yankees, Red Sox, Cubs, or Dodgers. Unexpectedly, player rank – whether against the league or against the player’s team – was completely unnecessary in the model, and did not predict an All-Star Game appearance any better than by simply using the statistics themselves.

First, I will discuss a few unsuccessful modeling strategies - that is to say, techniques that did not significantly improve over my initial, simpler model. For instance, I had little success oversampling from the minority class with SMOTE and ADASYN. Only 18% of first-half player seasons in our dataset resulted in that player making the All-Star team, so I was concerned that this imbalance would prove problematic when training and testing models. Surprisingly, though, oversampling did not improve this process (and the class imbalance, in the end, was not problematic at all).

Additionally, I initially suspected that ridge or lasso regression would work well for the project, as they penalize - or, in the case of lasso regression, completely remove – highly correlated variables (of which there were many). As it turned out, logistic regression consistently outperformed both lasso and ridge regression, and, as a result, my final model is built using logistic regression. I achieved similar results using random forest modeling as with logistic regression, but would like to maintain the explanatory power that logistic regression allows. All models were trained and tested with an 80/20 split, respectively.

The logistic regression model achieved an AUC of 92.5. Using a classification threshold of 0.40 (which allows for a roughly equal number of Type I and Type II errors), the model performs as follows on the test dataset:

The model performed very well, both in precision and recall, for non-All-Stars. For players that made the All-Star team, the model is 70% correct in each measure. This means that of every actual All-Star in the dataset, the model correctly identifies 70% of them, and of all times that the model classifies a test row as an All-Star, 70% of these classifications are correct.

As follows is the set of feature coefficients. A one-unit change in each predictor will result in a corresponding change to the odds of the given player being named an All-Star. Positive weights correspond with increased chances at playing in the Midsummer Classic, and negative weights the opposite.

Feature	Coef.	Position	Coef.
AVG	0.012	C	1.07
DEF	-0.08	1B	-1.13
HR	0.14	2B	0.00
K%	-0.09	3B	-0.31
SB	0.03	SS	0.62
WAR	1.15	OF	-0.85
NYY/BOS/CHC/LAD	0.27	DH	-1.26
Won prior-year WS	1.00
Lost prior-year WS	1.17

Unsurprisingly, WAR is one of the most important predictors of an All-Star Game appearance. It is also interesting that playing on a prior-year World Series team – whether or not that team won – is another key positive predictor (although there may be some inherent bias in this conclusion; World Series teams, generally being better than an average team, are more likely to have All-Star-quality players).

Additionally, first basemen and designated hitters are the least likely to make the All-Star Game, and catchers and shortstops are the positions most likely to do so. This phenomenon may relate to position specialty; first base and designated hitter are arguably the least specialized of any defensive position, and very few designated hitters typically get selected for the All-Star Game at all. Since a higher number of players is likely to qualify for either of these positions, the chances of any individual first baseman making the All-Star Game are much lower than for a catcher or shortstop - two of the most specialized defensive positions in the game. There are, in fact, nearly two hundred more primary first baseman than catchers in the dataset, adding further credence to this theory.

By deploying our model on the full set of players from 1988 to 2017, we can see the biggest All-Star snubs (i.e., those who did not make their league’s All-Star roster, but deserved to). Percentages refer to the model’s predicted probability of each player making the All-Star Game.

(1) Gary Sheffield, 2007 (96.4%)	DH, Tigers	.303/.410/.560, 21 HR, 3.2 WAR
(2) Travis Hafner, 2006 (95.4%)	DH, Indians	.322/.461/.650, 25 HR, 4.2 WAR
(3) J.D. Drew, 2004 (94.7%)	OF, Braves	.312/.434/.628, 21 HR, 5.1 WAR
(4) Paul Lo Duca, 2001 (93.9%)	C, Dodgers	.346/.384/.615, 14 HR, 2.8 WAR
(5) Hanley Ramirez, 2007 (93.6%)	SS, Marlins	.331/.388/.538, 14 HR, 2.6 WAR

Conversely, as follows are the players who were voted to the All-Star Game, but should not have been. For at least a few of these players, their defense appears to have played a large role in their respective elections, although the rule requiring at least one player per Major League team was likely the key contributor. Cal Ripken’s 2001 All-Star appearance, on the other hand, is likely more a result of voter sentimentality than anything else.

(1) Cal Ripken, Jr., 2001 (0.46%)	3B, Orioles	.240/.270/.324, 4 HR, -0.6 WAR
(2) Lenny Dykstra, 1995 (0.68%)	OF, Phillies	.262/.347/.325, 0 HR, 0.7 WAR
(3) Carlos Garcia, 1994 (0.80%)	2B, Pirates	.267/.307/.332, 3 HR, 0.1 WAR
(4) Ken Caminiti, 1997 (0.82%)	3B, Padres	.247/.337/.379, 6 HR, 0.2 WAR
(5) Scott Rolen, 2011 (1.36%)	3B, Reds	.241/.276/.398, 5 HR, 1.0 WAR

Just for fun, here are the three least-deserving first halves overall (whether or not the player actually made the All-Star team):

(1) Adam Dunn, 2011 (0.008%)	DH, White Sox	.160/.292/.305, 9 HR, -1.7 WAR
(2) Ike Davis, 2013 (0.009%)	1B, Mets	.165/.255/.250, 5 HR, -1.5 WAR
(3) Ryan Howard, 2016 (0.015%)	1B, Phillies	.154/.214/.353, 12 HR, -1.6 WAR

Finally, we will deploy the model on players from the 2018 season. With the same 0.40 threshold here as used above, the model correctly classifies approximately 62% of All-Stars and 92% of non-All-Stars.

Besides the Dodgers’ Max Muncy (94.3%), there were no particularly heinous snubs in the National League; the biggest missed predictions were the Mets’ Asdrubal Cabrera (60.7%), the Dodgers’ Yasmani Grandal (58.9%), and the Nationals’ Anthony Rendon (45.8%), none of whom made the All-Star Game. On the other hand, Bryce Harper was voted in despite a probability of just 12.3%.

The American League, however, had a number of deserving players left off of the All-Star roster. The Angels’ Andrelton Simmons (77.9%), Eddie Rosario (76.7%), and Didi Gregorius (76.6%) all achieved statistics worthy of an All-Star bid, but none could surpass fellow American League infielders Jose Ramirez (99.9%), Francisco Lindor (99.0%), Alex Bregman (98.9%), Manny Machado (98.9%), or Jose Altuve (96.4%). Also noteworthy is Orioles first baseman Chris Davis, whose historically bad 2018 resulted in an All-Star probability of just 0.004%, and the Indians’ Jose Ramirez (99.87%), who narrowly beat out Boston’s Mookie Betts (99.86%) for the highest All-Star probability in the majors.

Thanks for reading – I hope you found this a fun project! All code is available on my GitHub.

Simulating the 2018-19 NBA Season

2018-10-12T00:00:00+00:00

The 2018-19 season doesn’t start until October 16, but there’s no need to wait until then to observe league results – instead, we can simulate the season before its first game even begins. I won’t be spending too much time discussing the technical details of my work, but will instead outline my steps in preparing and running the simulation.

1. Web scraping - 2016-17 and 2017-18 seasons

We will be training a model on the past two seasons’ worth of NBA games, and will deploy the model on a dataset of the 2018-19 schedule. First, though, comes data collection. Scraped datasets include the following; when not otherwise specified, data comes directly from basketball-reference:

The last two seasons’ worth of starting lineup and bench history (not scraped; directly from Kaggle
The last four seasons’ worth of player-by-player statistics
Current team-by-team depth charts (from ESPN)
Every team’s 2018-19 schedule
The league’s draft history since 2000, including every player’s respective draft position
Every individual season of rookie-year statistics from the last ten seasons

2. Lineup-level data aggregation – 2016-17 and 2017-18 seasons

For every game over the last two seasons, I subset each team’s lineup to only the five starting players and the three bench players with the most minutes in the game. Any ties (for example, the third and fourth bench players playing the same number of minutes) were broken at random. Note that each of the two lineups in a game are given separate rows in the dataset – one row from the home team’s perspective and one row from the away team’s perspective.

Using the past three seasons of player data, I computed weighted totals of four separate statistics – Win Shares (WS), Box Plus/Minus (BPM), minutes played per game (MP/G), and percentage of games started (GS%) – for each NBA player. All four statistics were weighted by the number of minutes each player played in a given season. The Celtics’ Gordon Heyward, for instance, played only five minutes in the 2017-18 campaign, so his statistics from that season bear virtually no weight in his metrics.

I then calculated the average WS, BPM, MP/G, and GS% of both teams’ respective starting lineups and bench players, as well as the differences between a team’s WS, BPM, MP/G, and GS% and their opponent’s WS, BPM, MP/G, and GS%, respectively. Finally, for each lineup, I added columns for the time and distance (Euclidean, based on city latitude and longitude) since the team’s previous game.

3. Train model

Before training my model, I performed principal components analysis (PCA) on the differences between team and opponent starting lineup and bench WS, MP/G, and GS%. Because Win Shares are a counting statistic, players accumulate more Win Shares the more they play, which will presumably lead to high correlation between the three metrics. The combination of MP/G and GS% has the same problem, as players who starts a large percentage of their games are also likely to play a large number of minutes per game.

BPM, however, I excluded from PCA, as it is a rate statistic rather than a counting one. A player might have a relatively high WS total simply by virtue of receiving a large share of minutes, but this is no guarantee of a high BPM. Andrew Wiggins and Dwight Howard, for instance, both started virtually all of their teams’ games in 2017-18, but neither played particularly well; both players have lower BPMs than their minutes totals would suggest.

Condensing WS, MP/G, and GS% with PCA (separately for starters and bench players) confirms that the three variables are very highly correlated; the first principal component contains 92% of starters’ total variance, and 97% of bench players’ total variance.

After trying different types of modeling – logistic regression, decision trees, random forests, and gradient boosted trees – I found that random forests’ results consistently outperformed those of the other three models. The best model consisted of the following features, with win/loss as a response:

Team location (home/away)
Time since previous game
(Euclidean) distance from previous game
Team BPM less opponent BPM (starters)
Team BPM less opponent BPM (bench players)
Principal component 1 (starters)
Principal component 2 (bench players)

Aside from the seven features above, I also initially included team winning percentage less opponent winning percentage as a feature, and ran a version of the model with starter and bench BPM included in their respective principal component analyses.

To tune the random forest model, I performed a repeated grid search across three parameters: maximum tree depth, maximum number of features, and minimum samples per leaf. Because the training & testing sets differed with each iteration, the optimal parameters changed slightly each time. The most common set of optimal parameters, however, were a maximum tree depth of four, five maximum features, and five minimum samples per leaf.

Of the seven final features, by far the most important for win/loss classification is the difference in starting BPM. Team location and starters’ principal component 1 come in a distant second and third, respectively. Overall model accuracy varies with each training & testing split, but for the most part hovers somewhere between 62% and 66%. This result serves our purposes perfectly well, although it is less accurate than Vegas predictions.

4. Player-level data aggregation – 2018-19 season

For the 2018-19 season, lineups are uncertain, since each of our season simulations will generate new player injuries and will randomly choose different bench players to fill injured starters’ spots in the starting lineup. Therefore, my script does not aggregate any 2018-19 lineup-level statistics until it actually determines what each game’s lineups will be. In Step 2, I described the process of collecting and weighting player WS, BPM, MP/G, and GS%; this same process is used to gather 2018-19 player metrics.

This method, however, is ineffective for any rookies, as they have no prior-year numbers to aggregate. To combat this issue, I imputed all 2018-19 rookie statistics based on their draft positions. Looking the medians of the last ten years of rookie-season numbers, there are clear trends in each of the four relevant player metrics as we move from pick #1 to pick #60. Take GS%, for example:

For #1 pick DeAndre Ayton, then, I assumed his GS% to be approximately 80% - in line with the orange data point that corresponds with the first overall pick. Similar intuition is used for all rookie WS, BPM, and MP/G.

5. Simulate the 2018-19 season

To simulate the upcoming season, I initialized a dictionary of player injuries – sourced from CBS Sports and RotoWorld – which determines how many games at the start of the season each player will miss. DeMarcus Cousins, for example, is still recovering from his Achilles injury, so I projected him to miss the first ten games of the year. Jimmy Butler, on the other hand, I have conservatively projected to miss the entirety of the season, since he is still yet to be traded from the Timberwolves at the time of writing.

Using the 2018-19 schedule we scraped earlier, we can now produce simulated sets of starters and bench players for every game in the season, all of which come directly from the first- and second-string players on the ESPN depth chart. Because my model aggregates bench statistics from only the first three bench players, I randomly select three players from all second-stringers. When randomly selecting bench players, probabilities are weighted by each player’s respective weighted MP/G (or, for rookies, imputed MP/G), so that bench players who have historically received the most playing time will be selected more often.

Next, for every game, my script iterates through every player on each team and assigns them a random number. If that random number is suitably small, that player has a one-in-eight chance of a “severe” injury, and a seven-in-eight chance of a “mild” injury. Depending on the severity of a player’s injury, their number of games missed will be randomly chosen from one of two exponential distributions. The nature of an exponential distribution makes it more likely for a player to miss only a small number of games, but the larger scale of the “severe” distribution increases the odds of a severely injured player missing a larger number of games than a mildly injured player.

Once we know which players will miss each game, we will be left with new gaps in starting lineups and benches. Any lineup with fewer than five starters will have their gaps filled with second-string players, and any bench with under four players will have their gaps filled with third-stringers. Just as bench players were randomly selected with weights based on their MP/G, gaps in starting lineups and benches are filled in the same manner.

By applying the random forest model on the full 2018-19 schedule (now with full sets of starting and opposing lineups and bench players), we now have continuous zero-to-one predictions of a given lineup winning their matchup. Again, remember that each game occurs from two teams’ perspectives, and therefore comprises two distinct rows in the dataset. As such, each team for any given game will have its own winning likelihood - and both teams’ likelihoods do not necessarily sum to 100%. A game between Boston and Philadelphia, for example, might result in the Celtics being given a 57% winning probability and the Sixers being given a 48% winning probability. (These, of course, add up to 105%, not 100%.) For more separation between team-opponent winning likelihoods, I raise each percentage to power of 1.4. Boston and Philadelphia’s respective 57% and 48% winning probabilities, then, become 46% and 36%. After standardizing this set of figures to sum to 100%, the Celtics are left with a 56% chance to win, and the Sixers with a 44% chance. A winner for the game is randomly chosen using these two probabilities.

Once the above process completes for all 2,460 matchups, we will have successfully simulated the 2018-19 NBA season. To properly account for the in-season and between-season variance that can occur, though, we need to perform more than just one simulation. Certain teams, like Golden State, will win the majority of their games even after an injury to one of their star players; the Lakers, on the other hand, will struggle without LeBron. Each of our simulations has different randomly generated injuries, so by simulating many different times – say, five hundred – we can gain a much clearer picture of every team’s projected win total for the 2018-19 season.

Results

After five hundred simulations, as follows are the projected final standings of the upcoming NBA season:

To view full-size image, right-click and select "Open image in new tab."

With the exception of the Raptors, who project to win the East by thirteen games, the East’s playoff picture appears very competitive. Boston, Philadelphia, Milwaukee, and Washington all finish a game apart from one another, and the Hornets surprisingly manage to sneak ahead of the Heat for the sixth seed in the conference. Indiana, unexpectedly, misses the playoffs altogether, and the Knicks bumble their way to the worst record in the league. In the West, Golden State outpaces both the Rockets and Thunder for the top seed in the conference, and LeBron’s Lakers finish with a respectable 45 wins. Notably, neither the Butler-less Timberwolves nor the Pelicans reached the playoffs, and Portland finished with eight fewer wins than in 2017-18. Even with these surprises, though, these projections appear totally plausible, and by next spring we’ll know just how accurate they were.

Thank you for reading – I hope you found the project interesting!

All code is available on my GitHub: https://github.com/saisenberg

Which NFL Fanbase Had the Worst Week 1?

2018-09-13T00:00:00+00:00

I’ll be collecting fanbase statistics throughout the entire 2018 NFL season - click here for a Google Sheet of every team’s week-by-week metrics, and click here for an interactive Shiny application of team data.

This past weekend, the American sports world turned its eyes to the first matchups of the 2018 NFL regular season. From Thursday night to Monday night, we watched Sam Darnold and Patrick Mahomes impress in their debuts, and watched as a one-legged Aaron Rodgers commandeered a twenty-point comeback against Chicago. We saw teams that were expected to lose – the Bills and Raiders, for example – get blown out, and teams that were supposed to win – most notably, the Saints and Lions – upset by their presumed weaker opponents. We even watched as Cleveland finally ended their seventeen-game losing streak in the most quintessentially Browns way possible – by playing their AFC North rival Steelers to a draw.

After the Bills-Ravens game mercifully concluded, with Baltimore walloping Buffalo by a final score of 47-3, morbid curiosity brought me to r/buffalobills, the Bills-specific forum (or “subreddit”) of Reddit, where Bills fans’ early reviews of the team’s first game were less than stellar:

Intuition might suggest that since Buffalo had lost by the largest margin of any team in the league, their fanbase would have been the most negatively impacted by the loss. On the other hand, the Bills were largely expected to struggle this season; after all, they traded starting quarterback Tyrod Taylor in March, and were starting a quarterback who had, less than a year ago, thrown five interceptions in only one half of play. A fun exercise, therefore, would be quantifying Bills fans’ negativity – was the Buffalo subreddit demonstrably the most negative of all thirty-two teams? And if not, which fanbase had the worst Week 1?

First, I scraped thousands of rows of data for each NFL team, with each row representing a comment for any post left on that team’s subreddit between five minutes before the end of their Week 1 game and twenty-four hours after the start of their Week 1 game. A Python library, PRAW, exists for the express purpose of scraping Reddit submissions and comments, and although PRAW can collect only the thousand most recent posts in a subreddit, this proved a non-issue, as no team’s subreddit had even close to that many posts in such a short time span.

After data collection, I used the VADER SentimentIntensityAnalyzer algorithm within Python’s nltk library to perform sentiment analysis on each individual comment. Sentiment analysis determines the overall positivity or negativity in a given piece of text, as well as the sentiment’s overall intensity (or “polarity”), while also accounting for slang, negation, and strength of language. Every subreddit comment is given four scores: positivity, negativity, neutrality (i.e., neither positive nor negative), and “compound,” which is an amalgam of the others. While a comment can be partly positive, partly negative, and partly neutral, all three scores must sum to 1 (see the attached link for examples). While all individual positivity, negativity, and neutrality scores can take values in the range [0, 1], I will be multiplying every score by a hundred for ease of reading.

However, before performing sentiment analysis, I first had to separate all contractions (e.g., “don’t” or “hasn’t”) and remove all Reddit-style HTML from each comment, as well as additional unnecessary punctuation. I also deleted every mention of “Luck” from the Colts’ subreddit, as VADER cannot distinguish between the dictionary definition of luck (which it considers a “positive” word) and the word in the context of Colts quarterback Andrew Luck. The word “super” was also subject to removal - this time league-wide. Because teams with 2018 Super Bowl aspirations are more likely to use the word “super” (which, again, VADER considers positive), inclusion of the word would artificially inflate that team’s positivity scores.

But enough of the boring technical details – on to the results!

Teams

As follows is a scatterplot of the positivity and negativity of each subreddit’s average comment between Thursday and Tuesday nights. The happiest fanbases (i.e., the subreddits with the highest average positivity and lowest average negativity) appear in the lower right-hand corner of the plot, and the least happy fanbases appear in the upper-left. Each data point has also been color-coded in accordance with the outcome of that team’s game.

While Buffalo may have suffered the worst loss of Week 1, their fanbase actually had nowhere near the highest mean negativity in the league. That honor instead goes to the New Orleans Saints, who squeaked by the Tennessee Titans (by less than a one-hundredth of a percentage point!) to be titled the league’s most negative Week 1 fanbase. Tennessee, however, also has a significantly lower mean positivity score than New Orleans, and are the only team in the league with a lower average positivity than average negativity. Sitting through the longest game in NFL history clearly soured the mood of Titans fans, and losing their starting quarterback and tight end to injuries - as well as the game itself - certainly didn’t help matters.

We can also observe the weather delay’s affect on Miami fans. Although the Dolphins won the game, their mean positivity and negativity scores land them firmly among Week 1 losers Atlanta and Houston. For the most part, however, the graph cleanly splits between winning and losing teams; r/Colts and r/49ers both buck this trend, with the former subreddit actually finishing the weekend with the highest positivity score in the league. The Colts, in fact, finish with both higher positivity and lower negativity than the Bengals - even though Cincinnati won their matchup!

The Redskins and Vikings clearly have the two happiest fanbases of Week 1, as both teams emerged victorious in their opening games - the latter, incidentally, over the (curiously positive) 49ers. Also of note is that while New York Jets fans had the lowest negativity score in the league – which makes sense, given the fantastic Week 1 play of rookie quarterback Sam Darnold – they still finished with a relatively moderate positivity score. Perhaps years of disappointment have simply left the Jets fanbase incapable of happiness.

So how do we definitively decide which team’s fans had the best and worst Week 1 in the league? By subtracting each subreddit’s mean negativity from its mean positivity, we can rank teams by their net positivity scores. Here are the results, ranked from best to worst:

By net positivity, the Minnesota Vikings’ fanbase had the happiest Week 1, and the Tennessee Titans’ fanbase had the worst. Looking at the Game Result column, we can still observe a fairly clean split between winning teams and losing teams, showing that fanbases of winning teams generally exhibit more positivity than fanbases of losing ones. Winning margin, however, appears to play little part in a team’s ranking; the #1-ranked Vikings won their matchup by eight points, while the #10 Ravens beat the Bills by over forty.

Posts

As follows are the individual posts which resulted in comments with the highest mean positivity and negativity. I have excluded posts with fewer than twenty-five comments from this section of results so that posts with very few – but very polarized – comments do not dominate the list. I have also excluded posts whose inclusion on the list was only due to one user’s repetition of the same word or phrase.

Here are the three most positive posts, along with each post’s mean positivity score and relevant team:

Patriots (50.3): Phillip Dorsett becomes the 69th player to catch a TD from Tom Brady
Bears (31.8): Game day giveaway!
Bears (25.0): Aaron Rodgers

Now, as follows are the three most negative posts of Week 1. If you look closely, you might spot another explanation for the Dolphins’ low net positivity score:

Dolphins (44.5): IT’S F*** THE JETS WEEK
Lions (36.0): Everyone say it with me
Ravens (20.9): Bills fans being classy towards Matt Judon…

Comments

Just as we did on a post-by-post level, we can also identify the overall most positive and negative comments of the weekend. For each subreddit, I found the 67th percentile of the length (in characters) of that subreddit’s comments, and, for purposes of this list, am only including comments between the 67th and 100th percentiles. Additionally, the following lists comprise only comments with at least twenty-five upvotes.

Here are the three most positive comments of Week 1:

Browns (57.2): “Feeling pretty good about this. They played like they belonged and I feel like we woulda won if not for the weather.” (Long_island_iced_Z)
Ravens (54.2): “Baltimore Ravens - 2000 - 4-0 - 12-4 - NFL Champion / Won the Super Bowl. Seems like good news to me!” (NeonFlame126)
Rams (51.6): “I loved when McVay ran down there to hug Peters and celebrate with him. Our coach is the best.” (mynameisdan6)

And the three most negative:

Panthers (53.4): “DIRTY F****** B*******. SHOVING YOUR HELMET INTO A GUY’S KNEE. THAT DUDE GO GET F*****. F*** THE COWBOYS.” (nglove492)
Bengals (44.7): “F*** THE NFL PANSY RULES, F*** THE REFS, F*** AJ’S FUMBLEITIS, AND HARDCORE F*** THE ANDREW LUCK COMEBACK NARRATIVE B******* AND THE STEELERS! WHO DEY!” (PM_Me_Your_PBnJ)
Eagles (42.7): “We force a punt and then pull that s***. Completely unacceptable, standards have to be higher. Sit that f***er” (philcoke12)

Words

Finally, we can use nltk’s FreqDist function to identify common words contained in each subreddit’s most negative comments. This way, we can achieve a rough idea of what specific topics have inspired the most negativity for each fanbase. I have removed all curse words from the following results, but rest assured that every single team’s list of most common negatively-associated words included at least four or five curses. Results have also been lightly edited for brevity.

49ers (r/49ers): jimmy (5), cancer (4), injury (4)
Bears (r/chibears): packers (10), loss (9), collinsworth (8)
Bengals (r/bengals): penalty (11), nfl (9), steelers (8), dirty (4), refs (4)
Bills (r/buffalobills): tyrod (6), benjamin (5), terrible (4), peterman (4)
Broncos (r/denverbroncos): raiders (6), von (6), offense (4)
Browns (r/browns): hue (6), sucks (6), offense (6)
Buccaneers (r/buccaneers): fire (10), saints (10), smith (6), cannons (5)
Cardinals (r/azcardinals): rosen (7), bradford (7), riot (6), mccoy (5)
Chargers (r/chargers): cut (5), hate (5), us (5)
Chiefs (r/kansascitychiefs): defense (6), terrible (3), crazy (3), killing (3)
Colts (r/colts): mad (4), doyle (4), hit (4)
Cowboys (r/cowboys): fire (15), offense (13), romo (9), dak (9)
Dolphins (r/miamidolphins): jets (119), bills (29), patriots (17), delay (5)
Eagles (r/eagles): penalty (11), collinsworth (7), sullivan (5), brutal (4)
Falcons (r/falcons): sark (23), offense (11), fire (10), refs (9), ryan (8)
Giants (r/nygiants): flowers (17), cut (7), dallas (6), offensive (4)
Jaguars (r/jaguars): penalty (7), offense (6), dumb (6), cam (5)
Jets (r/nyjets): hate (5), announcers (3), sad (3), suck (3)
Lions (r/detroitlions): die (18), fire (14), sorry (11)
Packers (r/greenbaypackers): bears (15), suck (15), miss (9)
Panthers (r/panthers): hate (5), lawrence (4), saints (4)
Patriots (r/patriots): hate (5), hill (5), post (4), pissed (4), brady (4)
Raiders (r/oaklandraiders): carr (18), talib (17), gruden (9), hate (8)
Rams (r/losangelesrams): hold (12), cook (5), gruden (5), dirt (5), jared (4)
Ravens (r/ravens): steelers (8), fans (6), bills (5)
Redskins (r/redskins): dallas (4), peterson (3), viking (3), giants (3)
Saints (r/saints): ryan (7), rob (6), pizza (6), offense (5)
Seahawks (r/seahawks): announcers (12), ifedi (7), refs (5)
Steelers (r/steelers): ben (11), offense (9), penalty (7), haley (7)
Texans (r/texans): johnson (8), kevin (7), watson (5)
Titans (r/tennesseetitans): worst (8), refs (7), dolphins (7)
Vikings (r/minnesotavikings): rodgers (13), rule (8), bears (5)

A few takeaways:

Dolphins fans exhibit negativity towards their entire division, but especially towards the Jets (as we learned in the Posts section).
Bills fans still miss Tyrod Taylor, and are less than thrilled with wideout Kelvin Benjamin.
Atlanta’s fanbase is fed up with their offensive coordinator.
Giants fans have seen enough of offensive tackle Ereck Flowers.
r/oaklandraiders is concerned with the recent play of Derek Carr. They also associate new coach Jon Gruden and opposing cornerback Aqib Talib with negativity.
Seahawks and Jets fans were both evidently displeased with the announcers of their respective games. The Bears and Eagles fanbases both seemed particularly upset with NBC commentator Cris Collinsworth.

Thank you for reading - I hope you enjoyed the project! For anyone interested in using PRAW to scrape Reddit, this tutorial can get you started.

All code is available on my GitHub: https://github.com/saisenberg