Welcome to Funfetti Chicken
Final Writeup for Funfetti Chicken
Big Idea
Our goal was to generate the perfect meal to complement a song, based on the song’s mood. We wanted to capture the complexities of the music and learn a bit about machine learning by analyzing various factors such as tempo, lyrics and key to predict the song’s mood.
Process
We wanted to go beyond directly basing food suggestions on lyrics, tempo, or key - all of which we could just use APIs to acquire. As such, we set out to apply machine learning techniques to predict a song’s mood--effectively making our goal the creation of a program which could determine the subjective sentiment, or mood, of a song--using only objective and quantifiable parameters. We anticipated having to make a few decisions regarding the format of the mood predictions. For one, we needed to figure out whether they would be ranked on scales such as happy to sad, mellow to angry, etc. or if they would simply be categorized as a set of mood descriptors. Based on that decision, we could determine what kind of machine learning algorithm to use to predict a mood’s classification or scale value. In the end, however, it turned out that these decisions were largely made for us. Because machine learning requires a large set of training data in order to work, we had to collect a database of songs and their associated parameters, as well as assign a mood to each of them. It was impractical for us to manually decide on the moods of thousands upon thousands of songs; rather, we elected to use Gracenote’s API. Gracenote has a substantial collection of the moods of popular songs - determined through human judgment - and was the most accessible and wide-ranging database of human-chosen moods we could find. As such, the Gracenote mood for a song is considered the correct mood for ML training and testing purposes. One side effect of this is that Gracenote’s evaluations are distinct categories (e.g., ‘Empowering’, ‘Melancholy’, ‘Upbeat’), rather than numerical data; as such, we found ourselves taking a classification, rather than calculation, approach to the machine learning.
Attribution:
APIs and Packages: Echonest API, which provides the numerical data for songs through an API call. Pyechonest, a python implementation of Echonest. LYRICSnMUSIC API, which provides the first twelve lines of a song’s lyrics through a web call. Gracenote API, which provides the mood data used in training and testing. Pygn, a python implementation of Gracenote. Requests, a python library for getting HTML data from the web. BeautifulSoup, a set of tools for harvesting specific information and fields from an HTML file. Pickle, which we used to create a persistent database - in turn allowing us to use more training data, with less downtime. Pattern, which allowed us to quantify the sentiment of lyrics. Scikit-learn, which contains a wide variety of tools and frameworks for machine learning (including Support Vector Classification, which we ultimately used). Tkinter, a python library for creating GUIs. Swampy, which provides framework for GUIs and accepting user input. Billboard, which (among other things) maintains a website listing the top hundred songs for a given week, and has this data for the last few decades stored on a week-by-week basis on their website.
Github, which provides hosting and made git version control easier and more accessible. Python (and the Python documentation), which has examples for pretty much every technical detail one could imagine. Techniques from StackOverflow - specifically, http://stackoverflow.com/a/15743618/4698228.
What It Does
Our program takes a song title and artist as input, uses SVC to predict its mood, then returns a food associated with that mood.
This uncertainty (confusion) matrix shows the number of songs our program predicted to fall under a given mood category versus the number of songs which actually had that mood.
[[40 13 4 1 8 29],
[ 8 54 8 24 6 1],
[10 7 55 16 6 12],
[ 1 22 12 61 3 0],
[ 8 11 10 8 55 10],
[15 1 7 0 7 67]]
The algorithm we generated was able to identify an Empowering song 42% of the time, an Excited song 54% of the time, Sensual songs 52% of the time, Energizing songs 62% of the time, Lively songs 54% of the time, and Sentimental songs 69% of the time - making the total accuracy a hair above 55%.
Each mood corresponds to a different food in our mood to food dictionary, which is currently personalized to accommodate the needs of Gabe, who only eats various quantities of bananas and plantains. When he enters the song ‘Never Gonna Give You Up,’ our program categorizes the song as brooding and determines that Gabe should eat bananas. When Gabe listens to yearning songs like Coldplay’s ‘Clocks,’ he tends to lose his appetite, so our program returns that Gabe should only eat a single banana in this instance. If Gabe were to listen to an empowering song, he would be told to eat plantains, and for an excited song a single plantain would be recommended.
How It Works
The central focus of our program is its machine learning aspect, which we built around scikit-learn’s LinearSVC. LinearSVC takes in a large database of songs and their associated parameters (key, tempo, etc.) and learns how these parameters are associated with the mood of each song. Then, given a song with parameters but no mood associated with it, the program is able to predict the song’s mood. The program then refers to a dictionary of moods and their associated foods to return the appropriate food for the song.
In order to create a database of training data for LinearSVC, we scraped hundreds of song titles and artists off of Billboard’s lists and used Echonest’s API to get each song’s associated parameters. We then used the LYRICSnMUSIC API to get a snippet of each song’s lyrics, and used pattern’s sentiment analysis to get a happiness/sadness value for the lyrics. Finally, we used Gracenote’s API to get the song’s mood classifier (one of about 25 different moods). This information was all stored in a Song class for each song, and each song class was pickled and added to the database. LinearSVC was then able to analyze the database of songs, determining the relationship between each song’s parameters and its associated mood. After this training process, it could be passed a song whose mood had not yet been determined, and predict what its mood classifier should be. This is where our user would become involved. We constructed a simple graphical user interface to ask our user for the title and artist of the song they wanted a food for. When the user submitted this information, our program went through a similar process as it had in constructing the database- it used APIs to find the parameters and lyric sentiment associated with the song, then stored that information in a Song class for that song. However, this time the mood was not included. When LinearSVC was passed the information about the song, it predicted which mood classifier should be associated with the song. From a dictionary which mapped mood classifiers to foods, the program then selected the appropriate food to eat while listening to the song. The name of this food was returned to the user in the GUI.
README
Project Funfetti Chicken Matching Music to Sentiment, and Sentiment to Food.
Olin College, Software Design Spring 2015.
EXPLANATION:
get_a_bunch_of_songs.py is a script which only needs to be run once. It uses the internet to generate a huge list of recently-popular songs, which it then stores in billboard_song_list.txt
billboard_song_list.txt is a text file which contains a list of pickled (artist, song_name) tuples from Billboard.
songtext.py contains a function to get the lyrics from a song. It is used as part of the Song_data object's initialization function.
make_echonest_database.py establishes the Song_data class. When a Song_data object is initialized, with artist & name as its parameters, it automatically calls echonest api to get things like 'tempo' and 'key.' This info then gets stored as parameters for this object. Additionally, the object also stores lyrics gotten from songtext.
wrapper.py is a wrapper. It uses the list from get_a_bunch_of_songs and the class from make_echonest_database, and then makes a big database of Song_data objects - one for every entry in the list of songs. This then gets pickled for later use, under song_database.txt.
song_database.txt is a file containing a list of pickled Song_data objects. In other words, it's a big list of objects which have things like 'name', 'artist', 'mood,' 'lyrics,' etc. as parameters.
all_billboard_songs.txt is a pickled database of song titles and artists acquired by scraping billboard lists.
ML_algorithm_creator is the main code that takes information from the database to train the ml and prompts the user to enter a song and artist, which it returns a corresponding food to.
bigger_database_creator.py takes 57 years of weekly top 100 songs and tries to add them to a database, excepting those that are already a part of the database. This allows us to generate our training and test data for the ml code. Since it’s already been used to create fifty_seven_years_database, this file need not be run unless the database needs to be recreated.
fifty_seven_years_database.txt contains all the artists and song names gathered by bigger_database_creator.py
pygn.py gathers mood information. This saves us from having to generate the “correct” moods for songs ourselves and instead get an impartial third party to do it for us. This allows us to train the ml with thousands of songs easily instead of us having to manually enter in a few hundred over a long period of time.
Process Review Timeline
Funfetti Chicken Project Proposal
The Big Idea
The main idea of the project is to take the tone, lyrics, tempo, and rating of music to determine the emotion expressed by the music and then generate a food list that perfectly matches the song. We will be exploring the concept of audio sentiment analysis that goes beyond positive or negative and extends to anger, sadness, happiness, lethargy, and whatever else we can pull out of music. The minimum viable product will be an algorithm that judges a song’s emotion based on one of the four song aspects described above (e.g., takes in song lyrics and outputs an emotional evaluation). The stretch goal is to have algorithms looking at those four aspects, paying special attention to those we deem most capable of determining the emotion of songs, and returning a perfect accompanying food.
Learning Goals
We want to learn to analyze data more specifically and more in depth than we had in previous projects. We hope to learn to effectively interpret multiple different types of data at a time, and figure out how to prioritize each type or combine multiple types.
Implementation Plan
Our implementation would divide the overarching goal into multiple sub-functions or modules, which would converge to produce the output.
Libraries which might prove useful: Pattern (for lyric analysis), Librosa (for tempo/key analysis), Echo Nest (for key/instrument analysis),
Project schedule
weeks 1-2: complete MVP weeks 3-4: finish algorithms for at least ¾ major song aspects weeks 5-6: finish algorithms, add the food side, combine the algorithms into one program that work together
Collaboration plan
We’re used to working on a coding problem individually for a long period of time until eventually we figure it out, but we decided that it may be beneficial to try a different approach for this project. We’re going to try pair programming to see if that method works better for us, and if it doesn’t work very well we may switch to individually working on modules of the program and then combining them. We have four elements of music that we plan to analyze, so that will divide nicely among four people whether we’re pair programming or not. Once we’ve finished the algorithms for these elements, we’ll put our combined code on github and work on it as a whole.
Risks
Integrating the algorithms to analyze the music and translate them to food. In order to get an accurate translation we have to analyze many variables for each song and match them with correlated food. Getting the algorithms to effectively evaluate the music for sentiment as complicated as what we want could prove very tricky.
Additional Course Content
If we were to cover ways in which text analysis determines sentiment, that would be helpful for us in branching our sentiment analysis beyond just positive or negative. Otherwise we would have to rely mostly on libraries already available instead of building our own tools.
Technical Review
Background and Context:
Our project is an investigation of song sentiment and the ways we can use programming to analyze it. The end goal is to use machine learning to give our program the ability to categorize new songs based on a training set. We are using a variety of APIs to gather essential data on songs in order to create the training set and give the program the information it needs to analyze new songs. Once the sentiment of the song has been realized, the program will offer choice foods to go with the song.
Key Questions:
What we most want to learn from the review is what machine learning technique will be best for the information we are trying to gather. Also, we need to find an efficient way of generating a training data set, so any opinions on other APIs that would help our project would be greatly appreciated.
Agenda:
We plan to give a brief summary of what our project is, what the finished product will look like, and what we have so far. The rest of the time will be allotted to surveying the group about the key questions we have listed above and facilitating general discussion of our ideas and chosen methods.
After the Review:
Feedback and Decisions:
The advice we received on machine learning was that we should try to look for trends in the data, and that thought pushed us towards Association rule learning because it is effective at finding correlations between variables, which is exactly what we are looking for. For generating the training data, we were told that the Rovi API was a good option for getting the moods of songs, saving us the trouble of having to manually listen to songs and enter in how we thought they felt. This API allows us to get an unbiased opinion on the mood of the songs.
Review process reflection:
The review was an overall success. We got the questions we needed answered answered and we got interesting new leads regarding everything we were concerned about. We may have provided too little context for the groups to be of a large help in regard to the machine learning, but we had decided it was too hard to get a good enough understanding of machine learning to really help our team in a short period of time. In regard to the agenda, we stuck to it fairly well, only needing to backtrack and re-explain things once or twice. The time we allotted for general discussion was very rewarding and worth the effort. We are going to plan our discussion of machine learning a little better and try to present people with the opportunity to go beyond what we even considered.
Preparation & Framing Document for team Funfetti Chicken:
Background and Context:
Our project is based on taking a song, finding relevant information (e.g., lyrics, tempo, key) about it, and translating that to a sentiment. As a bit of a "flourish," the sentiments will then be mapped to an appropriate food with a (very subjective) pseudo-dictionary of our own devising.
Currently, we've written and assembled the bulk of the "infrastructure" needed to get song information in the first place, and have built a small database of songs & song data for us to access online. The "moving parts" which we currently have are:
- Song data harvesting--using Echo Nest API to search for, and retrieve, information about a song.
- Database construction/storage - automating Echo Nest searches, and storing information (like tempo, name, artist) in a song_data object which is then preserved (with the Pickle module) for future reference.
- Lyrics harvesting - getting lyrics from the web for a song using requests (a web accessing package) and string parsing (to turn the web output into something which python can "eval").
- Lyrics sentiment analysis - using Pattern to get the overall "mood" of a song's lyrics. The lyrics themselves come from the aforementioned lyrics harvesting.
- Song gathering - getting songs from the "Billboard" top 100 weekly songs. Uses HTML to gather songs from several years of Billboard data, and then collate this information into a list of ("artist","song") tuples which can then be passed on to the database construction.
Key Questions:
We've created something to get, and store, nearly every bit of information we could want about a song. However, we're somewhat at a loss for what to do next.
In particular: we want to use a machine learning approach to prepare our sentiment-classifying algorithm. We've got a couple of major questions as to how this can be addressed:
- What sort of machine learning algorithms and/or packages would be best to use?
- Relatedly, what approach should we take?
- Should we, say, have an input of (tempo, key, lyric_sentiment, etc) and an output of (happiness, anger, etc)?
- Or is it better to just train for a single output: say, 3-4 variables in and 1 out?
-Relevant info: None of us know anything about writing an algorithm for ML. We could try to as a stretch goal, but it would be handy to adapt existing libraries/tools.
- How should we acquire training data?
- We can get very general/vague information from the "Rovi" API - should we just get the moods for a lot of songs, and use some to train and some to test?
- Alternatively, should we try to use human classification to get initial training data?
- (Or, something else?)
Agenda:
1. Begin with a brief overview of our project, and the state we're at right now.
2. Move on to ideating on the ML front. We've addressed most of our pressing concerns vis-a-vis gathering information, songs, etc. in the first place - but, now we're uncertain what to *do* with this data; as such, this seems like a good focus.
3. Depending on which ideas seem promising (and, using a few of our current ideas/possibilities), some "Technical discussion" as to what sorts of structures, algorithms, packages, etc. could be used to make ML ideas work.
After the Code Review
Feedback and Decisions:
We did not get a large amount of feedback on what path we should take in terms of our machine learning choices, leaving us to decide on the path ourselves through independent research. It was agreed upon that having multiple outputs from the ML was superior to having just one, and after careful consideration we have decided to push off making multiple outputs our goal until our base model works.
Reflection
Overall our key questions went unanswered, most likely because we failed to communicate our needs correctly to those that knew enough to answer. Alternatively, there was no one in the group with the in depth knowledge of machine learning required to guide us in our quest. We do not believe this could have been solved with pre-class reading simply because the documentation for machine learning in general is dense and borderline impossible to summarize into a small reading source. Unfortunately, aside from attempting to better communicate our needs in terms of the machine learning guidance, there isn't much we could have done to get more from the review.