Geometric Analysis of Musical Audio Data

Project Summary

In this work, we turn musical audio time series data into shapes for various tasks in music matching and musical structure understanding. 

Themes and Categories
Year
Contact
Christopher Tralie
Electrical and Computer Engineering
chris.tralie@gmail.com

In particular, we use sliding window representations of chunks of audio to create high dimensional time-ordered point clouds, and we extract information by analyzing the geometry of these clouds.  We have shown, for example, that sequences of these shapes can be used to identify two different versions of the same song, or "cover songs."  We have also shown that both local and global musical properties can be expressed in geometric language.  For instance, hip hop is very "wiggly" while classical music is very "smooth."  Choruses and verses tend to live in distinct clusters connected by paths, and "bridges," or detours to a different musical idea, show up as large loops.

Related People

Related Projects

Annie Xu (Rice, CEE), Liuren Yin (ECE), and Zoe Zhu (Data Science) spent ten weeks analysing usage data for MorphoSource, a publicly available 3D data repository maintained by Duke University. Working with Python and Tableau, the team developed an interactive dashboard that allows MorphoSource staff to explore usage patterns for site visitors who view 3D files representing objects from primate skulls to historical art pieces.

 

View the team's project poster here

Watch the team's final presentation on Zoom here:

 

Project Leads: Doug Boyer, Julia Winchester

After London was destroyed during the Great Fire of 1666, it was reconstructed into the “emerald gem of Europe,” a utopian epicenter focused on England’s political and economic interests. For whom was the utopia constructed? Who determined its architectural choices? And what did such a utopia look like in seventeenth-century London?


Our research uses Natural Language Processing to analyze semantic trends in digitized text from the online database “Early English Books Online” (EEBO-TCP https://textcreationpartnership.org/tcp-texts/eebo-tcp-early-english-books-online/) to answer such questions. After applying methods such as word-embedding, sentiment analysis, and hapax richness, we provide an overview of themes in the seventeenth century; specifically, we conducted case studies on changes to coal taxes within the period and the reconstruction of St Paul's Cathedral. Our results thus show that, while a utopian society was originally intended to be built for the people, the project’s motivation eventually shifted to a political purpose, as evidenced by the approval of more costly city projects. In response to backlash against the increase of taxes on coal to support large-scale building projects, the ruling class highlighted positive outcomes in printed materials in order to convince working class persons that their collected taxes contributed to a greater good, despite evidence to the contrary. Finally, during key historical events, sentiment and hapax richness are shown to have an inverse relationship, the results of which can demonstrate how London writers engaged with text and genre as forms of protest.

View the team's project poster here

Watch the team's final project presentation on Zoom:

 

Is there a right type and amount of consumption? The idea of ethical consumption has gained prominence in recent discourse, both in terms of what we purchase (from fair trade coffee to carbon off-sets) and how much we consume (from rechargeable batteries to energy efficient homes).