Geometric Analysis of Musical Audio Data

Project Summary

In this work, we turn musical audio time series data into shapes for various tasks in music matching and musical structure understanding. 

Themes and Categories
Christopher Tralie
Electrical and Computer Engineering

In particular, we use sliding window representations of chunks of audio to create high dimensional time-ordered point clouds, and we extract information by analyzing the geometry of these clouds.  We have shown, for example, that sequences of these shapes can be used to identify two different versions of the same song, or "cover songs."  We have also shown that both local and global musical properties can be expressed in geometric language.  For instance, hip hop is very "wiggly" while classical music is very "smooth."  Choruses and verses tend to live in distinct clusters connected by paths, and "bridges," or detours to a different musical idea, show up as large loops.

Related People

Related Projects

Nathan Liang (Psychology, Statistics), Sandra Luksic (Philosophy, Political Science),and Alexis Malone (Statistics) spent ten weeks using tools from text and image analytics to understand the evolving representations of women on magazine covers. They worked with a large collection of magazine covers from Duke’s library archive.

Click here to read the Executive Summary

Gabriel Guedes (Math, Global Cultural Studies), Lucian Li (Computer Science, History), and Orgil Batzaya (Math, Computer Science) spent ten weeks using text analytics and interactive mapping tools to understand the geographic spread of 1,482 versions and editions of the Robinson Crusoe story.  They worked with data provided by the Hathi Trust, the University of Florida, and the Internet Archive.

Click here for the Executive Summary

Ashley Murray (Chemistry/Math), Brian Glucksman (Global Cultural Studies), and Michelle Gao (Statistics/Economics) spent ten weeks using sentiment and image analysis to understand semantic shifts in the way American presidents have used the term “poverty,” stretching from the 1930s to the present day.  

In addition to many YouTube videos of presidential speeches, they made use of a large archive of presidential addresses provided by the American Presidency Project.

Click Here for the Executive Summary