Skip to main content

Geometric models for musical audio data

Publication ,  Conference
Bendich, P; Gasparovic, E; Harer, J; Tralie, C
Published in: Leibniz International Proceedings in Informatics, LIPIcs
June 1, 2016

We study the geometry of sliding window embeddings of audio features that summarize perceptual information about audio, including its pitch and timbre. These embeddings can be viewed as point clouds in high dimensions, and we add structure to the point clouds using a cover tree with adaptive thresholds based on multi-scale local principal component analysis to automatically assign points to clusters. We connect neighboring clusters in a scaffolding graph, and we use knowledge of stratified space structure to refine our estimates of dimension in each cluster, demonstrating in our music applications that choruses and verses have higher dimensional structure, while transitions between them are lower dimensional. We showcase our technique with an interactive web-based application powered by Javascript and WebGL which plays music synchronized with a principal component analysis embedding of the point cloud down to 3D. We also render the clusters and the scaffolding on top of this projection to visualize the transitions between different sections of the music.

Duke Scholars

Published In

Leibniz International Proceedings in Informatics, LIPIcs

DOI

ISSN

1868-8969

ISBN

9783959770095

Publication Date

June 1, 2016

Volume

51

Start / End Page

65.1 / 65.5

Related Subject Headings

  • 46 Information and computing sciences
 

Citation

APA
Chicago
ICMJE
MLA
NLM
Bendich, P., Gasparovic, E., Harer, J., & Tralie, C. (2016). Geometric models for musical audio data. In Leibniz International Proceedings in Informatics, LIPIcs (Vol. 51, pp. 65.1-65.5). https://doi.org/10.4230/LIPIcs.SoCG.2016.65
Bendich, P., E. Gasparovic, J. Harer, and C. Tralie. “Geometric models for musical audio data.” In Leibniz International Proceedings in Informatics, LIPIcs, 51:65.1-65.5, 2016. https://doi.org/10.4230/LIPIcs.SoCG.2016.65.
Bendich P, Gasparovic E, Harer J, Tralie C. Geometric models for musical audio data. In: Leibniz International Proceedings in Informatics, LIPIcs. 2016. p. 65.1-65.5.
Bendich, P., et al. “Geometric models for musical audio data.” Leibniz International Proceedings in Informatics, LIPIcs, vol. 51, 2016, pp. 65.1-65.5. Scopus, doi:10.4230/LIPIcs.SoCG.2016.65.
Bendich P, Gasparovic E, Harer J, Tralie C. Geometric models for musical audio data. Leibniz International Proceedings in Informatics, LIPIcs. 2016. p. 65.1-65.5.

Published In

Leibniz International Proceedings in Informatics, LIPIcs

DOI

ISSN

1868-8969

ISBN

9783959770095

Publication Date

June 1, 2016

Volume

51

Start / End Page

65.1 / 65.5

Related Subject Headings

  • 46 Information and computing sciences