There are two main groups of resources:
One is related to content based audio analysis (such as the extraction of Key, tempo, beats, etc.. form audio) and includes the QM Vamp Plugins, an open source C++ SDK and Python bindings for writing your own plugin and host programs, and command line tools to analyse large music collections locally or via the internet.
We also provide an ontology (The Music Ontology) for describing music related information including editorial metadata, musicological data, and content-based features through various extensions, and a web resource DBTune exposing lots of music related data using this framework. You may also be interested in our tutorial about using these resources and technologies.
1) Content-based audio analysis:
- Vamp Plugins are feature extractor plugins for content based analysis of musical audio. Vamp plugins are written using the open source Vamp plugin API, and return structured data resulting from a wrapped algorithm. They require a host application to run such as Sonic-Visualiser, Sonic-Annotator, Vamp-Simple-Host or Audacity. A large number of plugins are available for tasks such as Beat Tracking, Onset detection, Key and Tonality estimation, as well as lower level transformations like Spectrograms, Chromagrams, MFCCs or Wavelet transforms.
Available Plugins: http://vamp-plugins.org/download.html
How to install: http://vamp-plugins.org/download.html#install
- The Vamp Plugin API is a C/C++ plugin API for audio feature extraction. The API is conceptually similar to audio processing plugin APIs such as LADSPA or VST, however Vamp plugins return structured data describing the results of content based analysis as opposed to processed audio. The Vamp plugin API comes with an easy to use C++ SDK. Python bindings are also available using the VamPy wrapper plugin.
- VamPy is a wrapper plugin which provides Python bindings for the Vamp plugin API. VamPy may be installed as a conventional Vamp plugin. Once installed, it enables any Vamp host to use Python scripts as if they were compiled Vamp plugins. VamPy is open source and cross platform. Binaries are available for Linux, Mac, Windows and Solaris. To use VamPy, you have to be familiar with the Vamp C++ API, although in-depth knowledge of C++ is not required.
VamPy Example Plugin:
VamPy Template Plugin:
Tutorial: Getting Started with VamPy:
- Sonic Visualiser is a cross platform application for visualising content-based audio features. It can host Vamp or VamPy feature extractor plugins written in C/C++ or Python.
- Sonic Annotator is a command line batch feature extractor. It hosts Vamp or VamPy plugins and produces results in RDF and CSV formats. It can analyse large collections of audio files on your machine or on a network resource.
Source code: http://sourceforge.net/projects/sv1/
- Vamp Simple Host is part of the Vamp plugin SDK. It is the simplest program just to test (or run) a Vamp plugin and can be useful during development. It is the resource to look at if you are developing your own host program.
Available from source and cross-platform binaries:
- The Music Ontology is a Semantic Web ontology to describe music related information. It also provides the basis for numerous extensions including musicological and content-based features.
- DBTune hosts SPARQL end-points exposing interlinked music related data from Magnatune, Jamendo, The BBC John Peel sessions, Last-FM, MySpace and MusicBrainz .
More information about the above (and related) SPARQL-endpoints can be found here.
- GNAT is a small audio collection tool to associate local audio files with semantic web URIs.
- RDF and Audio Features The Audio Feature Ontology can be used to express content based audio features in RDF format used on the Semantic Web. Sonic Annotator and SAWA can be used to produce annotations in this format.
The ontology is available here (RDF resource):
- SAWA is a web application which ties some of the previously mentioned technologies together. It can be used as reference to familiarise yourself with Content-based Audio Analysis, Vamp Plugins, RDF and the Audio Feature Ontology. SAWA can be used to automatically extract features in batch from a small collection of uploaded audio files using several Vamp plugins at once.
- Tutorial A tutorial presented at the ISMIR conference last year provides lots of useful information about the technologies above, however it's main focus is "Music and the Semantic Web" a.k.a. Linked Data.
Main tutorial page: http://ismir2009.grasstunes.net/
VamPy and Sonic Annotator tutorial:
2) Ontologies and music related data resources: