Graduate School and Research Center in Digital Sciences

Audio-visual guitar transcription

Paleari, Marco ; Huet, Benoit ; Schutz, Antony ; Slock, Dirk T M

Jamboree 2008, CEUR Workshop By and For KSpace PhD Students, 25 July 2008, Paris, France

Music transcription refers to extraction of a human readable and interpretable description from a recording of a music performance. Automatic music transcription remains, nowadays, a challenging research problemwhen dealing with polyphonic sounds or when removing certain constraints. Some instruments like guitars and violins add ambiguity to the problem as the same note can be played at different positions. When dealing with guitar music tablature are, often, preferred to the usual music score, as they present information in a more accessible way. Here, we address this issue with a system which uses the visual modality to support traditional audio transcription techniques. The system is composed of four modules which have been implemented and evaluated: a system which tracks the position of the fretboard on a video stream, a system which automatically detects the position of the guitar on the first fret to initialize the first system, a system which detects the position of the hand on the guitar, and finally a system which fuses the visual and audio information to extract a tablature. Results show that this kind of multimodal approach can easily disambiguate 89% of notes in a deterministic way.

Document Hal Bibtex

Title:Audio-visual guitar transcription
Department:Data Science
Eurecom ref:2579
Copyright: © EURECOM. Personal use of this material is permitted. The definitive version of this paper was published in Jamboree 2008, CEUR Workshop By and For KSpace PhD Students, 25 July 2008, Paris, France and is available at :
Bibtex: @inproceedings{EURECOM+2579, year = {2008}, title = {{A}udio-visual guitar transcription}, author = {{P}aleari, {M}arco and {H}uet, {B}enoit and {S}chutz, {A}ntony and {S}lock, {D}irk {T} {M}}, booktitle = {{J}amboree 2008, {CEUR} {W}orkshop {B}y and {F}or {KS}pace {P}h{D} {S}tudents, 25 {J}uly 2008, {P}aris, {F}rance}, address = {{P}aris, {FRANCE}}, month = {07}, url = {} }
See also: