Meta-Song evaluation for Chord Recognition
Yizhao Ni, Matt McVicar, Raúl Santos-Rodríguez and Tijl De Bie
In: 12th International Society for Music Information Retreival (late-breaking demo), 24 Oct - 28 Oct 2011, US.

Abstract

We present a new approach to evaluate chord recognition (CR) systems on songs which do not have full annotations. The principle is to use online chord databases to generate high accurate "pseudo annotations" for these songs and compute "pseudo accuracies" of test systems. Statistical models that model the relationship between "pseudo accuracy" and real performance are then applied to estimate test systems' real performance on these songs. This approach tackles the data limitation posed in current CR evaluations, allowing us to carry out extensive analysis on CR systems, such as their generalizations to different genres. In the experiments we applied this method to evaluate three publicly available CR systems, of which the results verified its reliability.

	PDF (The late-breaking demo) - Requires Adobe Acrobat Reader or other PDF viewer.
	PDF (The technique report) - Requires Adobe Acrobat Reader or other PDF viewer.

[Edit]