Read-only mirror of the MIREX Wiki, snapshot of 29 October 2026. Open this page on the original wiki to edit it or to see newer changes.

2008:Audio Melody Extraction Results

From MIREX Wiki

Introduction[edit]

These are the results for the 2008 running of the Audio Melody Extraction task set. For background information about this task set please refer to the 2008:Audio Melody Extraction page. Special thanks to Jean-Louis Durrieu for doing the vocal/non-vocal split summaries.

General Legend[edit]

Team ID[edit]

PC = P. Cancela
CLLY1 = C. Cao, M. Li, J. Liu, Y. Yan 1
CLLY2 = C. Cao, M. Li, J. Liu, Y. Yan 2
DRD1 = J-L. Durrieu, G. Richard, B. David 1
DRD2 = J-L. Durrieu, G. Richard, B. David 2
RK = M. Ryynänen, A. Klapuri
VR = V. Rao, P. Rao

Table Headings[edit]

Vx Recall = Voicing Detection
Vx False Alm = Voicing False Alarm
Vx d' = Voicing d-prime
Raw pitch = Raw Pitch Accuracy
Raw Chroma = Raw Chroma Accuracy
Overall Acc = Overall Acuuracy

Overall Summary Results[edit]

MIREX 2008 Audio Melody Extraction Overall Summary results - Weighted (by Number of Files) Avg. of all Datasets - All[edit]

Avg. Overall Acc.
clly1 49.8%
clly2 62.1%
drd1 58.6%
drd2 73.2%
pc 76.1%
rk 71.1%
vr 67.1%

download these results as csv

MIREX 2008 Audio Melody Extraction Summary results - MIREX 2008 Dataset - All[edit]

Vx Recall Vx False Alm Vx d' Raw pitch Raw Chroma Overall Acc
drd1 98.7% 68.9% 1.74 85.8% 88.3% 76.0%
rk 90.4% 48.5% 1.34 83.5% 83.8% 75.3%
drd2 96.6% 57.1% 1.65 81.8% 82.6% 75.0%
pc 95.7% 65.4% 1.32 83.9% 84.0% 73.3%
vr 68.3% 21.1% 1.28 88.2% 88.6% 66.7%
clly1 77.6% 43.4% 0.92 54.7% 55.3% 51.4%
clly2 86.1% 67.8% 0.62 54.7% 55.3% 49.7%

download these results as csv

2008 am08 m08 all.png

Download the Excel workbook for MIREX 2008 Dataset - All (NB: it seems that all the songs of this dataset are Vocal ones, there is therefore no vocal/non-vocal separate results).

MIREX 2008 Audio Melody Extraction Summary results - MIREX 2005 Dataset - vocal[edit]

Vx Recall Vx False Alm Vx d' Raw pitch Raw Chroma Overall Acc
pc 83.1% 19.7% 1.81 77.6% 78.1% 76.8%
vr 81.5% 20.5% 1.72 76.1% 79.2% 71.8%
rk 89.5% 45.0% 1.38 74.6% 76.6% 67.6%
drd2 92.6% 51.1% 1.42 75.0% 77.9% 66.7%
clly2 81.5% 46.0% 1.00 70.1% 72.8% 63.7%
clly1 75.1% 37.5% 1.00 70.1% 72.8% 62.7%
drd1 91.7% 63.6% 1.04 54.7% 62.4% 48.0%

download these results as csv

2008 am08 m05 vocal.png

Download the Excel Workbook for MIREX 2005 Dataset - vocal.

MIREX 2008 Audio Melody Extraction Summary results - MIREX 2005 Dataset - nonvocal[edit]

Vx Recall Vx False Alm Vx d' Raw pitch Raw Chroma Overall Acc
drd2 90.4% 39.7% 1.56 67.2% 72.6% 64.1%
drd1 88.1% 44.4% 1.32 60.1% 68.3% 57.7%
pc 80.9% 21.1% 1.68 58.3% 62.4% 56.5%
clly2 71.1% 50.0% 0.56 64.6% 68.3% 55.1%
rk 78.4% 41.5% 1.00 63.4% 68.7% 54.8%
vr 84.7% 40.7% 1.26 56.9% 69.9% 51.5%
clly1 27.8% 50.5% -0.60 64.6% 68.3% 24.1%

download these results as csv

2008 am08 m05 nonvocal.png

Download the Excel Workbook for MIREX 2005 Dataset - nonvocal.

MIREX 2008 Audio Melody Extraction Summary results - MIREX 2005 Dataset - All[edit]

Vx Recall Vx False Alm Vx d' Raw pitch Raw Chroma Overall Acc
pc 83.9% 19.9% 1.83 71.0% 72.8% 69.8%
drd2 92.0% 49.2% 1.42 72.4% 76.2% 66.0%
vr 85.1% 23.8% 1.75 69.7% 76.5% 64.9%
rk 85.6% 44.4% 1.21 71.2% 74.4% 63.5%
clly2 77.8% 46.7% 0.85 68.9% 72.0% 61.4%
drd1 91.9% 60.4% 1.14 57.4% 65.3% 52.2%
clly1 58.2% 42.2% 0.40 68.9% 72.0% 48.9%

download these results as csv

2008 am08 m05 all2.png

Download the Excel Workbook for MIREX 2005 Dataset - All.

MIREX 2008 Audio Melody Extraction Summary results - ADC 2004 Dataset - vocal[edit]

Vx Recall Vx False Alm Vx d' Raw pitch Raw Chroma Overall Acc
pc 91.4% 11.0% 2.60 89.5% 89.8% 88.3%
drd2 92.7% 30.9% 1.95 87.4% 87.6% 83.2%
rk 89.2% 29.3% 1.78 84.0% 84.2% 79.9%
vr 80.2% 20.9% 1.66 81.6% 88.1% 72.0%
drd1 95.4% 59.8% 1.44 77.2% 83.4% 69.6%
clly2 87.3% 53.1% 1.06 71.8% 72.3% 64.9%
clly1 62.5% 32.2% 0.78 71.8% 72.3% 52.3%

download these results as csv

2008 am08 adc04 vocal.png

Download the Excel Workbook for ADC 2004 Dataset - vocal.

MIREX 2008 Audio Melody Extraction Summary results - ADC 2004 Dataset - nonvocal[edit]

Vx Recall Vx False Alm Vx d' Raw pitch Raw Chroma Overall Acc
pc 82.6% 6.9% 2.42 78.5% 80.1% 80.2%
drd2 90.2% 33.2% 1.72 83.3% 84.1% 79.0%
rk 88.4% 18.9% 2.07 80.2% 82.6% 77.2%
clly2 80.2% 39.7% 1.11 80.4% 83.4% 72.7%
vr 85.7% 17.7% 1.99 70.4% 80.8% 67.2%
clly1 45.4% 26.1% 0.53 80.4% 83.4% 47.2%
drd1 91.3% 53.5% 1.27 49.0% 62.5% 44.6%

download these results as csv

2008 am08 adc04 nonvocal.png

Download the Excel Workbook for ADC 2004 Dataset - nonvocal.

MIREX 2008 Audio Melody Extraction Summary results - ADC 2004 Dataset - All[edit]

Vx Recall Vx False Alm Vx d' Raw pitch Raw Chroma Overall Acc
pc 88.0% 9.8% 2.47 85.1% 85.9% 85.1%
drd2 91.7% 31.6% 1.86 85.7% 86.2% 81.5%
rk 88.9% 26.2% 1.86 82.4% 83.5% 78.8%
vr 82.3% 20.0% 1.77 77.1% 85.2% 70.1%
clly2 84.6% 49.1% 1.04 75.3% 76.7% 68.0%
drd1 93.8% 57.9% 1.34 65.9% 75.0% 59.6%
clly1 55.7% 29.7% 0.67 75.3% 76.7% 50.2%

download these results as csv

2008 am08 adc04 all.png

Download the Excel workbook for ADC 2004 Dataset - All.

MIREX 2008 Audio Melody Extraction Runtime Data[edit]

Team ID Data set Machine Run-time(seconds)
clly1 M05 FAST3 56
clly1 ADC04 FAST3 36
clly1 M08 FAST3 49
clly2 M05 FAST3 57
clly2 ADC04 FAST3 39
clly2 M08 FAST3 50
drd1 M05 ALE 3 days
drd1 ADC04 ALE 5 days
drd1 M08 ALE 4 days
drd2 M05 ALE 3294
drd2 ADC04 ALE 2082
drd2 M08 ALE 2318
pc M05 FAST 61470
pc ADC04 FAST 33974
pc M08 FAST 47031
rk M05 ALE 115
rk ADC04 ALE 73
rk M08 ALE 100
vr M05 FAST 200
vr ADC04 FAST 110
vr M08 FAST 184

download these results as csv