Song Similarity Subtask[edit]

Description[edit]

This subtask deals with targeted recommendation. It deals with tag-based radio, audio search, and prestige-ranked browsing (say by genre).

Targeted Search without Personalization[edit]

Source Data[edit]

• Audio Data
• Song Tags
• Artist Tags

Query[edit]

Hand-picked playlist of songs describing a particular constraint

Ground Truth[edit]

• Option 1: set of hand-picked ordered playlists
• Option 2: ranked set of songs by play list co-occurence

Execution[edit]

Algorithms recieve the source data and the query playlist. Output is then matched against the ground truth.

Evaluation[edit]

Evaluate each output against ground truth and combine results.

• Option 1:
  • Pearsons Correlation (order does not matter) against all play-lists in ground truth, best score is reported
  • Kendall Tau (order matters) same as above
• Option 2: Numeric output (strength of a recommendation)
  • Mean error
  • Recommendation error
• Option 2: Ordered sets
  • ROC area
  • Kendall tau
• Option 2: Unordered sets
  • Pearson correlation
  • F-Measure

Targeted Search with Personalization[edit]

This one would be really neat to run, but would require some innovative ground truth...

• user profiles and user playcounts would be added as source data
• ground truth playlists on a topic are hand picked from a group of playlists submitted by a LastFM user
• evaluation is the same as above

Targeted Search for New Music[edit]

Both of the above could remove a random sample of social data, re-run the evaluation using playlists of only removed music, using cross-validation