Fork me on GitHub

MyMediaLite: KDD Cup 2011

News

MyMediaLite 3.11 has been released.


KDD Cup 2011

Since version 1.0, MyMediaLite supports reading in the KDD Cup 2011 data files, so that you can run existing recommenders on that data set, or implement new ones using the MyMediaLite infrastructure.

To find out more about the challenge, please go the official KDD Cup 2011 website.

Lots of Data

The KDD Cup 2011 datasets are huge: The training set for Track 1 contains about 300 million ratings, while the training set for Track 2 contains about 60 million ratings. We took care to implement the MyMediaLite algorithms and data structures so that they do not waste memory, but we recommend to run MyMediaLite for the KDD Cup 2011 on a computer with at least 8 GB (track 1) or 4 GB (track 2).

Patches that improve memory use and runtime are, as well as other improvements, of course always welcome.

If you run into trouble with Mono, please upgrade to Mono 2.10.2 or later and make sure it is compiled with the option --with-large-head=yes.

The Command-Line Tools

We provide one tool for each track, which we describe in the following.

Track 1

Usage:
MyMediaLite KDD Cup 2011 Track 1 tool

 usage:  KDDCup.exe METHOD [ARGUMENTS] [OPTIONS]

  use '-' for either TRAINING_FILE or TEST_FILE to read the data from STDIN

  methods (plus arguments and their defaults):
   - BiasedMatrixFactorization num_factors=10 bias_reg=0.0001 reg_u=0.015 reg_i=0.015 learn_rate=0.01 num_iter=30 bold_driver=False init_mean=0 init_stdev=0.1
   - MatrixFactorization num_factors=10 regularization=0.015 learn_rate=0.01 num_iter=30 init_mean=0 init_stdev=0.1
   - GlobalAverage
   - SlopeOne
   - BipolarSlopeOne
   - UserAverage
   - ItemAverage
   - UserItemBaseline reg_u=0 reg_i=0
   - ItemKNNCosine k=inf reg_u=10 reg_i=5
   - UserKNNCosine k=inf reg_u=10 reg_i=5
   - ItemKNNPearson k=inf shrinkage=10 reg_u=10 reg_i=5
   - UserKNNPearson k=inf shrinkage=10 reg_u=10 reg_i=5
   - ItemAttributeKNN k=inf reg_u=10 reg_i=5 (needs --item-attributes=FILE)
   - UserAttributeKNN k=inf reg_u=10 reg_i=5 (needs --user-attributes=FILE)
method ARGUMENTS have the form name=value

  general OPTIONS have the form name=value
   - option_file=FILE           read options from FILE (line format KEY: VALUE)
   - random_seed=N              set random seed to N
   - data_dir=DIR               load all files from DIR
   - save_model=FILE            save computed model to FILE
   - load_model=FILE            load model from FILE
   - no_eval=BOOL               do not evaluate
   - prediction_file=FILE       write the predictions to  FILE ('-' for STDOUT)
   - cross_validation=K         perform k-fold crossvalidation on the training data
                                 (ignores the test data)
   - sample_data=BOOL           assume the sample data set instead of the real one
   - track2=BOOL                perform rating prediction on track 2 data
   - good_rating_prob=BOOL      try to predict the probability of a good rating (>= 80)

  options for finding the right number of iterations (MF methods)
   - find_iter=N                give out statistics every N iterations
   - max_iter=N                 perform at most N iterations
   - epsilon=NUM                abort iterations if RMSE is more than best result plus NUM
   - rmse_cutoff=NUM            abort if RMSE is above NUM
   - mae_cutoff=NUM             abort if MAE is above NUM
   - compute_fit=BOOL           display fit on training data every find_iter iterations
  

Track 2

Usage:
MyMediaLite KDD Cup 2011 Track 2 tool

 usage:  KDDTrack2.exe METHOD [ARGUMENTS] [OPTIONS]

  use '-' for either TRAINING_FILE or TEST_FILE to read the data from STDIN

  methods (plus arguments and their defaults):
   - ItemAttributeSVM C=1 Gamma=0.002 (needs --item-attributes=FILE)
   - BPRLinear reg=0.015 num_iter=10 learn_rate=0.05 fast_sampling_memory_limit=1024 init_mean=0 init_stdev=0.1 (needs --item-attributes=FILE)
   - BPRMF num_factors=10 bias_reg=0 reg_u=0.0025 reg_i=0.0025 reg_j=0.00025 num_iter=30 learn_rate=0.05 fast_sampling_memory_limit=1024 init_mean=0 init_stdev=0.1
   - ItemAttributeKNN k=80 (needs --item-attributes=FILE)
   - ItemKNN k=80
   - MostPopular
   - Random
   - UserAttributeKNN k=80 (needs --user-attributes=FILE)
   - UserKNN k=80
   - WeightedItemKNN k=80
   - WeightedUserKNN k=80
   - WRMF num_factors=10 regularization=0.015 c_pos=1 num_iter=30 init_mean=0 init_stdev=0.1
   - Zero
method ARGUMENTS have the form name=value

  general OPTIONS have the form name=value
   - option_file=FILE           read options from FILE (line format KEY: VALUE)
   - random_seed=N              set random seed to N
   - data_dir=DIR               load all files from DIR
   - save_model=FILE            save computed model to FILE
   - load_model=FILE            load model from FILE
   - prediction_file=FILE       write the predictions to  FILE ('-' for STDOUT)
   - sample_data=BOOL           assume the sample data set instead of the real one
   - predict_score=BOOL         predict scores (double precision) instead of 0/1 decisions
   - predict_rated=BOOL         instead of predicting what received a good rating, try to predict what received a rating at all
                                (implies predict_score)

  options for finding the right number of iterations (MF methods)
   - find_iter=N                give out statistics every N iterations
   - max_iter=N                 perform at most N iterations
   - epsilon=NUM                abort iterations if error is more than best result plus NUM
   - err_cutoff=NUM             abort if error is above NUM
  

Item Data

If you want to use the item data (relations between tracks, artists, albums, genres), then you just need to implement the interface IKDDCupRecommender. Its property ItemInfo gives you access to a KDDCupItems object which contains all item data.

Dates and Times

We have currently no support for dates/times in track 1, it should be not difficult to add this, though.

ContactFollow us on Twitter