Fork me on GitHub

MyMediaLite: Example Experiments

News

MyMediaLite 3.11 has been released.


Datasets

Experimental results for some example datasets. The given hyperparameters may not be the optimal ones.

Netflix

Results on the probe dataset:
Method --recommender-options RMSE MAE
UserItemBaseline reg_u=4.5 reg_i=1.137 num_iter=10 0.98261 0.76832
BiasedMatrixFactorization num_factors=0 learn_rate=0.005 bias_reg=0.0001 reg=0.035 num_iter=80 0.9830 0.7710
BiasedMatrixFactorization num_factors=20 learn_rate=0.005 bias_reg=0.0001 reg=0.035 num_iter=40 0.9197 0.7157
BiasedMatrixFactorization num_factors=50 learn_rate=0.005 bias_reg=0.0001 reg=0.035 num_iter=90 0.9175 0.7135
BiasedMatrixFactorization num_factors=80 learn_rate=0.005 reg=0.035 num_iter=26 0.9169 0.7126

Results on the quiz dataset:
Method --recommender-options RMSE MAE
GlobalAverage 1.13092 0.95377
UserAverage 1.06506 0.84848
ItemAverage 1.05326 0.85045
UserItemBaseline reg_u=4.5 reg_i=1.137 num_iter=10 0.98013 0.76673
BiasedMatrixFactorization num_factors=120 learn_rate=0.005 bias_reg=0.0001 reg_u=0.035 reg_i=0.035 num_iter=50 bold_driver=True 0.9086 0.70775

KDD Cup 2011, Track 1

Results on the validation set:
Method --recommender-options RMSE MAE
GlobalAverage 38.17409 34.5378
UserAverage 29.30487 22.91169
ItemAverage 34.27364 29.41114
UserItemBaseline reg_u=4.5 reg_i=1.137 num_iter=10 27.415 21.14358
BiasedMatrixFactorization num_factors=20 bias_reg=0.001 reg_u=1 reg_i=1 learn_rate=0.0005 num_iter=370 bold_driver=True 22.2001 14.01087

Yahoo! Music (about 700 million ratings)

Results on the validation set:
Method --recommender-options RMSE MAE
GlobalAverage 1.58333 1.4266
UserAverage 1.29421 1.03074
ItemAverage 1.49744 1.31978
UserItemBaseline reg_u=4.5 reg_i=1.137 num_iter=10 1.23747 0.97561
BiasedMatrixFactorization num_factors=10 bias_reg=0.00001 reg_u=0.01 reg_i=0.01 learn_rate=0.0005 num_iter=100 bold_driver=True 1.07446 0.81445

MovieLens 1M

Results for 5-fold cross-validation on the complete dataset:
Method --recommender-options RMSE MAE
GlobalAverage 1.117 0.934
UserAverage 1.036 0.827
ItemAverage 0.983 0.783
SlopeOne 0.902 0.712
UserItemBaseline reg_u=25 reg_i=10, num_iter=10 0.908 0.719
ItemKNNPearson k=80 shrinkage=10 reg_u=25 reg_i=10 0.871 0.683
FactorWiseMatrixFactorization num_factors=11 shrinkage=115 0.860 0.673
MatrixFactorization num_factors=10 num_iter=75 reg=0.05 learn_rate=0.005 0.857 0.675
BiasedMatrixFactorization num_factors=6 bias_reg=0.25 reg_u=0.4 reg_i=1.2 frequency_regularization=true learn_rate=0.03 num_iter=80 bold_driver=true 0.854 0.674
BiasedMatrixFactorization num_factors=20 bias_reg=0.25 reg_u=0.4 reg_i=1.2 frequency_regularization=true learn_rate=0.03 num_iter=80 bold_driver=true 0.852 0.672
BiasedMatrixFactorization num_factors=40 bias_reg=0.001 regularization=0.060 learn_rate=0.07 num_iter=110 bold_driver=true 0.855 0.676
BiasedMatrixFactorization num_factors=60 bias_reg=0.001 regularization=0.060 learn_rate=0.07 num_iter=100 bold_driver=true 0.854 0.676
BiasedMatrixFactorization num_factors=80 bias_reg=0.001 regularization=0.060 learn_rate=0.07 num_iter=100 bold_driver=true 0.854 0.676
BiasedMatrixFactorization num_factors=120 bias_reg=0.001 regularization=0.055 learn_rate=0.07 num_iter=100 bold_driver=true 0.854 0.676
SVDPlusPlus num_factors=10 num_iter=80 reg=0.05 learn_rate=0.005 0.852 0.668
SVDPlusPlus num_factors=20 num_iter=80 reg=0.05 learn_rate=0.005 0.851 0.668

MovieLens 100k

5-fold crossvalidation with --random-seed=1
Method --recommender-options RMSE MAE
BipolarSlopeOne 0.96754 0.74462
UserItemBaseline reg_u=5 reg_i=2 0.94192 0.74503
SlopeOne 0.93978 0.74038
UserKNNCosine k=40 reg_u=12 reg_i=1 0.937 0.737
UserKNNPearson k=60 shrinkage=25 reg_u=12 reg_i=1 0.92971 0.72805
ItemKNNCosine k=40 reg_u=12 reg_i=1 0.924 0.727
FactorWiseMatrixFactorization num_factors=5 num_iter=5 shrinkage=150 0.9212 0.7252
BiasedMatrixFactorization num_factors=5 bias_reg=0.1 reg_u=0.1 reg_i=0.1 learn_rate=0.07 num_iter=100 bold_driver=true 0.91678 0.72289
BiasedMatrixFactorization num_factors=10 bias_reg=0.1 reg_u=0.1 reg_i=0.12 learn_rate=0.07 num_iter=100 bold_driver=true 0.91496 0.72209
SVDPlusPlus num_factors=4 regularization=0.1 bias_reg=0.005 learn_rate=0.01 bias_learn_rate=0.007 num_iter=50 0.9138 0.71836
ItemKNNPearson k=40 shrinkage=2500 reg_u=12 reg_i=1 0.91327 0.7144
BiasedMatrixFactorization num_factors=40 bias_reg=0.1 reg_u=1.0 reg_i=1.2 learn_rate=0.07 num_iter=100 frequency_regularization=true bold_driver=true 0.90764 0.71722
BiasedMatrixFactorization num_factors=80 bias_reg=0.003 reg_u=0.09 reg_i=0.1 learn_rate=0.07 num_iter=100 bold_driver=true 0.91153 0.72013
SVDPlusPlus num_factors=10 regularization=0.1 bias_reg=0.005 learn_rate=0.01 bias_learn_rate=0.007 num_iter=50 0.91096 0.7152
BiasedMatrixFactorization num_factors=320 bias_reg=0.007 reg_u=0.1 reg_i=0.1 learn_rate=0.07 num_iter=500 bold_driver=true 0.91073 0.72053
BiasedMatrixFactorization num_factors=160 bias_reg=0.003 reg_u=0.08 reg_i=0.1 learn_rate=0.07 num_iter=100 bold_driver=true 0.91047 0.71944
SigmoidItemAsymmetricFactorModel num_factors=5 regularization=0.005 bias_reg=0.1 learn_rate=0.006 bias_learn_rate=0.7 num_iter=65 0.91 0.71701
SVDPlusPlus num_factors=4 regularization=1 bias_reg=0.05 learn_rate=0.01 bias_learn_rate=0.07 num_iter=50 frequency_regularization=true 0.90906 0.71547
SigmoidItemAsymmetricFactorModel num_factors=10 regularization=0.005 bias_reg=0.1 learn_rate=0.006 bias_learn_rate=0.7 num_iter=90 0.9086 0.71522
SVDPlusPlus num_factors=20 regularization=0.1 bias_reg=0.005 learn_rate=0.01 bias_learn_rate=0.007 num_iter=50 0.90829 0.713
SVDPlusPlus num_factors=20 regularization=1 bias_reg=0.005 learn_rate=0.01 bias_learn_rate=0.07 num_iter=50 frequency_regularization=true 0.90783 0.71413
SVDPlusPlus num_factors=50 regularization=1 bias_reg=0.005 learn_rate=0.01 bias_learn_rate=0.07 num_iter=50 frequency_regularization=true 0.90651 0.71352
SigmoidUserAsymmetricFactorModel num_factors=5 regularization=0.003 bias_reg=0.01 learn_rate=0.006 bias_learn_rate=0.7 num_iter=70 0.89062 0.69995

Epinions

5-fold crossvalidation with --random-seed=1
Method --recommender-options RMSE
GlobalAverage 1.20684
UserAverage 1.19884
ItemAverage 1.09421
UserItemBaseline 1.04722
BiasedMatrixFactorization num_iter=30 num_factors=5 learn_rate=0.01 reg=3.5 1.04265

Flixster

5-fold crossvalidation with --random-seed=1
Method --recommender-options RMSE MAE
GlobalAverage 1.092 0.871
UserAverage 1.032 0.719
ItemAverage 1.097 0.852
UserItemBaseline reg_u=15 reg_i=10 0.904 0.685
BiasedMatrixFactorization num_factors=5 bias_reg=0.0001 regularization=0.03 learn_rate=0.051 num_iter=50 bold_driver=True 0.851 0.633
BiasedMatrixFactorization num_factors=10 bias_reg=0.0001 regularization=0.015 learn_rate=0.051 num_iter=50 bold_driver=True 0.845 0.625

ContactFollow us on Twitter