Datasets
Experimental results for some example datasets. The given hyperparameters may not be the optimal ones.Netflix
Results on the
Method | --recommender-options | RMSE | MAE |
---|---|---|---|
UserItemBaseline | reg_u=4.5 reg_i=1.137 num_iter=10 | 0.98261 | 0.76832 |
BiasedMatrixFactorization | num_factors=0 learn_rate=0.005 bias_reg=0.0001 reg=0.035 num_iter=80 | 0.9830 | 0.7710 |
BiasedMatrixFactorization | num_factors=20 learn_rate=0.005 bias_reg=0.0001 reg=0.035 num_iter=40 | 0.9197 | 0.7157 |
BiasedMatrixFactorization | num_factors=50 learn_rate=0.005 bias_reg=0.0001 reg=0.035 num_iter=90 | 0.9175 | 0.7135 |
BiasedMatrixFactorization | num_factors=80 learn_rate=0.005 reg=0.035 num_iter=26 | 0.9169 | 0.7126 |
Results on the
Method | --recommender-options | RMSE | MAE |
---|---|---|---|
GlobalAverage | 1.13092 | 0.95377 | |
UserAverage | 1.06506 | 0.84848 | |
ItemAverage | 1.05326 | 0.85045 | |
UserItemBaseline | reg_u=4.5 reg_i=1.137 num_iter=10 | 0.98013 | 0.76673 |
BiasedMatrixFactorization | num_factors=120 learn_rate=0.005 bias_reg=0.0001 reg_u=0.035 reg_i=0.035 num_iter=50 bold_driver=True | 0.9086 | 0.70775 |
KDD Cup 2011, Track 1
Results on the validation set:
Method | --recommender-options | RMSE | MAE |
---|---|---|---|
GlobalAverage | 38.17409 | 34.5378 | |
UserAverage | 29.30487 | 22.91169 | |
ItemAverage | 34.27364 | 29.41114 | |
UserItemBaseline | reg_u=4.5 reg_i=1.137 num_iter=10 | 27.415 | 21.14358 |
BiasedMatrixFactorization | num_factors=20 bias_reg=0.001 reg_u=1 reg_i=1 learn_rate=0.0005 num_iter=370 bold_driver=True | 22.2001 | 14.01087 |
Yahoo! Music (about 700 million ratings)
Results on the validation set:
Method | --recommender-options | RMSE | MAE |
---|---|---|---|
GlobalAverage | 1.58333 | 1.4266 | |
UserAverage | 1.29421 | 1.03074 | |
ItemAverage | 1.49744 | 1.31978 | |
UserItemBaseline | reg_u=4.5 reg_i=1.137 num_iter=10 | 1.23747 | 0.97561 |
BiasedMatrixFactorization | num_factors=10 bias_reg=0.00001 reg_u=0.01 reg_i=0.01 learn_rate=0.0005 num_iter=100 bold_driver=True | 1.07446 | 0.81445 |
MovieLens 1M
Results for 5-fold cross-validation on the complete dataset:
Method | --recommender-options | RMSE | MAE |
---|---|---|---|
GlobalAverage | 1.117 | 0.934 | |
UserAverage | 1.036 | 0.827 | |
ItemAverage | 0.983 | 0.783 | |
SlopeOne | 0.902 | 0.712 | |
UserItemBaseline | reg_u=25 reg_i=10, num_iter=10 | 0.908 | 0.719 |
ItemKNNPearson | k=80 shrinkage=10 reg_u=25 reg_i=10 | 0.871 | 0.683 |
FactorWiseMatrixFactorization | num_factors=11 shrinkage=115 | 0.860 | 0.673 |
MatrixFactorization | num_factors=10 num_iter=75 reg=0.05 learn_rate=0.005 | 0.857 | 0.675 |
BiasedMatrixFactorization | num_factors=6 bias_reg=0.25 reg_u=0.4 reg_i=1.2 frequency_regularization=true learn_rate=0.03 num_iter=80 bold_driver=true | 0.854 | 0.674 |
BiasedMatrixFactorization | num_factors=20 bias_reg=0.25 reg_u=0.4 reg_i=1.2 frequency_regularization=true learn_rate=0.03 num_iter=80 bold_driver=true | 0.852 | 0.672 |
BiasedMatrixFactorization | num_factors=40 bias_reg=0.001 regularization=0.060 learn_rate=0.07 num_iter=110 bold_driver=true | 0.855 | 0.676 |
BiasedMatrixFactorization | num_factors=60 bias_reg=0.001 regularization=0.060 learn_rate=0.07 num_iter=100 bold_driver=true | 0.854 | 0.676 |
BiasedMatrixFactorization | num_factors=80 bias_reg=0.001 regularization=0.060 learn_rate=0.07 num_iter=100 bold_driver=true | 0.854 | 0.676 |
BiasedMatrixFactorization | num_factors=120 bias_reg=0.001 regularization=0.055 learn_rate=0.07 num_iter=100 bold_driver=true | 0.854 | 0.676 |
SVDPlusPlus | num_factors=10 num_iter=80 reg=0.05 learn_rate=0.005 | 0.852 | 0.668 |
SVDPlusPlus | num_factors=20 num_iter=80 reg=0.05 learn_rate=0.005 | 0.851 | 0.668 |
MovieLens 100k
5-fold crossvalidation with --random-seed=1
Method | --recommender-options | RMSE | MAE |
---|---|---|---|
BipolarSlopeOne | 0.96754 | 0.74462 | |
UserItemBaseline | reg_u=5 reg_i=2 | 0.94192 | 0.74503 |
SlopeOne | 0.93978 | 0.74038 | |
UserKNNCosine | k=40 reg_u=12 reg_i=1 | 0.937 | 0.737 |
UserKNNPearson | k=60 shrinkage=25 reg_u=12 reg_i=1 | 0.92971 | 0.72805 |
ItemKNNCosine | k=40 reg_u=12 reg_i=1 | 0.924 | 0.727 |
FactorWiseMatrixFactorization | num_factors=5 num_iter=5 shrinkage=150 | 0.9212 | 0.7252 |
BiasedMatrixFactorization | num_factors=5 bias_reg=0.1 reg_u=0.1 reg_i=0.1 learn_rate=0.07 num_iter=100 bold_driver=true | 0.91678 | 0.72289 |
BiasedMatrixFactorization | num_factors=10 bias_reg=0.1 reg_u=0.1 reg_i=0.12 learn_rate=0.07 num_iter=100 bold_driver=true | 0.91496 | 0.72209 |
SVDPlusPlus | num_factors=4 regularization=0.1 bias_reg=0.005 learn_rate=0.01 bias_learn_rate=0.007 num_iter=50 | 0.9138 | 0.71836 |
ItemKNNPearson | k=40 shrinkage=2500 reg_u=12 reg_i=1 | 0.91327 | 0.7144 |
BiasedMatrixFactorization | num_factors=40 bias_reg=0.1 reg_u=1.0 reg_i=1.2 learn_rate=0.07 num_iter=100 frequency_regularization=true bold_driver=true | 0.90764 | 0.71722 |
BiasedMatrixFactorization | num_factors=80 bias_reg=0.003 reg_u=0.09 reg_i=0.1 learn_rate=0.07 num_iter=100 bold_driver=true | 0.91153 | 0.72013 |
SVDPlusPlus | num_factors=10 regularization=0.1 bias_reg=0.005 learn_rate=0.01 bias_learn_rate=0.007 num_iter=50 | 0.91096 | 0.7152 |
BiasedMatrixFactorization | num_factors=320 bias_reg=0.007 reg_u=0.1 reg_i=0.1 learn_rate=0.07 num_iter=500 bold_driver=true | 0.91073 | 0.72053 |
BiasedMatrixFactorization | num_factors=160 bias_reg=0.003 reg_u=0.08 reg_i=0.1 learn_rate=0.07 num_iter=100 bold_driver=true | 0.91047 | 0.71944 |
SigmoidItemAsymmetricFactorModel | num_factors=5 regularization=0.005 bias_reg=0.1 learn_rate=0.006 bias_learn_rate=0.7 num_iter=65 | 0.91 | 0.71701 |
SVDPlusPlus | num_factors=4 regularization=1 bias_reg=0.05 learn_rate=0.01 bias_learn_rate=0.07 num_iter=50 frequency_regularization=true | 0.90906 | 0.71547 |
SigmoidItemAsymmetricFactorModel | num_factors=10 regularization=0.005 bias_reg=0.1 learn_rate=0.006 bias_learn_rate=0.7 num_iter=90 | 0.9086 | 0.71522 |
SVDPlusPlus | num_factors=20 regularization=0.1 bias_reg=0.005 learn_rate=0.01 bias_learn_rate=0.007 num_iter=50 | 0.90829 | 0.713 |
SVDPlusPlus | num_factors=20 regularization=1 bias_reg=0.005 learn_rate=0.01 bias_learn_rate=0.07 num_iter=50 frequency_regularization=true | 0.90783 | 0.71413 |
SVDPlusPlus | num_factors=50 regularization=1 bias_reg=0.005 learn_rate=0.01 bias_learn_rate=0.07 num_iter=50 frequency_regularization=true | 0.90651 | 0.71352 |
SigmoidUserAsymmetricFactorModel | num_factors=5 regularization=0.003 bias_reg=0.01 learn_rate=0.006 bias_learn_rate=0.7 num_iter=70 | 0.89062 | 0.69995 |
Epinions
5-fold crossvalidation with --random-seed=1
Method | --recommender-options | RMSE |
---|---|---|
GlobalAverage | 1.20684 | |
UserAverage | 1.19884 | |
ItemAverage | 1.09421 | |
UserItemBaseline | 1.04722 | |
BiasedMatrixFactorization | num_iter=30 num_factors=5 learn_rate=0.01 reg=3.5 | 1.04265 |
Flixster
5-fold crossvalidation with --random-seed=1
Method | --recommender-options | RMSE | MAE |
---|---|---|---|
GlobalAverage | 1.092 | 0.871 | |
UserAverage | 1.032 | 0.719 | |
ItemAverage | 1.097 | 0.852 | |
UserItemBaseline | reg_u=15 reg_i=10 | 0.904 | 0.685 |
BiasedMatrixFactorization | num_factors=5 bias_reg=0.0001 regularization=0.03 learn_rate=0.051 num_iter=50 bold_driver=True | 0.851 | 0.633 |
BiasedMatrixFactorization | num_factors=10 bias_reg=0.0001 regularization=0.015 learn_rate=0.051 num_iter=50 bold_driver=True | 0.845 | 0.625 |