Learning General Preference Models from Physiological Responses in Video Games: How Complex is it?

An affective preference model can be successfully learnt from pairwise comparison of physiological responses. Several approaches to do this obtain different performances. The higher ranked seem to use non linear models and complex feature selection strategies. We present a comparison of three linear and non linear classification methods, combined with a simple and a complex feature selection strategy (sequential forward selection and a genetic algorithm), on two datasets. We apply a strict crossvalidation framework to test the generalization capability of the models when facing physiological data coming from a new user. We show that, when generalization is the goal, complex non-linear models trained using fancy strategies might easily get trapped by overfitting, while linear ones might be preferable. Although this could be expected, the only way to appreciate it has to pass through proper cross-validation, and this is often forgot when rushing in the “best” performance challenge.