Investigating the Persuasion Potential of Recommender Systems from a Quality Perspective: an Empirical Study

Recommender Systems (RSs) help users search large amounts of digital contents and services by allowing them to identify the items that are likely to be more attractive or useful. RSs play an important persuasion role, as they can potentially augment the users’ trust towards in an application and orient their decisions or actions towards specific directions. This article explores the persuasiveness of RSs, presenting two vast empirical studies that address a number of research questions. First, we investigate if a design property of RSs, defined by the statistically measured quality of algorithms, is a reliable predictor of their potential for persuasion. This factor is measured in terms of perceived quality, defined by the overall satisfaction, as well as by how users judge the accuracy and novelty of recommendations. For our purposes, we designed an empirical study involving 210 subjects and implemented seven full-sized versions of a commercial RS, each one using the same interface and dataset (a subset of Netflix), but each with a different recommender algorithm. In each experimental configuration we computed the statistical quality (recall and F-measures) and collected data regarding the quality perceived by 30 users. The results show us that algorithmic attributes are less crucial than we might expect in determining the user’s perception of an RS’s quality, and suggest that the user’s judgment and attitude towards a recommender are likely to be more affected by factors related to the user experience. Second, we explore the persuasiveness of RSs in the context of large interactive TV services. We report a study aimed at assessing whether measurable persuasion effects (e.g., changes of shopping behavior) can be achieved through the introduction of a recommender. Our data, collected for more than one year, allow us to conclude that, (1) the adoption of an RS can affect both the lift factor and the conversion rate, determining an increased volume of sales and influencing the user’s decision to actually buy one of the recommended products, (2) the introduction of an RS tends to diversify purchases and orient users towards less obvious choices (the long tail), and (3) the perceived novelty of recommendations is likely to be more influential than their perceived accuracy. Overall, the results of these studies improve our understanding of the persuasion phenomena induced by RSs, and have implications that can be of interest to academic scholars, designers, and adopters of this class of systems.