Robust aggregation of GWAP tracks for local image annotation

The possibility of assigning labels to localized regions in an image enables flexible image retrieval paradigms. However, the process of automatically segmenting and tagging images is notoriously hard, due to the presence of occlusions, noise, challenging illumination conditions, background clutter, etc. For this reason, human computation has recently emerged as a viable alternative when computer vision algorithms fail to provide a satisfactory answer. For example, Games with a purpose (GWAP) represent a powerful crowdsourcing mechanism to collect implicit annotations from human players. In this paper we consider the problem of aggregating the gaming tracks collected by a GWAP we developed to solve challenging instances of image segmentation problems. In particular we consider the existence of malicious players, who might try to fool the rules of the game to achieve higher rewards. The proposed solution can automatically estimate the reliability of human players, thus identifying cheaters. This information is exploited to aggregate the gaming tracks, thus significantly improving the image segmentation result and the quality of local image annotations.