Audiovisual sonifications. A design map for multisensory integration in data representation

In the field of data sonification, the construction of meaning is hampered by the lack of shared perceptual codes derived from common modes of perception, as it happens for the visual register. In this paper, we re-organize knowledge from previous experimental projects to build the foundations of future work in data representation. This experimental investigation aims to identify patterns in the translation process from different sensory modalities. To this end, 80 audiovisual sonifications have been collected and analyzed through phenomenological analysis with the goal of recording sensory correspondences. The resulting cross-sensory design map is a visual synthesis of the analysis, and it has a dual function. In the research domain, it proposes testable hypotheses for a systematic approach to data sonification. In the practice, it offers a space that is based on shared conventions that aim to standardize the actions and the choices of both sonification experts and communication designers.