In the field of data sonification, the construction of meaning is hampered by the lack of shared perceptual codes derived from common modes of perception, as it happens for the visual register. In this paper, we re-organize knowledge from previous experimental projects to build the foundations of future work in data representation. This experimental investigation aims to identify patterns in the translation process from different sensory modalities. To this end, 80 audiovisual sonifications have been collected and analyzed through phenomenological analysis with the goal of recording sensory correspondences. The resulting cross-sensory design map is a visual synthesis of the analysis, and it has a dual function. In the research domain, it proposes testable hypotheses for a systematic approach to data sonification. In the practice, it offers a space that is based on shared conventions that aim to standardize the actions and the choices of both sonification experts and communication designers.

Audiovisual sonifications. A design map for multisensory integration in data representation

Caiola Valentina;Lenzi Sara;Ricco' Dina
2022-01-01

Abstract

In the field of data sonification, the construction of meaning is hampered by the lack of shared perceptual codes derived from common modes of perception, as it happens for the visual register. In this paper, we re-organize knowledge from previous experimental projects to build the foundations of future work in data representation. This experimental investigation aims to identify patterns in the translation process from different sensory modalities. To this end, 80 audiovisual sonifications have been collected and analyzed through phenomenological analysis with the goal of recording sensory correspondences. The resulting cross-sensory design map is a visual synthesis of the analysis, and it has a dual function. In the research domain, it proposes testable hypotheses for a systematic approach to data sonification. In the practice, it offers a space that is based on shared conventions that aim to standardize the actions and the choices of both sonification experts and communication designers.
Proceedings DRS2022
978-1-91229-457-2
Data sonification
Visualization
Synaesthetic design
Perceptual codes
File in questo prodotto:
File Dimensione Formato  
2022_Caiola_Lenzi_Riccò_Audiovisual sonifications_con ISSN.pdf

accesso aperto

: Publisher’s version
Dimensione 2.07 MB
Formato Adobe PDF
2.07 MB Adobe PDF Visualizza/Apri

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11311/1219003
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus ND
  • ???jsp.display-item.citation.isi??? ND
social impact