Auditory Interfaces

Stefania Serafin is Professor in Sonic Interaction Design at Aalborg University in Copenhagen.Bill Buxton is a partner researcher at Microsoft Research and adjunct professor of computer science at the University of Toronto.Bill Gaver is Professor of Design and co-director of the Interaction Research Studio at Goldsmiths, University of London. Sara Bly is an independent consultant focused on user practice, particularly in designing technologies tosupport collaboration

• List of Figures List of Tables Preface0.1 Introduction0.2 Overview0.3 The Authors1 Nonspeech audio: an introduction1.1 Introduction1.2 What About Noise?1.3 Figure and Ground in Audio1.4 Sound and the Visually Impaired1.5 Auditory Display Techniques1.6 Some Examples1.7 Sound in Collaborative Work1.8 Function and Signal Type1.8.1 Alarms and Warning Systems1.9 Audio Cues and Learning1.10 Perception and Psychoacoustics1.11 The Logistics of Sound1.12 Summary2 Acoustics and psychoacoustics2.1 Introduction2.2 Acoustics2.2.1 Waveforms2.2.2 Fourier analysis and spectral plots 2.3 More Complex waves 2.3.1 Sound, Obstacles, Bending and Shadows2.3.2 Phase: its Implication on Sound and Representations2.3.3 The Inverse Square Law2.3.4 Helmholtz Revisited2.3.5 Spectrograms2.3.6 Formants vs Partials2.4 Some digital signal processing concepts2.5 Spatial Hearing2.5.1 Head-related transfer functions (HRTF)2.5.2 3D sound distance and reverberation2.6 Psychoacoustics2.6.1 Just Noticeable Difference (JND)2.6.2 Critical Bands2.6.3 Pitch2.6.4 Pitches, Intervals, Scales and Ratios2.6.5 Loudness2.6.6 Duration, Attack Time and Rhythm.2.6.7 Microvariation and Spectral Fusion2.6.8 Timbre2.6.9 Masking2.6.10 Auditory Streaming2.6.11 Sounds with Variations2.6.12 Psychoacoustic Illusions2.7 Perception of 3D sound2.7.1 Precedence / Hass effect2.7.2 Binaural Rendering2.8 Hearing versus listening2.9 Annoying sounds2.10 Pleasant sounds2.11 Embodied sound and music cognition2.12 Conclusions3 Sonification3.1 Introduction3.2 History3.3 Model based sonification3.4 Case Studies3.4.1 Case Study 1: Presenting Information in Sound3.4.2 Case Study 2: Dynamic Representation of Multivariate Time Series Data3.4.3 Case Study 3: Stereophonic and Surface Sound Generation3.4.4 Case Study 4: Auditory Presentation of Experimental Data3.4.5 Case Study 5: Sonification of EEG data3.5 Discussion3.6 Issues3.7 Issues of Data3.7.1 Issues of Sound Parameters3.7.2 Issues of Evaluation3.8 Conclusions4 Earcons4.1 Introduction4.2 Case Studies4.2.1 Case Study 1: Alarms and Warning Systems4.2.2 Alarms as Applied Psychoacoustics4.2.3 Problems With Traditional Alarms and Convergences with Audio Interfaces4.2.4 Case Study 2: Concurrent earcons4.2.5 Case Study 3: Earcons for visually impaired users4.3 Conclusions5 Everyday listening5.1 Introduction5.2 Musical and Everyday Listening5.2.1 Musical and Everyday Listening are Experiences5.3 The Psychology of Everyday Listening5.3.1 Knowledge About Everyday Listening5.4 The Ecological Approach To Perception5.4.1 Developing An Ecological Account Of Listening5.5 What Do We Hear?5.6 The Physics of Sound-Producing Events 5.7 Vibrating Objects 5.7.1 Aerodynamic Sounds 5.7.2 Liquid Sounds 5.7.3 Temporally Complex Events 5.8 Asking People What They Hear 5.9 Attributes of Everyday Listening 5.10 Patterned, Compound, and Hybrid Complex Sounds 5.10.1 Problems and Potentials of the Framework 5.11 How Do We Hear It? 5.12 Analysis and Synthesis of Sounds and Events 5.12.1 Breaking and Bouncing Bottles 5.12.2 Impact Sounds 5.12.3 Material and Length 5.12.4 Internal Friction and Material 5.13 Sound synthesis by physical modelling 5.14 Conclusions 6. Auditory icons 6.1 Introduction 6.2 Advantages of Auditory Icons 6.3 Systems Which Use Auditory Icons 6.3.1 Case Study 1: The SonicFinder: Creating an Auditory Desktop 6.3.2 Case study 2: SoundShark: Sounds in a Large Collaborative Environment 6.3.3 Case study 3: ARKola: Studying the Use of Sound in a Complex System 6.3.4 Case study 4: ShareMon: Background Sounds for Awareness 6.3.5 Case study 5: EAR: Environmental Audio Reminders 6.3.6 Case study 6: Shoogle: Excitatory Multimodal Interaction on Mobile Devices 6.3.7 Summary 6.4 Issues for Auditory Icons 6.4.1 Mapping Sounds to Events 6.4.2 What is Being Mapped to What? 6.4.3 Types of Mapping6.5 The Vocabulary of Auditory Icons6.5.1 Beyond Literal Mappings: Metaphors, Sound-effects, Cliche´s, and Genre Sounds6.6 Annoyance6.7 The Psychoacoustics of Annoying Sounds6.7.1 The Principle of Optimal Complexity6.7.2 Semantic Effects6.7.3 The Tension Between Clarity and Obtrusiveness6.8 Conclusions6.9 What’s Next?7 Sonic Interaction Design7.1 Introduction7.2 Psychology of sonic interactions7.3 Sonic interactions in products7.4 Examples of objects with interesting sounds7.5 Methods in sonic interaction design7.6 Case studies7.6.1 Case study 1: Naturalness influences perceived usability and pleasantness7.6.2 Case study 2: The Ballancer: continuous sonic feedback from a rolling ball7.7 Challenges of evaluation7.8 Conclusions8 Multimodal Interactions8.1 Introduction8.2 Audio-visual Interactions8.3 Embodied interactions8.4 Audio-haptic Interactions8.5 Case study 1: Haptic Wave8.6 Conclusions9 Spatial auditory displays9.1 Introduction9.2 Hearables9.3 Case studies9.3.1 Case study 1: the LISTEN system9.3.2 Case study 2: Soundscape by Microsoft9.3.3 Case study 3: SWAN: a system for wearable audio navigation9.3.4 Case study 4: Superhuman hearing9.4 Conclusions 10 Synthesis and control of auditory icons 10.1 Introduction10.2 Generating and Controlling Sounds10.3 Parameterized Icons10.3.1 Creating Parameterized Auditory Icons10.3.2 Acoustic Information For Events10.3.3 Analysis and Synthesis of Events10.3.4 Impact Sounds10.3.5 Mapping Synthesis Parameters to Source Attributes10.3.6 An Efficient Algorithm for Synthesis10.3.7 Breaking, Bouncing, and Spilling10.3.8 From Impacts To Scraping10.3.9 Machine Sounds10.4 Physics based simulations10.5 Communicating with sound models10.6 Evaluation of sound synthesis methods10.7 Conclusions11 Summary and future research Bibliography Index