Hearing is the New Seeing

Hearing Is The New Seeing

"ICAD 2018 shows that as products become more automated, auditory displays help make our interactions more seamless and simple."

With the increase of home automation products, autonomous vehicles, and new products like the space station assistant, our products keep getting smarter. As a result, product designers face a unique challenge in balancing what appears to be a paradox: our products perform more but they need our attention less. How should products communicate more information to us, when we’re looking at and engaging with them less? One answer is by using auditory displays.

From Siri’s confirmation chimes to the artificial engine sounds added to electric vehicles, auditory displays are sounds that convey information to us – machine status, warnings, navigational cues, alerts, alarms, or nearly any other data you can imagine. Auditory displays leverage the advantages of aural perception to help make our product interactions more seamless, intuitive and emotive.

At ICAD 2018 (International Conference for Auditory Display), the atmosphere was charged with innovative ideas and collaboration around how best to design our auditory displays to this rapidly evolving world. Sound designers, musicians, researchers, scientists, programmers, and engineers met to share their latest research and explorations in auditory displays. The work was diverse and spanned the medical, automotive, consumer, architectural, and gaming industries.

Product developers can learn from academic studies that seek to redefine our product experiences. We’ll recap several emerging trends from the conference: Passive communicationsensory substitution, and expressive sounds.

Passive Communication

Autonomous driving may offer a plethora of advantages to our lives, but it also can be frightening to consider that we’re paying less attention to our large and dangerous vehicles. In many aspects of autonomous vehicles, auditory displays are serving as a vital link to keep us connected.

As Carryl Baldwin presented in her keynote speech, “Attention Management in Highly Automated Systems”, partially autonomous vehicles feature an alarm sound to indicate when the autonomous system faulters. According to Baldwin, these sounds are not up to par. The sounds are critical – the difference between a full speed collision and a softened one – and therefore demand a cautious, thorough approach to sound design, to ensure the sounds communicate effectively. Baldwin outlined several alarm standards and parameters that automotive vehicles and sound designers must meet to ensure alarm sounds are effective at communication when our attention is elsewhere.

Imagine riding in your self-driving car to work. As you are sipping your morning coffee, your vehicle hits a massive pothole or quickly swerves to avoid a collision. Or perhaps it elects to take an alternate route to avoid traffic. In any of these scenarios, autonomous vehicles must communicate to us to maintain our trust. In the workshop “Sonic Interaction in Intelligent Vehicles”, participants were asked to consider the ideal sonic experience in fully autonomous vehicles. We discussed issues and scenarios like these and devised novel acoustic experiences for passengers – i.e. abstracted acoustic representations of road conditions; continuous, modulating soundscapes that gently probe our attention. One thing was clear: auditory displays have a future and need in the automotive space, and a holistic design approach is necessary to craft the appropriate acoustic experience.

Within the medical industry, sounds are providing non-visual communication in surprising new ways. As Andrea Lorena Aldana Blanco mentions in her paper, “CardioSounds: A Portable System to Sonify ECG Rhythm Disturbances In Real Time,” aural perception is highly effective at recognizing changes and patterns. Blanco’s paper presents a new innovative sonification system for improving the audible monitoring of ECG data. CardioSounds uses more information within the ECG signal than traditional monitors to build a dynamic, musical soundscape. This auditory display sonically presents arrhythmia data to clinicians and patients, to let them monitor it even when they’re focused on other tasks. The ability to monitor biofeedback while attention is focused elsewhere will continue to be vital for medical, automotive, and other environments.

Sensory Substitution

Auditory displays can be a useful avenue as a sensory substitution technique, especially for visually impaired users. Sensory substitution uses a sense to access information that is typically available from a different sense. Braille, for example, substitutes touch for typically visual information. ICAD 2018 featured several presenters discussing new auditory displays that help visually impaired users hear what is typically seen – like photographs, maps, data sets, and more. With a growing rate of the population with some or all visual impairment, the need for sensory substitution is only increasing.

Rodrigo Cadiz presented his work, Audiograph, an auditory display plug-in for Microsoft Excel, that plays sounds to represent a section of selected data in excel. This simple sonification of visual data brings life, expression, and movement to otherwise inaccessible information to visually impaired users.

Another presenter, Tim Ziemer, created an auditory navigational system which helps users blindly pinpoint a location in two-dimensional space. The psychoacoustic auditory display guides users with changing pitch and changing beating or roughness to guide them towards a small target. His work has been conducted in Germany at the University of Bremen’s Spatial Cognition Center, and has potential applications in the surgical environment, or practically anywhere users need assistance navigating 2D and 3D spaces.

Expressive Sounds

Hearing certain melodies, chords, or rhythms can have a large emotional impact on us. Couple that with the meaning and information sound can carry, and you have a powerful combination. The ideal auditory displays play to both sides of sound’s potential: to convey information with expression.

The automotive industry is one of the first to invest in emotive sounds. Volvo sound designer Fredrick Hagman spoke to me about the emotive nuances of designing a sound for the new series of Volvo vehicles. He wanted to capture the organic, warm qualities of the Volvo brand, and imbue them into one of the most commonly heard sounds in the vehicle: the turn indicator. His resulting sound was based on the sound of a branch snapping — a sort of “wooden” click. This subtle branding reinforcement helps build connections to users, and strengthen the product experience.

With industries like automotive leading the way, musicians and sound designers are needed to better craft these branded, emotive experiences of auditory displays. Those with the musical experience of using pitch, melody, rhythm, and timbre to effectively elicit emotional reactions are well suited to elevate auditory displays to their most impactful potential.

What's Next?

Music, film, advertising, and theater have long used sound design to captivate users. But product manufacturers are catching up. Since sounds have unique power to quickly grab users’ attention, seamlessly deliver information, and elicit emotion, better product sounds is making our products an even more integral part of our lifestyle.

As ICAD 2018 showed, there is deep exploration of auditory displays in the academic space. But, there is ample opportunity and potential for the medical, automotive, industrial, consumer, and other industries to utilize sound design teams to improve their products’ acoustic experiences.

Designing acoustic product experiences requires a diverse team. Specialists in human factors, research, psychology, sound design, product design, and engineering need to come together to design, test, prototype and validate appropriate sounds for the end user. The outcome of such a process is an environment where users experience sounds that are non-intrusive, simple, and meaningful.

About The Author

Whether he’s composing music or designing products, Eddie loves seeking opportunities within limitations, and is passionate about all things design. As one of the resident musicians at Priority Designs, Eddie also helps lead our product sound design projects, understanding how to bridge the gap between product usability, branding, and musical expression.

  • SHARE:

LET'S GET STARTED.

Get in touch to see how we can help you.

, ,