Designing with Sound
Sound is a pervasive part of our daily experiences and yet rarely considered in design. Here’s how to start incorporating it in your work.
Sound is one of the more commonly overlooked parts of product design and yet—it can have profound impact. Well-placed sounds convey subtle distinctions, evoke emotion, and express information without adding visual clutter. So why isn’t sound seen as a more important factor?
To get a better understanding of how sound can be better utilized, we spoke to Pedro Botsaris, creative director at Antfood, an award-winning creative audio studio that creates original music, sound design, sonic branding, and experimental audio solutions. Their client portfolio includes tech titans like Facebook and Google, consumer products like PlayStation and Beats as well as architectural icons the likes of the Empire State Building, Zaha Hadid, and more. Pedro himself is a versatile composer, a multi-instrumentalist, and music lover. Born and raised in Rio de Janeiro, Pedro now heads up the Amsterdam branch of Antfood and his work has garnered multiple Cannes Lions awards.
There’s a similarity between visual design and sound. Typography is often described in musical terms, both are used as a form of communication and blend multiple layers to create an end result. So then — how would you describe the way sound design differ from visual design?
Hearing is a mechanical stimuli – a visceral sensation that is directly connected to how we feel.
It comes down to how we are stimulated sonically vs. visually. Graphic design is generally associated with ergonomics and functionality. Sound design aims to create a sensation, an impression. Literature immerses you in the story, while in theatre and opera you may feel incredibly touched by the eloquence of an actor or by the voice of a soprano. In a film or animation, the visuals lay out the narrative, but the music score is stimulating your emotional response to the storyline.
“We can perceive emotion in spoken language, but music allows us to experience sadness without actually being sad.”
— Pedro Botsaris, Creative Director at Antfood
Vision and hearing are complementary but essentially different. We design graphics and sounds reflecting how we perceive these two distinct sensory worlds.
That’s interesting. So sound not only enhances but can also change the way we experience things?
Yes, sound shapes how we experience the world in very unique ways. Our hearing, for example, is hardwired to our primitive brain alert system. We understand language and appreciate music through our ears, but the auditory sense is also our primary system responsible for detecting danger. The sensory nervous system is especially sensitive to abrupt changes in the soundscape around us and can be easily manipulated by sound design to be in a state of hyper-vigilance. We have all experienced this response watching horror films or when we jump because we’re scared by a cheeky friend. Other senses, such as vision and touch may be employed to potentialize this effect, but our hearing is the main trigger for such a visceral reaction.
Enjoying this post?
This close relationship with our alert system is directly connected to another very important quality of our hearing – the ability to localize sounds. Differently from our vision, we can hear on a 360° circumference around the head, whereas touch is limited by physical reach and our sense of smell does not have the capacity to pinpoint a source location as our hearing. Our brain relies heavily on our ears to keep us safe and alert, even when we are sleeping.
I’ve never thought of it that way. So our ability to process sound is something we never really turn off. Considering that, how might designers of future tech use that knowledge to guide users?
Understanding how humans react to sound is the first step in creating a successful interaction between us and the things we create. There is sound design in horror films, alarm clocks, ambulances, and mobile phones but we take that for granted because they are so ingrained in our modern life. As we move forward to the future, new tools, gadgets, and experiences will emerge, posing new challenges to sound designers. VR, for example, is an incredibly immersive and exciting way to experience a story. But how do you follow the intended narrative if you can look anywhere, go anywhere? By using subtle sonic cues and stimulating the alert system, we can help guide listeners to follow a storyline, instinctively creating a completely novel experience.
It's clear that sound design is not only complex but also nuanced. Can you elaborate on the different disciplines within the sound design industry?
Designing sound for UI
I always think of Sonic UI as an extension of both the product and the brand - it has both a mechanical purpose as well as the purpose of affirming your brand voice. Making these two halves work together is always the goal. For example, if I’m building a set of UI sounds for an app, instead of jumping straight into making 20 UI sounds, I start with determining the broader sonic palette. Ideally that palette fits into the broader sonic and brand strategy. Then using a hierarchy, I start chipping away at the most important sonic elements.
Creating a fresh beat
We are all individuals. Our story shapes our passions, aspirations, beliefs and, naturally, how we express ourselves to the world around us.
Music is self-expression, your inner state manifested in an external form. If you want your beat to be “fresh” or your music to be original, all you have to do really is to follow your intuition. You already have a very unique perspective on what it means to exist; with practice you should be able to express exactly that, just like in any other language.
Designing sound for video animation
If you imagine a car passing by in front of you, can you imagine what it sounds like? Recreating the sound of objects you’ve heard before is much simpler than designing for something you’ve never heard before, like creating the sound of a lightsaber for the first time, for example.
Because animation and computer graphics offer filmmakers the freedom to create whatever world they desire, most things are lightsabers. When it comes to sound design, you are creating a completely novel interaction between an action or object and a sound. This is both challenging but a lot of fun and incredibly exciting to anyone into synthesizers.
My approach generally consists of a lot of experimentation. First of all, I decide on a direction. For example, I might decide that my palette will consist only of percussive instruments and vocals. I normally break those rules later in the process but putting up boundaries for yourself to start with is a must to any creative process.
The next step is a lot of fun – creating the raw material. It is basically playing around with your instruments and gear while recording yourself. I then go through everything recorded, curating and organizing the sounds I like. With an initial sound library for the project, it is finally time to start actually scoring to picture. This process consists of a lot of trial and error, experimentation and some frustration.
Depending on the complexity of the project, I go through this process a few times.
Experimenting with foley
Foley is the reproduction of everyday sounds for audio effects in movies, videos and other media. Working with foley is an exercise for your ears. You start hearing all the nuances of everyday sounds that you never noticed. Have you ever thought of the hundred different sounds you could create just with a plastic bottle? If not, your first step is to start critically listening to everything.
A good foley designer can detach the sound from the thing. It’s like having a completely different way to process what you hear - where you associate a sound to its quality as opposed to what it means. For example, you hear a tree branch being broken and recognize it as such. A sound designer might think – this sound is perfect for bones being crushed in a fight scene.
All photos are credited to Antfood, learn more about their work here. Enjoyed this article? Read more stories: