Game Audio

Welcome the first of many articles about video game sound design. Before we dive into this article, I would like to introduce myself better and a small little history of my professional career. My name is Gustav van Rooyen and I own the company Maroon Productions. I have been a sound designer for 5 years now. I studied Entertainment Technology at Tshwane University of Technology. From then one I went into the theatre business where I basically got my first glimpse in sound design. I did a couple of sound designs for productions like Liewe Heksie, Die Rypmaakkamer (The Curing Room) and Fiela se Kind. But I have a hidden talent that only presented itself in early 2015 I figured out on how to program. Especially video game programming. I developed a video game called The Witch of Erast, a small little Android game that is still available today. When the game was almost done, I realized I need music and sound effects. The music was a hassle for me because I am not a composer. Although a know minimal music theory and I have a good ear, I tried something. But it failed. I reached out to a friend of mine, Pieter Smal. He did his master’s degree at Tikkies. He studied video game music composition. But I went on and did the sound design and I love it. In early 2017 I descried to develop my first commercial release game called The Adventures of Sam Carlisle and the Hunt for the Lost Treasure.In 2018 I started doing video game sound design full time on several video games including mobile titles, that according to me, are a bit more difficult but look out for the article about sound design for mobile games.

Back to the article. Sound design and music play an important role in the overall experience of modern videogames. Game audio can help designers to create tension, add emotion, build immersion in the game world, and even solve design problems. However, many of the audio techniques we benefit from today are only possible because of the ingenuity of game composers and sound designers of the past.

The earliest attempts to create games on computers happened in the 1950s when people seemed really into jukeboxes. The games built during this period were made by scientists who tested them on extremely huge computers that took up almost the whole room. People were lining up to go and test… Tic Tac Toe? Wait, that does not have sound, as well as being really boring by today’s standards.

Twenty years on in the 1970s, people are finally making commercial games for people to enjoy. Games are still very simple by our standards, but they have started to employ sound to enhance the experience for players, like in the early arcade classic Pong (1972).

Sounds like it is just a bunch of bleeps and bloops right? Well, there is a little more to it.

With sound capabilities far worse than a modern washing machine, the creators of Pong varied the pitch and duration of the beep to differentiate between the ball bouncing off a wall, being hit by a player’s paddle or when a point was scored. With just three simple bleeps and bloops, the actions on screen were differentiated from each other, adding more tangibility to the events of the game.

Sound design for games seems cool now, right? Well by the 80s, videogames were as popular and complicated as a nice mullet and leg warmer combo. People weren’t just playing games in arcades anymore; the decade saw the rise of home computers that play games on floppy discs as well as consoles like the Nintendo Entertainment System (NES).

There was one challenge bigger than Donkey Kong for game composers of the 80s – memory. Did you know that the average total size of an NES game was 384kb? Last year I got a 4GB thumb drive for free from my dentist. That thumb drive could have stored over ten thousand copies of an average NES game, but all I got was some program called “Teeth.exe” that I am too scared to open. You also need to remember that the 384kb of these game cartridges had to store everything needed for the whole game, which meant that the memory dedicated to music and sounds was only a fraction of that.

To deal with the memory constraints of games at the time, 80s game composers created music using basic, computer-y sounding waveforms. They also could not have too many sounds playing at one time, so their music generally has only four or so different “tracks” for instruments or sound effects to play on. However, it is these very limitations that will create the quintessential “videogame music” sound – let us take a listen.

Check out this excerpt from the 2017 survival/horror game Resident Evil 7: Biohazard (Jump ahead to 10min 57s)

It is dark, scary, atmospheric – it sounds like the score to a scary film. Why? Because by 2017, there is nothing stopping a game composer from hiring an orchestra and using the same techniques that film composers use to scare us in cinemas.

So, what if a composer wanted to make scary music for a game built for the NES? Due to the device’s limitations, you would never be able to match an orchestra’s dynamic range and variety of sounds and textures.

Game composers had to get clever. They had to create songs that gesture in the direction of other scary music that we have heard by using one of the greatest compositional tools available to them – melody.

Listen to a bit of this excerpt from Toccata & Fugue in D minor by composer Johann Sebastian Bach.

A sinister piece of music that has long been associated with the horror genre due to its extensive use in films.

Now check out the intro to this piece from Castlevania II: Simon’s Quest for the NES (1987)

Can you hear the melodic similarities?

The composers of Castlevania II use a melody that invokes our association with the horror genre and blend it with the rock music of their present-day – resulting in a spooky tune that also makes me want to pump my fist in the air every time.

It’s no coincidence that so many themes from 80s game franchises have been reimagined time and time again – the strong melodies that were a necessity of the time are extremely memorable, and have become as iconic as Mario’s overalls. Almost anybody can hum the theme from Super Mario Bros – will you remember that excerpt from Resident Evil 7 in even a week’s time?

By the 90s games were becoming 3D, and thanks to advances in technology, music in games could be made with real instruments. Some composers continued to make the melodic music that we have come to expect from games, but others were starting to get creative with the possibilities. This would eventually lead to the use of “real world music” in the form of in-game radio stations or as a backdrop to the action, as well as the use of live instruments and orchestral music in games.

With the technological limitations of the past lifted, game composers and sound designers could now take the ingenuity which they had used for so long to emulate other forms of music and turn their attention to solving bigger design problems.

One notable example is Foley – the creation of sound effects for media. Games of the past had to settle for electronic noises (like Pong) or generic, multipurpose sampling. Sound designers can now record real-world audio and create libraries of sounds to be used in their game. This technique was used in the game Never Alone (2014). The sound designers travelled to snowy environments to record audio that would then be used in the game. There is a whole blog post about the techniques that they used here.

Another notable staple of modern game audio is the use of dynamic music. Games have become complex and no longer need to be played linearly or have entirely scripted events, meaning that the best game music would need to react to the events in the game, whatever they may be. Perhaps the most well documented use of this process can be found in the game Red Dead Redemption (2010). The music in this game was made so that sections of music, or “stems”, could come in and out of the score based on what was happening on screen. (Jump to 6min 58s)

As we move further into the future, the role of sounds in games will continue to play a huge part in shaping our experiences. So, what can we expect? With virtual reality games growing rapidly, I am excited to see how composers and sound designers tackle the challenges of 3D audio.