Game designers should think like musicians

or

WHAT CAN MUSIC COMPOSITION TEACH US ABOUT GAME DESIGN?


As an independent musician working in animation/design, I often think about the relationship between the different art forms in terms of the creative process. There are an incredible amount of similarities across disciplines, and as I learn more and more about game design with each semester at the Entertainment Technology Center, I notice some overarching patterns emerge. Especially after taking the Building Virtual Worlds course, there are a number of transferable concepts from popular song compositions that I think are worth exploring for their insights into creation in general, and specifically towards making video games. Hopefully through exploration I can better synthesize the two disciplines as they are so innately connected (especially in the digital streaming era of the music industry and with the influence of video game music in popular genres like EDM and hip hop). In doing so, I attempt to analyze and extrapolate learnings to better understand how to score video games, to better understand how to make video games, and to perhaps lay down a framework around designing games with music in mind from the beginning to optimize the game experience for all players.

Iteration

I remember my very first composition made by programming notes into my Sony Ericsson cellphone on the train to class in grade 7. It was the Super Mario theme song and got me looking for better tools to try producing songs. I was an ambitious child, trying to compose pieces and write songs on the piano, guitar, and drums without having much experience nor talent in playing, but I realized early on that I wanted to create. The first thing I noticed when composing for instruments was that all of my compositions seemed to be lacking in comparison to the songs I was trying to emulate on the radio, and that musical notation and nomenclature acted as a barrier to putting my ideas into musical form. Not only did I lack enough knowledge of music theory, I lacked the skill required to play multiple instruments (one instrument never sounded complete enough), and most importantly, I lacked the ability to listen to sections of my composition because my playing skills were far behind my vision. The cell phone introduced me to a new concept - digital programming allowed instant replay of the sections, allowing me to make my ideas tangible and therefore iterate on the idea simply by listening and comparing the result to what I had in mind. This holds true across any discipline, and is especially important in video game development with its focus on iteration. The sooner one can directly put their ideas into form, the sooner one can tackle problems crop up you could never have anticipated without going through the experience once yourself. The sooner you have something tangible, the sooner other people can listen to your song or playtest your game, and the quicker you can improve your product based on their feedback.

Engine

Later that year, I found a computer program designed for creating songs with realistic instrument sounds and textures (FL Studio). I was obsessed. Every day I would think about lyrics I could write with interesting rhyme structures, and melodies I could add into the beat I was producing. The interface allowed me to learn more about the multiple processes and disciplines involved in music production like recording, mixing, and mastering, and allowed me to use interfaces I had previously used (piano keys and drum pads as compared with using the mouse to program notes). This reminds me of my experience designing VR/AR/XR video games last semester - Unity is a similar multidisciplinary engine with many processes built in for artists and programmers alike, and tools like Tilt Brush allow me to use interfaces I feel familiar with (like sculpting or painting) to create assets for Unity. As my music career grew, I started working with others to help create songs and through an amateur experience with a professional music mastering engineer, I realized the importance of file management when collaborating across artistic and technical disciplines.

Interface

Playing an instrument itself is akin to playing games - one can think of instruments as the controller of the ‘game’ of music production. There are specific ‘buttons’ to press in different locations that have different effects on the composition (whether they are variations in pitch, tone, and/or timbre), and you often use or abuse knowledge of theoretical frameworks (popular tropes, musical theory, or game design principles/best practices) to create micro and macro patterns that feel or sound engaging. In my time as a drummer, I’ve learned that the simple snare has infinite potential and all it takes is understanding the instrument innately to be able to play expressively, even if just on a triangle. Some of the best disco in my opinion utilize amazing triangle percussionists. It’s all about getting into the flow - if there is a strong flow state that your gameplay allows through its pacing, the experience will be that much more addictive. It’s through those colors and accents that the character of your piece shines through.

Core Loop

Over the years I’ve been making music, I’ve learned that the fastest way to ensure a good final song is the idea of the core loop. Pop songs often repeat bars in 4/4 time and change structure slightly every 4 or 8 bars. The key building block usually includes the chord progression, bass line and drums, and ensuring that the most repetitive aspect of the song sounds amazing is priority number one. Creating that is like the minimum viable product that focuses on ensuring the core loop of gameplay feels satisfying and engaging. This is the bare bones of your final game, but requires at least some aspect of both art and programming to accomplish it. Recently I attended the Global Game Jam in Pittsburgh and created a dancing-platform pusher game about disco aliens strutting their stuff. We found that the core loop - that is, the part of the gameplay with no goal (known as the toy) was fun enough. We could dance around endlessly without knowing how to win, and so we were confident in our direction after that proof-of-concept.

Humanization

This segues into the concept of layering and variation, going back to the aforementioned ‘colors’. The idea of creating something ‘good’ involves multiple iterations to see what’s missing and additional layers (like shaders for game aesthetics and humor in narrative/design) all serve to keep the ‘guest’ interested. After getting the core loop for the instrumental, additional instruments or change-ups in melody/rhythm are added to keep things fresh as the human mind has an immense ability to identify patterns. Some repetition is required for both game and music experiences, but too much repetition feels robotic. In creating sound effects for the games in BVW, I realized the need to create multiple samples to reduce repetition fatigue for the same interaction type as the player will encounter that sound effect many times throughout gameplay. This allows NPC dialogue, for example, to feel more convincing and ‘human’, and that humanizing quality of variation and subtle errors helps drum and instrument programming sound like they’re being played by real musicians (instead of clicked in with a mouse). Many genres of music use a strict grid for note rhythm (called quantization) but many hip hop producers that program the grooviest beats go ‘off-grid’ to add soul to otherwise stiff, robotic drums. J Dilla is an amazing example of an artist that programs drums in that way. I think this applies to the uncanny valley - the repulsion we experience when we spot the robo-clone fakery. A little touch of intentional human ‘error’ goes a long way to add that touch of empathy and humanity to a work.

Tension

With variation comes the elements of tension, pacing, and surprise. This is essentially the interest curve of the song, and arises from setting up and playing with audience expectations. Building up towards something is key in games, both for player skill acquisition to the emotional arc of the game’s narrative, and creating tension and surprise are both good principles to increase engagement of games. EDM is built on the idea of a build and release, with the release corresponding with an emotional catharsis - often subverting expectations resulting in innovative ‘drops’ that break traditional EDM tropes (Bassnectar is a great example of a DJ that produces mind-blowing surprises in his songs). Tension is built through challenge in the game, ambiance of the sound/visual style, and flow in the narrative framing the game’s objective, and pacing is about balancing the game mechanics via playtesting and testing the story’s narrative arcs in their emotional impact. Usually aligning both the pacing of the song and game, from core loop to overall song/game structure, will lead to best results and maximize the emotional impact of the gameplay experience.

Movement

As I evolved as a musician, I started to realize the importance not just of the singular song you’re currently working on, but the style of your body of work. This also was something my dance and music pursuits pointed me towards - it’s not about what you can do, but what you add to the conversation through a number of pieces strung together. Through this introspective lens, I began to think beyond the core loop, and beyond the individual song, towards sets and albums and mixes and scores. This would be the macro or holistic part of music - how different pieces transition and create a narrative in the listener (even if there was no explicit narrative in the lyrics). The overall emotional arc of a longer experience lends to a deeper understanding on how to design one’s narrative or gameplay to move the audience energy up or down based upon the feeling or energy the creator is trying to paint. This broader sensibility has allowed me in my own work to understand narrative design better, tying in both interest curves and structures like the Hero’s Journey to the pacing of music and the physical body. Performing shows at live venues creates a deeper empathetic response in the performer towards audience members or guests of the experience, keying in on when the audience would like to dance, drink, or take a break, and I believe a lot of untapped experience design inspiration lays in this realm.

Curation

Another facet of style is knowing what sounds good, and what vision you have for the final result. Learning processes like scrum and agile, I’ve come to understand that while you can reasonably estimate a vision for the end product, you must acknowledge the change that will come as you iterate and playtest. Creating an initial vision for the song, and informing my decisions to pivot often come from the important act of self-curation. While the end song may not sound at all like what you had in mind, having that flash of initial inspiration provides the momentum needed to kick off the creative juices, and relating your work to additional existing works as you progress maintains that inertia when you alter or edit your composition. When deciding what to make next, a good strategy is to look within for inspiration as many wise people state there is no such thing as an original idea. Sampling (taking cuts of audio from other recordings) and remixing (using the stems of a source song to create an alternate version) are two musical art forms that play on this idea and utilize what’s already been ‘built’ in the world within their own experiences, often changing the source material so drastically that there is almost no resemblance. This is akin to the resourcefulness needed by game developers when searching for APIs and open-source code snippets on GitHub, or artists when they look on Mixamo or Sketchfab for animation cycles or pre-built models.

Experience

No experience is wasted, and all of those years spent listening to music on the subway or playing games in the basement have all filtered through my ears (and eyes) and distilled into what people have remarked as ‘my sound’. The resulting style is influenced by the collection of things you hold onto as interesting, and as we embark on fleshing out 100 games we’ve played over the course of our lifetime in this Game Design course, I’m excited to document and categorize my curated nuggets of inspiration. Like in music, there is no such thing as a ‘best’ game. You can have technically innovative games, but you can also have simple games that are amazing - in my opinion the most important thing is having a unique style to a good game.

Vocabulary

An interesting point was brought up in discussion of how to make this post more actionable for people less familiar with music, theory, and terminology. I think that in addition to curating your own library of sounds and contexts (essentially playing a mental exercise of placing different music in a hypothetical game context), the next step is to start describing pieces of music you like and what adjectives can communicate a musical concept to another most easily. This will streamline the output and reduce mismatched assumptions on what each collaborator means - but this takes a lot of practice via conversation. It can help to read up on the basics of musical theory and the nuanced definitions of musical descriptors like timbre, tone, texture, tension, groove, and momentum. On a more technical level, it can help to understand the difference between the mixing and mastering process, what’s involved in recording audio, and the technical flow of DAWs like FL Studio, Logic, or Ableton. All of this knowledge will carry forward in your work no matter what area you focus on, and serve to create a backdrop for setting an aligned vision for any team.

Challenge

Anyone can create great music. Many people say that they don’t have the talent or know-how (which can be learned), but all they need (like in game design) is to have a vision of what they want to hear in their heads and to tweak what they’ve programmed until it sounds right. I have noticed that in the Entertainment Technology program, not a lot of courses are dedicated to video game music and not a lot of focus is given to innovating on this topic. I believe that not only is there a lot to be learned from music making as applied to video game design, but that having an understanding of the principles of music production and composition can help improve the musical choices within games. Even simply curating a list of your favorite songs will help in selecting the right type of music or musician for the game moment, and I hope that more video game developers take risks when it comes to including innovative and underground music in their game. I encourage artists, designers, and programmers in the industry to reflect on what it takes to craft a good song, and how to use that towards innovating video games to come.



Comment