Developing mobile audio effects and soundtracks for mobile gaming poses significant challenges and raises several problems for developers to contend with. In this series of articles we’ll look at the main challenges of developing mobile audio and the workaround solutions for soundtrack development.
Many mobile devices simply cannot play multiple sound files at once, and are limited in most cases to playing just one sound. This can be very challenging for designing immersive games with good quality sound effects.
Audio playback on mobile devices has to be queued; this of course means that the design process needs to prioritize the choice between sound effects, or a dynamic soundtrack. The choice will depend on the nature of gameplay and the actual genre. For instance, an RPG game would be incomplete without a perpetual soundtrack. Equally, a puzzle platform type game would require more sound effects over a continual soundtrack.
Perhaps the challenge in this issue is how to combine a balance of sound effects and dynamic soundtrack that responds to player actions. If the mobile device only plays one sound at a time, the soundtrack design needs to be created using a crossfade at both the beginning and end points of the waveform so that music can seamlessly fade out and conjunctively play the next sound effect when it is called.
The next challenge of mobile audio is the technological limitation of current available devices. Not only is the playback method limited by headphones that are inevitably used by mobile gamers, the audio speakers and file formats of smartphones and tablets inevitably stunts all audio quality. Immersive audio is not possible in this situation, so this has to be kept in mind that audio will generally not be differentiated between left or right speakers for surround effects as expected in console or pc games.
The technical capabilities of headphones on mobile platforms such as Android or iOS means that many of the sound assets are simply wasted resources, as the higher frequencies are never going to be heard in this case. In essence the quality of audio will be limited by the current technology.
Device Limitations
Games developed for mobile platforms are played mainly on iPhone or Android devices, be it smartphones or tablets. The devices themselves feature limited mobile audio technology for audio playback.
The iPhone 5s supports such file formats as AAC, AIFF, WAV, and MP3. An android device such as the Samsung Galaxy s6 fares no better, but does offer slightly more file format support, including AAC, FLAC, MP3, OGG, WAV, WMA, and XMF. This makes it possible to utilize a high bitrate Flac file at 24bit encoding with manageable file sizes, still only producing one sound file at a time. There is still no M4A lossless support. The main problem with the iPhone is its inability to decode multiple sound compositions. It operates using uncompressed audio files such as WAV and AIFF formats, taking up a lot of file space in order to play audio within a game.
When developing an audio track for the iPhone, the workaround is to sample sounds at 24 kHz, this will drastically reduce the file size and not be such a drain on the battery life. If the iPhone is your primary platform for development releases, sampling all your assets at this bitrate is the way to go.
On the tablet side, the Galaxy Tab S 8.4 supports WAV, eAAC, WMA, and Flac playback, but no support for OGG, and OGG files are crucial for the most optimal file size compression with a maintained sound quality. Ultimately this means the development team needs to focus exactly on their primary choice of platform to release for when choosing the mobile audio format to encode their soundtrack with.
Perhaps the most obvious challenge of sound design for mobile audio becomes apparent here, a lack of cross platform capability. The biggest obstacle to overcome is designing sound to work on as many platforms as possible. The S 8.4 may not perform well for games using the OGG format. If sound designers want to release across all platforms, the challenge becomes one of designing only in WAV format and using limited compression for sound effects. Looping soundtracks could not make use of ogg files if the release was targeted for the Galaxy tab. This presents a difficult choice sound developers need to make when developing sound FX or musical soundtracks across platforms.
So how do developers actually create small file size dynamic soundtracks for mobile devices? To overcome this challenging limitation of developing mobile audio, the execution has to be clever and precise. Creating sound FX and longer musical scores must be done by designing small loops and using cross-fading techniques at the beginning and end point of each sound file. This duplicates the same sound file onto itself at the end point in order to complete the full sound effect before looping for constant playback. By using this method with only one file, the need for multiple files taking up storage is completely avoided.
Although developing mobile audio has its problems, using some of these techniques does provide a solution and allows indie developers to quickly gain an understanding of sound design for mobile gaming.
In Part 2 we’ll take a closer look at addressing the file size limit for audio files in mobile games.
