Thomas Edison and one of his early phonographs
Lately I’ve been digging into Android audio APIs. Earlier I wrote an introductory article that describes the three available APIs for WiseAndroid. Now this article assumes you are familiar with AudioTrack, SoundPool and MediaPlayer at the basic level.
What I want to present in this post is my experience with the existing audio APIs on Android, including the issues and problems I personally faced. I will also shortly cover OpenSL ES, the standard that is expected to be supported in one of the upcoming Android releases.
Known problems with existing APIs
All three of the existing audio APIs (SoundPool, AudioTrack and MediaPlayer) have bugs (behavior inconsistent with documentation), hidden (undocumented) limitations and design issues. Let’s review some of them per each API.
AudioTrack
AudioTrack is marketed as the most flexible and advanced audio API, allowing you to push raw audio data to the hardware. While it does work to some extent, and might allow you to solve some use cases, it still fails in some very basic scenarios.
I faced both of the following issues personally before I found that they had already been submitted to the Android bug database by Piotr Buła. Unfortunately, neither of them are resolved as of today:
Issue 3197: Audio breaks after fast, repeated attempts to replay sound using AudioTrack (in static mode)
This bug actually prevents developes from using AudioTrack for software synths and games (where you could benefit from its low latency and ability to play long audio samples, which is not possible with SoundPool).
From what I’ve seen, it seems to be a miserable race condition. If you add a delay (SystemClock.sleep() for 30-40 ms on Droid), this issue goes away. Of course, it’s a pretty wretched solution which I can’t recommend. So consider this use case broken.
Issue 3198: AudioTrack playback state does not reflect reality
Now this is arguably less of an issue, but the problem seems to be that when the sample is finished playing the status is not updated correctly in AudioTrack code.
As you might guess from the serious impact of those two issues, AudioTrack is generally not very stable or well implemented. So if you’re planning to use it in your app, do test your use cases before you go. And read on to learn limitations and problems of other APIs that you might consider as alternative.
SoundPool
SoundPool is the foundation of short, low-latency audio on Android. It is widely used in games and other apps that need sound effects. However, it is rather limited in its functionality, and poorly documented in terms of limitations and use cases it supports.
Although Piotr Buła reported an issue on SoundPool too, it’s not as serious (it’s related to release() method and seems to be not reproducible for me). I faced other rough edges though:
- Inability to detect whether samples are loaded (fixed in 2.2 but what about older versions, wasn’t it an obvious piece of functionality?)
- Hidden undocumented limit of sample size to 1 Mb (seems to boil down to the line
static size_t kDefaultHeapSize = 1024 * 1024; // 1MB
…in MediaPlayerService.cpp)
- Sudden periods of high latency (up to 1 second) with corresponding reports in the log from the platform such as write delayed for XXX ms
- Lack of support for some audio formats such as floating-point PCM WAVs. However, this might be documented somewhere and I might’ve just missed that.
As you can see, these issues together might create enough pain for developers who expect SoundPool to do what it promises. Combined with the AudioTrack issues described above, non-trivial low-latency audio on Android becomes a real headache, and this is an area in which the platform is quite backwards compared to iPhone. We have to admit that and work on this issue as a community. This is why I ask you to do the simplest thing you can – VOTE for each of those issues if they affect your current or future apps (the vote star is located at the bottom of each bug page).
MediaPlayer
MediaPlayer is probably the most widely used Audio APIs on Android, and it seems to be the best implemented too. You can hear various complaints in the dev groups, but the worst things I brushed into with MediaPlayer were random log messages and poor problem handling.
Another thing many people want is an option to play from an InputStream. This might be tough to implement given that most of MediaPlayer code is native, but has to be possible.
Native audio API & OpenSL ES
Support for fast native graphics, both 2D and 3D, has been improving with each Android version and seems to have reached the level required for very sophisticated apps. The situation is very different with audio, which is quite sad. After all, audio software has been a large industry on desktop computers for a while, and mobile features such as portability might be very appealing for audio software authors.
However, just about any audio software needs to do its job with high performance, good quality and low latency. All of that kind of suggests that you need native code (and decent APIs). However, as of Froyo, we don’t have that option.
What has been heard from the platform team is that one of the future versions might support OpenSL ES, a not very well known standard that is designed by the same organization as OpenGL and has a similar technical approach.
Having read the specification, I have the following comments:
- Some of the features OpenSL ES includes are not that demanded, such as MIDI support
- It’s not clear which profile Android is going to support, and if you ask me, none of them fit
- The feature set is mostly covered by other existing APIs – yes, with issues, but why not fix the existing APIs instead of introducing a new one
- No performance, latency etc. levels seem to be imposed by the spec – everything depends on the implementation
- If platform devs use the existing native code to implement the new API, we will have the same set of problems – just under a different cover
Thus, as you might notice, I’m highly uncertain whether OpenSL ES would fix the issues we are facing today. I think effort should better be spent on fixing the existing APIs. Once they’re good enough, we can implement OpenSL ES on top of them – if anyone actually needs it. For now, I’d just love an AudioTrack-like API in native code, better without the issues that AudioTrack has.
Tags: android, apps, architecture, audio, development, guts, hacks, native
