Android Audio Tutorial [Part Six] :Introduction to AudioPolicyService.

AudioPolicyService is a major service of the Android audio system. It works in conjunction with the AudioFlinger for all audio use-cases be it playback or recording. AudioPolicyService starts at boot during the mediaserver loading in the same way as AudioFlinger does.

Some Questions which might arise immediately with respect to AudioPolicyService  could be.

What is AudioPolicyService ?

Why should it exist?

How is the underlying Audio HAL layer integrated into Android?

We can answer all of the above by looking into the code etc,

However, Let us answer the above questions with couple of questions itself.

“After inserting the headphones, how does the sound change from the initial external speaker or output  to the headphone output?”

“How does the volume from calls/music etc get adjusted?”

The answer to some/all of these questions lies in a better understanding of the AudioPolicyService.

We earlier explained earlier that AudioFlinger only executes the policy which is framed or decided by the AudioPolicyService.

This separation effectively reduces the coupling of the entire system, and provides guarantees for independent expansion of functions of each service.

Startup of AudioPolicyService:

From the file frameworks/av/media/mediaserver/main_mediaserver.cpp, We see the place where AudioPolicyService starts.

The main tasks of AudioPolicyService are.

a)    Input and output device connection status.

b)    System's audio strategy switching.

c)    Volume/Audio parameter settings.

The below diagram shows the class Diagram for AudioPolicyService and its constituents.

Some important points worth mentioning after looking at the above diagram.

a)    AudioPolicyService inherits the IAudioPolicyService interface, so that AudioPolicyService can provide services to external parties based on Android's Binder mechanism. The calling application would need to us “media.audio_policy”.

b)    AudioPolicyService also inherits AudioPolicyClientInterface class, which has a member pointer mpPolicyManager of AudioPolicyInterface class, which actually points to AudioPolicyManager.

c)    The AudioPolicyManager class inherits the AudioPolicyInterface class to provide services to the AudioPolicyService.

d)    There is also an AudioPolicyClientInterface pointer that is initialized in the constructor and points to the AudioPolicyService.

e)    In fact, the AudioPolicyService accesses the AudioPolicyManager through the member pointer mpPolicyManager.

f)     The AudioPolicyService is accessed through AudioPolicyClientInterface(mpClientInterface).

g)    AudioPolicyService has an internal thread class AudioCommandThread, as the name suggests, all the commands (volume control, input, output switching, etc.) will eventually be queued in the thread.

Let us look at the AudioPolicyService::onFirstRef( ) which gets instantiated when the AudioPolicyService is first created.

AudioPolicyManager.

A large part of the AudioPolicyService management work is done by AudioPolicyManager.

The functions of AudioPolicyManager includes volume management, audio strategy management, input and output device management. Let us take a look at some of them.

Input and output device management.

The Android Audio system defines a set for being used as audio device.

Each enumerated value actually corresponds to a bit of a 32-bit integer, so these values ​​can be bits or operations.

For Example, If we want to turn on speakers and headphones at the same time for any specific usecase we can use something like.

newOutputDevice = DEVICE_OUT_SPEAKER | DEVICE_OUT_WIRED_HEADPHONE;

setOutputDevice(mHardwareOutput, newOutputDevice);

There are two important variables in the AudioPolicyManager as shown below mAvailableOutputDevices and mAvailableInputDevices.

They record the currently available input and output devices.

When the system detects that the headset or Bluetooth is connected, the member functions of the AudioPolicyManager are called.

This function sets mAvailableOutputDevices or mAvailableInputDevices according to the passed device value and state (DEVICE_STATE_AVAILABLE/DEVICE_STATE_UNAVAILABLE), and then selects the corresponding input or output device.

Let us take a step back and see the AudioPolicyService::onFirstRef() which gets called when the AudioPolicyService gets created.

  

Let us see the above function, AudioPolicyService::onFirstRef() in slightly more detail.

 rc = hw_get_module(AUDIO_POLICY_HARDWARE_MODULE_ID, &module);

Get the hw_module_t of Audio Policy. The implementation of Policy is in Audio_policy_hal.cpp.

We can see the details of struct audio_policy_service_ops aps_ops as shown below.

This way of implementation allows audio hardware manufacturers to develop their own audio strategy and providing greater flexibility.

rc = mpAudioPolicyDev->create_audio_policy(mpAudioPolicyDev, &aps_ops, this,

                                                   &mpAudioPolicy);

A policy is generated through the above Audio Policy device.

We can see the details of the legacy_audio_policy and its related structures below.

All further transactions will happen through the AudioPolicyInterface apm interface as below.

sp<AudioPolicyEffects>audioPolicyEffects = new AudioPolicyEffects();

Load the default AudioEffects from the files.

We already saw AudioPolicyManagerBase being called, Below is how it gets created.

AudioPolicyManagerDefault inherits from AudioPolicyManagerBase, Let us see the constructor of AudioPolicyManagerBase.

Let us look at some important items in this call.

mPhoneState(AudioSystem::MODE_NORMAL),

The AudioSystem maintains a an enum of audio modes.

The explanation for these states could be.

MODE_INVALID = -2, // invalid mode

MODE_CURRENT = -1, // current mode, related to the switching (routing) of the audio device

MODE_NORMAL = 0, / / ​​normal mode, no phone and ringtone

MODE_RINGTONE, / / ​​received the incoming signal, and there will be a ringtone

MODE_IN_CALL, / / ​​phone mode, which means that the call has been established

Let us continue further.

  

Android uses device categories which will be used when setForceUse( ) gets called.

So why use setForceUse, Well under different usecases we would need to overwrite the existing output with the one which has been identified in the normal case.

Let us see an example.

When we call setSpeakerphoneOn( ) during a call eg: sipcall, Let us see how through the flow.

  

 

Loading Audio Devices.

We briefly mentioned above that AudioPolicyService will load audio devices in the current system by parsing configuration files.

Specifically, when the AudioPolicyService is constructed, an AudioPolicyDevice (mpAudioPolicyDev) is created and an AudioPolicy (mpAudioPolicy) is opened.

The default implementation of this Policy is legacy_audio_policy::policy (data type audio_policy).

While legacy_audio_policy also contains an AudioPolicyInterface member variable, it is initialized to an AudioPolicyManagerDefault, which we analyzed in the previous section.

So when does AudioPolicyService load the audio device?

In addition to the later dynamic addition, another important way is through the parent class of AudioPolicyManagerDefault, which is the constructor of AudioPolicyManagerBase.

We can see from above that during the initialization of AudioPolicyManagerBase, We load the policy configuration files from the two directories.

If audio_policy.conf does not exist, the system will use the default configuration, which is implemented in defaultAudioPolicyConfig.

Through the configuration file one can can read the following information.

a)    What are the audio interfaces, such as "primary", "a2dp", and "usb"?

b)    The properties of each audiointerface.

a.     For example support for sampling_rates, formats, which devices are supported, etc.

c)    These properties are read in loadOutput() in AudioPolicyManagerBase and stored in HwModule->mOutputProfiles.

d)    Each audio interface may have a number of output and input, and each output/input has a number of specific support attributes, as shown in the following figure.

An example of the audio_policy.conf is shown below.

Let us take one more look at one of the devices.

audio_hw_modules {
	primary {
	outputs {
	primary {
	sampling_rates 44100|48000
	channel_masks AUDIO_CHANNEL_OUT_STEREO
	formats AUDIO_FORMAT_PCM_16_BIT
	devices AUDIO_DEVICE_OUT_EARPIECE|AUDIO_DEVICE_OUT_SPEAKER|AUDIO_DEVICE_OUT_WIRED_HEADSET|AUDIO_DEVICE_OUT_WIRED_HEADPHONE|AUDIO_DEVICE_OUT_LINE|AUDIO_DEVICE_OUT_ALL_SCO
	flags AUDIO_OUTPUT_FLAG_FAST|AUDIO_OUTPUT_FLAG_PRIMARY
	}

Let us look at the above and understand how does it translate in the relationship table.

After reading the relevant configuration, it is time to open these devices. AudioPolicyService is only a policy maker, not an executor.

These tasks are completed by AudioFlinger. We can see that there is a mpClientInterface variable in the above function segment.

Is it related to AudioFlinger? Let us see how did it come.

Obviously the mpClientInterface variable is initialized in the first line of the AudioPolicyManagerBase constructor, and trace back, you can find its root in the AudioPolicyService constructor, the corresponding code statement is as follows.

In other words, mpClientInterface->loadHwModule actually calls aps_ops->loadHwModule. Let us see how.

In a similar way it applies to mpClientInterface->openOutput

And, We comeback to where we started at AudioFlinger [Note, We started with AudioFlinger when we started this tutorial].

In other words, mpClientInterface->loadHwModule actually calls aps_ops->loadHwModule, ie:Back in the AudioPolicyManagerBase constructor, there are two goals for the for loop.

a)    Use loadHwModule to load the audio interface parsed from audio_policy.conf, ie the elements in the mHwModules array.

b)    Use openOutput to open all Output contained in each audio interface

About the implementation of these two functions in AudioFlinger, we have analyzed in the previous tutorials and we stich them together here.

Through AudioPolicyManagerBase, AudioPolicyService resolves the audio configuration in the settings, and uses the interface provided by AudioFlinger to complete the deployment of the entire audio system, which provides the underlying support for the upper-layer applications using audio devices.

Sounds good now that things are intertwined and getting clear.

In the next tutorials, we will see how the upper application uses this framework to play audio.