Android Input Framework Architecture[Part 1] : Introduction to InputReader & InputDispatcher.

Introduction:

User interface.

The user interface (UI), in the field of human–computer interaction, is the space where interactions between humans and machines occur.

Almost all great UI elements and platforms work in tandem with the basic user inputs though different ways.

In computing, an input device is a piece of computer hardware equipment used to provide data and control signals to an information processing system such as a computer or information appliance. Examples of input devices include keyboards, mouse, scanners, digital cameras and joysticks.

In today’s computing devices user input traditionally is moving from click to pointing/touch devices.

Types of Input on Android Devices.

Primarily Touch has been the most important way to receive user input on the Android devices, However the devices also come with buttons [Power/Volume rockers etc].

On non-traditional [non Handheld devices like Automotive Infotainment] devices have rotary knobs etc to accomplish user controls.

This blog is an honest try to introduce Android’s Input Framework and understand how Android OS behaves on receiving any user input.

Android is based on the Linux kernel and hence it primarily uses the Linux input subsystem as its base, The following is a great read on the Linux input subsystem.

https://www.kernel.org/doc/html/v4.15/input/input_uapi.html

Below is an architecture Diagram for the basic working model of the Android Input Subsystem. We will explain each of these blocks and how they are intertwined to achieve the user input/interface usecases.

 

Input Pipeline Explanation.

At the bottom level, the physical input hardware device indicates that the state of the device has changed by generating an electronic signal, such as a button or touch. The device firmware then encodes these signals and reports them to the system via the USB HID or via the I2C bus.

After receiving these signals, the Linux kernel decodes it through the device driver.

The Linux kernel provides drivers for many standard peripherals, especially the standard HID protocol.

The Linux input device driver is responsible for converting device-specific signals to standard input event formats via the Linux input protocol.

The Linux input protocol defines a standard set of event types in the /include/linux/input.h header file of the code .

Android EventHub then reads the input device event by opening the event device driver associated with the device.

The Android InputReader then parses the input events based on the device class and the Android input event stream.

In this process, the linux protocol event code will be converted to Android event code by input device configuration file (*.idc) , keyboard layout file (*.kl) , and various mapping tables ( *.kcm ) .

Finally, the InputReader sends the input event to the InputDispatcher , which then sends the event to the appropriate window using the ViewRootImpl.

Above can simply be put as.

input hardware  ------>   kernel/driver(input protocol)  -----> EventHub(framework/base/libs/ui)  getevent------> InputReader ----> inputDispatcher ---->  Window manager.

Let us continue and see each of these components in a greater detail.

The main piece of code we are interested is in the frameworks/native/services/inputflinger/.

InputManagerService which is started by the systemServer during booting is the starting point of our analysis.

The core InputManagerService is implemented in the native layer and is exposed to java using JNI.

  

The entry to the Native side of input manager is implemented by NativeInputManager.

EventHub waits for events from the kernel for all the input devices.

Android inherently supports a number of input devices which are listed in EventHub.h.

Let us continue and see the EventHub constructor. 

Coming back, We could also see the creation of InputManager. InputManager is responsible for creating the InputReader and the InputDispatcher as explained in the diagram above. 

We can see from above that the initialize routine in the InputManager creates the 2 threads InputReaderThread & InputDispatcherThread.

 Next, we will visit each of these components and see their working in detail.

 InputReader.

InputReader is by far the most important component in our analysis.

InputReader is an important part of the Android system. According to the description in the Android documentation, the main functions of InputReader are.

1.     reading events from EventHub.

a.      These events are meta-events, ie events that have not been processed or simply processed by simple processing;

2.     Process these events to generate inputEvent events, so that the encapsulated events can meet some of the requirements of the Android system.

3.     Send these events to the event listener, QueuedInputListener, which can pass events to InputDispatcher.

Let us analyze the implementation of these functions step by step from the point where the thread starts executing. 

Note that while creating the InputReader we pass the mDispatcher [InputDispatcher] instance.

When the InputReader is created, the InputDispatcher is passed as a parameter, and the QueuedInputListener object is constructed with the InputDispatcher as a parameter.

Therefore, Now the InputReader holds a QueuedInputListener, and QueuedInputListener holds an InputDispatcher object. Next, we will continue to use the threadloop as a clue to analyze our code and then look at it.

Since, This is the most important part of the InputReader piece. Let us take sometime and relook at some of the most important constituents.

size_t count = mEventHub->getEvents(timeoutMillis, mEventBuffer, EVENT_BUFFER_SIZE);

EventHub.

Let us briefly introduce the EvenHub. The main function of this class is to actively monitor the changes of the Input driver. Once an event occurs, the event is read from the driver that generated the event.

The implementation of this monitoring driver is implemented by the epoll mechanism provided by Linux. The epoll mechanism is simply an efficient I/O multiplexing mechanism that uses epoll_wait to listen for changes to the required file descriptors.

The main function of EventHub is implemented by epoll_wait, so the thread where EventHub is located should be blocked in the epoll_wait method and wait until the timeout period set by epoll_wait.

Now let us take a look at the implementation of EventHub. 

In the constructor of EventHub, we create a pipe and add the file descriptors of the read and write ends of this pipe to the monitor of epoll, so that other threads or processes can Make the thread where the EventHub is located return from the blocking of epoll_wait.

After EventHub is created, the first method to be called is getEvents, and this method is also the main function of EventHub.

For this method to be carefully analyzed, we divide the getEvents method into three functions.

1.     open the device.

2.     event reading.

3.     waiting for more.

Among the three parts, the focus is on the read part of the event.

The device open part is generally called when the Input system is established, so after the system startup is completed and stabilized, this part of the content should not be executed again; and the waiting part is relatively simple.

However, these are indispensable parts of the system, but they must be explained one by one.

Let’s talk about the device opening part. The code is as follows:

The next part in the series of operations in the threadLoop is reading the events and creating the event structure.

Let us look at the RawEvent structure to find out what are the details which get filled by the EventHub for further processing.

Essentially , The RawEvent structure contains the time,the deviceId,Type of Event and the value of the event received from the input device. 

The next operation obviously would be to get into a epoll state to watch for the next set of events.

 Once the event is received, It is also processed by the InputReader. 

 

Events could be of many types viz: A device gets added /removed etc and also the raw events sent by the device itself.

Let us see some of these events in the driver framework which would be sent to the processing layers. 

Let us see some of these events and their meaning and significance.

  

Okay, Let us go back to the processing of these events by the InputReader.

Events like DEVICE_ADDED, DEVICE_REMOVED, FINISHED_DEVICE_SCAN are handled by addDeviceLocked, removeDeviceLocked & handleConfigurationChangedLocked functions and we will skip them for simplicity.

The main function which handles the other RawEvents is processEventsForDeviceLocked, Let us look at that. 

Ohh Wait, We can see a mapper->process(rawEvent); above but we did not see what mappers are.

Essentially not all events are meant to be for all the listeners, So Android has this concept of InputMapper which map the devicetype and the eventtype.

These Mappers are created when a device gets added, Let us see how.

  

The classes of these device are generated based on the type of events they support during the opening of the devices in the EventHub.cpp.

 The different Mappers which android provides are.

class SwitchInputMapper : public InputMapper {

class VibratorInputMapper : public InputMapper {

class KeyboardInputMapper : public InputMapper {

class CursorInputMapper : public InputMapper {

class TouchInputMapper : public InputMapper {

class ExternalStylusInputMapper : public InputMapper {

class JoystickInputMapper : public InputMapper {

 The mapping of the EventTypes and the Mappers which can handle them is given below.

Let’s proceed further and see a sample inputMapper processing, In this case let us take the KeyboardInputMapper.

  

We can see from above the KeyboardInputMapper handles the keyevents and notifies the listener for further processing.

One important thing to note here in the call to getListener()->notifyKey(&args); is the InputListenerInterface.

The InputListenerInterface is nothing but an instance of the InputDispatcher, How, Well we already saw in the constructor of InputReader. 

Continuing further, The InputDispatcher gets the actual events for further processing when the mQueuedListener->flush(); is called. 

We just completed the initial analysis of the InputReader in our quest to understand the Input Framework in Android devices.

To sum up the InputReader process in simple steps.

1.     InputReader reads the meta event from EventHub.

2.     Preprocesses these meta events into NotifyArgs.

3.     Notifies them to InputDispatcher via QueuedInputListener.

InputDispatcher.

We now have come to a point where we understand the role of the InputDispatcher which essentially delivers the Raw input events to the required recipients.

 Before we introduce the working of the InputDispatcher, Let's understand the core functionality of the InputDispatcher which are.

1.     InputDispatcher send [Read as Dispatches] input events to the required target.

2.     It’s end target could be an application or it may be WindowManagerService.

3.     If it is an application, you can use registerInputChannel to define the target of the input event.

We have learned that the only function of InputDispatcher is to distribute events.

Let us look at the constructor first, InputDispatcher creates a Looper.

  

This means that InputDispatcher has its own Looper, which is not shared with others, and the information is looped by itself.

In the process of building the Looper, a new pipeline gets created, which only wakes up the Looper and allows it to return from the blocking wait.

The pipeline created in Looper is an important way to implement the Looper function. It is generic, not just for InputDispatcher.

After reading the constructor, we then analyze the function of the InputDispatcher, and then the QueuedInputListener in the previous section tells the InputDispatcher to have a new button event.

There are three important steps in the notifyKey function.

1.     KeyEvent event;

    event.initialize(args->deviceId, args->source, args->action,

            flags, keyCode, args->scanCode, metaState, 0,

            args->downTime, args->eventTime);

2.     mPolicy->interceptKeyBeforeQueueing(&event, /*byref*/ policyFlags);

3.     mLooper->wake();

The above 3 steps , initialize the KeyEvent based on the RawEvent sent by the InputReader, calls the WindowManager to intercept the key before queuing and wakes up the looper for further processing. 

   

 As mentioned earlier the InputManager is responsible for interacting with other modules of the system as one of its functions.

After passing this KeyEvent to the InputManager, continues to distribute, and finally pass the KeyEvent to the PhoneWindowManager to handle the event.

The transfer process is as follows: InputManager->interceptKeyBeforeQueueing ----> InputManagerService.interceptKeyBeforeQueueing ----> InputMonitor.interceptKeyBeforeQueueing - ---> PhoneWindowManager.interceptKeyBeforeQueueing.

After the PhoneWindowManager handles the event, there will be a return value to mark what the result of this event processing is, in preparation for the subsequent events entering the queue. When the PhoneWindowManager intercepts the event in the early stage, the event is first marked with PASS_TO_USER, that is, the event is handed to the application for processing, but in the process of determining, some events are not necessary to be passed to the application.

For example: press volume related events during the pass, hang up the phone event, power button processing, and call events. The processing result of these events is not necessarily passed to the application. This result is the most return value, and will eventually return to the InputManager step by step.

This return value will be used as part of the policyFlags of the NativeInputManager for InputDispatcher.

After the event processing of the PhoneWindowManager is completed, the event will be constructed into a form of EventEntry into the queue.

At this point, our work is still in the thread of InputReaderThread, although it is an operation on InputDispatcher. The next step is to actually enter the InputDispatcherTread thread on the InputDispatcher operation. Wake up the InputDispatcherThread thread via the Inputooppater's mLooper wake method.

Coming back to the InputDispatcher path.

  

Let us see the dispatchOnceInnerLocked function and its core steps in detail.

 We pick and explain the following lines in detail.

1.     mInboundQueue.isEmpty()

a.     If there is no event in the inbound Queue , We do nothing.

2.     pokeUserActivityLocked(mPendingEvent);

a.     Some apps might be waiting for an event and we need to poke that activity in advance.

3.     done = dispatchKeyLocked(currentTime, typedEntry, &dropReason, nextWakeupTime);

a.     If the event needs to be dispatched and not dropped, We move ahead and dispatch it for further processing.

Let us proceed and see the next function dispatchKeyLocked.

Before entering the queue, there is an intercept for the event, here is the processing of the intercept result of the event.

Let’s proceed further.

We will come back to the doPokeUserActivityLockedInterruptible function when we talk about how the Framework handles this in Java side of Android. 

Finally the key event is published to the dispatch listener. This makes the processing goto the Java part of Android which we will in the next tutorial.