# Support and Use Extra Input Messages

_Support modern input devices and use the extra input messages in your MiniGUI apps._

Table of Contents

* [Overview](#overview)
* [Using the Extra Input Messages](#using-the-extra-input-messages)
* [Compile-time Configuration](#compile-time-configuration)
* [Run-time Configuration](#run-time-configuration)
* [Writing an IAL Engine to Support Extra Input Messages](#writing-an-ial-engine-to-support-extra-input-messages)

## Overview

In MiniGUI 4.0.0, we introduced extra input messages to support modern input devices including multiple touch panel, gesture, joystick, tablet tool, table pad, and even switch.

The extra input messages have prefix `MSG_EXIN_`. If a MiniGUI app needs to handle these extra input events such as gestures, you need to handle the `MSG_EXIN_XXX` messages in your window procedure.

Currently, there are two built-in IAL engines which can generate extra input messages:

* The IAL engine of `libinput` supports all modern input devices on a Linux box. This engine runs on `libinput` introduced by Free Desktop project.
* The enhanced IAL engine of `random` generates extra input messages automatically for testing.

You can also write your own IAL engines to generate extra messages. Please see the implementation of `libinput` and `random` engines for details.

This document describes how to handle extra input messages and how to use `libinput` and `random` engines, or how to write your own IAL engine to support modern input devices, for example, a multi-touch panel.

## Use Extra Input Messages

In addition to standard keyboard and mouse messages, MiniGUI generates extra input messages for input events from other input devices, including multi-touch panel, tablet pad, joystick, and so on. We call these messages as 'extra input messages'.

The extra input messages can be classified into the following types:

* Axis messages: messages generated by a pointer axis like mouse wheel.
  * `MSG_EXIN_AXIS`
* Button messages: messages generated by a button on joystick. The buttons other than left, right, and middle buttons on a mouse will be treated as generic buttons.
  * `MSG_EXIN_BUTTONDOWN`
  * `MSG_EXIN_BUTTONUP`
* Multi-touch messages: messages generated by a multi-touch panel.
  * `MSG_EXIN_TOUCH_DOWN`
  * `MSG_EXIN_TOUCH_UP`
  * `MSG_EXIN_TOUCH_MOTION`
  * `MSG_EXIN_TOUCH_CANCEL`
  * `MSG_EXIN_TOUCH_FRAME`
* Gesture messages: gesture messages.
  * `MSG_EXIN_GESTURE_SWIPE_BEGIN`
  * `MSG_EXIN_GESTURE_SWIPE_UPDATE`
  * `MSG_EXIN_GESTURE_SWIPE_END`
  * `MSG_EXIN_GESTURE_PINCH_BEGIN`
  * `MSG_EXIN_GESTURE_PINCH_UPDATE`
  * `MSG_EXIN_GESTURE_PINCH_END`
* Tablet tool messages: messages generated by a tablet tool.
  * `MSG_EXIN_TABLET_TOOL_AXIS`
  * `MSG_EXIN_TABLET_TOOL_PROXIMITY`
  * `MSG_EXIN_TABLET_TOOL_TIP`
  * `MSG_EXIN_TABLET_TOOL_BUTTON`
  * `MSG_EXIN_END_CHANGES`
* Tablet pad messages: messages generated by a tablet pad.
  * `MSG_EXIN_TABLET_PAD_BUTTON`
  * `MSG_EXIN_TABLET_PAD_RING`
  * `MSG_EXIN_TABLET_PAD_STRIP`
  * `MSG_EXIN_END_CHANGES`
* Switch messages: messages generated by a switch.
  * `MSG_EXIN_SWITCH_TOGGLE`
* User-defined messages: messages generated by a user-defined device.
  * `MSG_EXIN_USER_BEGIN`
  * `MSG_EXIN_USER_UPDATE`
  * `MSG_EXIN_USER_END`

All extra input messages will be sent to the current active main window. In your window procedure, you can handle the messages to reflect user's input, for example, zooming in or out a picture.

For the complete description of the messages, please refer to:

[http://www.minigui.com/doc-api-ref-minigui-sa-4.0.0/html/group\_\_extra\_\_input\_\_msgs.html](http://www.minigui.com/doc-api-ref-minigui-sa-4.0.0/html/group__extra__input__msgs.html)

For the sample to handle extra input messages, please refer to:

[https://github.com/VincentWei/mg-tests/tree/master/extra-input](https://github.com/VincentWei/mg-tests/tree/master/extra-input)

## Compile-time Configuration

There are two configure options related to the `libinput` and `random` IAL engines :

* `--enable-libinputial` or `--disable--libinputial` to enable or disable the `libinput` IAL engine which is enabled by default. Note that the `libinput` IAL engine is only available on Linux, and you need to install `libinput` 1.10.0 or later first.
* `--enable-randomial` or `--disable--randomial` to enable or disable the `random` IAL engine which is disabled by default. Note that `random` engine does not attach to any real input devices, it generates extra input events randomly.

## Run-time Configuration

For `libinput` engine, we introduced a new section in MiniGUI runtime configuration:

```
[libinput]
seat=seat0
```

The key `libinput.seat` specifies the seat identifier, the default value is `seat0`.

For `random` engine, we introduced a new section in MiniGUI runtime configuration:

```
[random]
logfile=events.out
eventtypes=mouse-keyboard-button-gesture-stouch
minkeycode=1
maxkeycode=128
minbtncode=0x100
maxbtncode=0x1ff
```

The MiniGUI runtime configuration key `random.logfile` specifies the log file which stores the input events generated by this engine. If MiniGUI failed to open the log file, the log feature will be disabled.

The MiniGUI runtime configuration key `random.eventtypes` specifies the input event types which is generated by this IAL engine, in the following pattern:

```
<event-type>[-<event-type>]*
```

The `<event-type>` can be one of the following values:

* `mouse`: mouse.
* `keyboard`: keyboard.
* `button`: buttons.
* `single_touch`: touch pen or single touch panel.
* `multi_touch`: multiple touch panel.
* `gesture`: gesture.
* `tablet_tool`: tablet tool.
* `tablet_pad`: tablet pad.
* `switch`: switch.

The MiniGUI ETC key `random.minkeycode` specifies the minimal key code which is generated by the engine if `keyboard` is included.

The MiniGUI ETC key `random.maxkeycode` specifies the maximal key code which is generated by the engine if `keyboard` is included.

The MiniGUI ETC key `random.minbtncode` specifies the minimal button code which is generated by the engine if `button` is included.

The MiniGUI ETC key `random.maxbtncode` specifies the maximal key code which is generated by the engine if `button` is included.

For invalid `random.eventtyps`, the engine uses `mouse` as default.

For invalid `random.minkeycode`, and/or `random.maxkeycode` key values, the engine uses `SCANCODE_ESCAPE`, and `SCANCODE_MICMUTE` respectively.

For invalid `random.minbtncode`, and/or `random.maxbtncode` key values, use `0x100` (`BTN_MISC` defined by Linux kernel), and `0x2ff` (`KEY_MAX` defined by Linux kernel) respectively.

This engine maintains a state machine for each input event type, and generates a reasonable event sequence for each type. If and only if an event sequence is finished or cancelled, the engine switches to another event type randomly.

Note that currently, the following event types (in `random` engine) are not implemented:

* `multi_touch`
* `tablet_tool`
* `tablet_pad`
* `switch`

## Write an IAL Engine to Support Extra Input Messages

On Linux, it is better to use `libinput` to support all input devices including multi-touch panel and table tool. However, for an embedded or IoT device, it may be too heavy to use `libinput` or there is no way to run `libinput` (e.g., when you run MiniGUI on a real-time operating system).

Under this situation, you need to implement your own IAL engine to support

extra input messages. The steps are as follow:

1. Use `--enable-customial` option to configure MiniGUI to include `custom` IAL engine.
2. Implement the following external stubs outside MiniGUI to initialize and terminate your `custom` IAL engine:
3. Change MiniGUI run-time configuration to specify key `system.ial_engine` to be `custom`.

```cpp
BOOL InitCustomInput (INPUT* input, const char* mdev, const char* mtype);
void TermCustomInput (void);
```

In `InitCustomInput` function, you need to fill the operations (the callbacks) of `input` argument, the pointer to a INPUT structure:

```cpp
typedef struct tagINPUT {
    char*   id;

    // Initialization and termination
    BOOL (*init_input) (struct tagINPUT *input, const char* mdev, const char* mtype);
    void (*term_input) (void);

    // Mouse operations
    int  (*update_mouse) (void);
    void (*get_mouse_xy) (int* x, int* y);
    void (*set_mouse_xy) (int x, int y);
    int  (*get_mouse_button) (void);
    void (*set_mouse_range) (int minx, int miny, int maxx, int maxy);
    void (*suspend_mouse) (void);
    int (*resume_mouse) (void);

    // Keyboard operations
    int  (*update_keyboard) (void);
    const char* (*get_keyboard_state) (void);
    void (*suspend_keyboard) (void);
    int (*resume_keyboard) (void);
    void (*set_leds) (unsigned int leds);

    // Event loop
    int (*wait_event) (int which, int maxfd, fd_set *in, fd_set *out,
            fd_set *except, struct timeval *timeout);

    // New wait event method for getting extra input events; since 4.0.0
    int (*wait_event_ex) (int maxfd, fd_set *in, fd_set *out,
            fd_set *except, struct timeval *timeout, EXTRA_INPUT_EVENT* extra);
} INPUT;
```

The key to support extra input messages is the new operation: `wait_event_ex`. If you implement `wait_event_ex`, you can set `wait_event` to be NULL.

In the implementation of `wait_event_ex`, when there is an extra input message, you need to:

1. Fill the fields of `extra` structure, e.g.:

```cpp
    extra->event = IAL_EVENT_AXIS;
    extra->wparam = MAKELONG(AXIS_SCROLL_VERTICAL, AXIS_SOURCE_WHEEL);
    extra->wparam = MAKELONG(AXIS_SCROLL_HORIZONTAL, AXIS_SOURCE_WHEEL);
    extra->lparam = MAKELONG(mouse_event->sv, mouse_event->dsv);
```

1. Make sure the return value of this operation have the bit of `IAL_EVENT_EXTRA` set:

```cpp
    retval |= IAL_EVENT_EXTRA;
```

For more information, please refer to the `libinput` and/or `random` IAL engines:

[https://github.com/VincentWei/minigui/blob/master/src/ial/linux-libinput.c](https://github.com/VincentWei/minigui/blob/master/src/ial/linux-libinput.c) [https://github.com/VincentWei/minigui/blob/master/src/ial/random.c](https://github.com/VincentWei/minigui/blob/master/src/ial/random.c)
