Gestures without libraries, toolkits or training: a $1 recognizer for user interface prototypes

Authors:

Jacob O. Wobbrock

Andrew D. Wilson

Yang Li

Summary:

Goal :

1. be resilient to variations in sampling due to movement speed or sensing;

2. support optional and configurable rotation, scale, and position invariance;

3. require no advanced mathematical techniques (e.g., matrix inversions, derivatives, integrals);

4. be easily written in few lines of code;

5. be fast enough for interactive purposes (no lag);

6. allow developers and application end-users to “teach” it new gestures with only one example;

7. return an N-best list with sensible [0..1] scores that are independent of the number of input points;

8. provide recognition rates that are competitive with more complex algorithms previously used in HCI to recognize the types of gestures

A class library contains templates . A template/ gesture is a sequence of points. A user entered gesture is first resampled, rotated to indicative angle and scaled to match the template. A MSE score for the best angle is then calculated for each class template. N - best list is then generated based on the MSE score.

User study : Data was collected from 10 users - 4800 gestures. 16 gestures were asked to draw in slow, medium and fast speeds.

The recognizer provides 97% accuracy on 1 training sample / class and 99.5% accuracy for 3+ training sample/ class.

Limitations: $1 cannot distinguish gestures whose identities depend on specific orientations, aspect ratios, or locations. 1D gestures like horizontal and vertical lines cannot be recognized - will be affected by non uniform scaling. The gestures cannot be differentiated based on speed.

Discussion:

A pretty extensive user study which analyzes effect number of training sample/ template, execution speed, accuracy,... I do not understand how qualitative data can be collected for a classifier. But the authors have collected qualitative data on the gestures that users liked. Analysis of the limitations of the recognizer was interesting. The authors also provide solution for removing some limitations.

Comments: Peschel, Franck