Whack gestures: inexact and inattentive interaction with mobile devices

Authors:

Scott E. Hudson

Chris Harrison

Beverly L. Harrison

Anthony LaMarca

Summary:

Goal: Inexact & inattentive interaction. interaction that does not require visual attention and does not disrupt the current activity.

Whack gestures - a set of gestures which are combinations of whack/ tap and wiggle/ shake. An adhoc recognition engine is set up to recognize the gestures. A user study with 11 people was performed to check the accuracy of the recognition.

Discussion:

I was totally taken back by the evaluation. I was expecting qualitative evaluation of the system as such user experience evaluation. I did not expect a recognizer accuracy evaluation. It would have been more interesting if they had evaluated the user experience given a certain scenario.

Comments: Murat

Gameplay issues in the design of spatial 3D gestures for video games

Authors:

John Payne

Paul Keir

Jocelyn Elgoyhen

Mairghread McLundie

Martin Naef

Martyn Horner

Paul Anderson

Summary:

effective implementation in 2D, 3D spatial gestures present their own problems in relation to: how to present 3D gesture feedback, user performance differences,how to instruct/learn user gestures,what are familiar semiotics for 3D gestures. The authors have created a device called 3motion and created several games to evaluate the useful ness of gestures in gaming scenarios. They identified intuitive gestures, clear user feedback and effective semiotics are important.

Discussion:

I did not realize that this was a paper before Wii was in the market. elaborate research on user feedback and recognizing features of feedback would be helpful. In fact, each topic identified as important needs an elaborate study.

Comments: Drew, franck

Wiizards: 3D gesture recognition for game play input

Authors:

Louis Kratz

Matthew Smith

Frank J. Lee

Summary:

Goal : Gesture recognition from 3D accelorometer data from Wiimote. Fluid tolerance and more immersive experience in gaming.

Paper evaluated a HMM model for recognizing Wii accelerometer gestures. The effect number of training data and number of states in HMM on the recognition speed and accuracy has been analyzed. The average correctness of HMM without training data is also measured which is around 50%. The HMM performed well at 15 states and after 10 samples per gesture yielded 80% accuracy and after 20 samples per gesture training 95% accuracy. The results showed that more states in HMM meant more training time and slower recognition speed.

Discussion:

I do not see what is novel about this system. Almost all the classifiers need some form of training so why not think about making the training process interesting or making the training part of the game.

Comments: Franck, Kevin

Device agnostic 3D gesture recognition using hidden Markov models

Authors:

Anthony Whitehead

Kaitlyn Fox

Summary:

Goal: test the accuracy of the 3D gestures by dividing the 3D space into subspaces.

The 3D space is divided into 27, 64 and 127 cubes. The data collected is then used to train HMM and tested on a gesture set. The number of training examples needed, recognition speed are measured. 27 state HMM acheived 800 recognition per second while a single recognition took seconds for 125 state HMM. The number of training data needed to train 25 state HMM was lesser. More states in the HMM meant more training data to train.

Discussion:

I am confused between the states and the segmentation of the space. By states do they mean the actual number of states in the HMM or the number of segmentations. The interesting part though is dividing the 3d space into cubes.

Comments: Franck,

An architecture for gesture-based control of mobile robots

Authors:

Iba, S.

Weghe, J. M. V.

Paredis, C. J. J.

Khosla, P. K.

Summary:

Goal: Gesture recognition system to interact with Robots. More appropriately gesture spotting with HMM.

Hardware used : Cyberglove + 6DOF location sensor.

Feature set: 18 sensor data is reduced to 10 dimension vector and each dimension taken with it derivatives increases the dimension of the vector to 20. This 20 dimension vector is coded on to a 32 bit integer. This codeword is then sent into HMM for recognition. HMM is trained with 5000 postures from full hand posture space. Restricting the observation sequence to better quantification. Wait state is included to provide a method to reject invalid gestures.

Gesture set : opening, opened, closing, pointing, waving left and waving right. These gestures carry different semantics in local mode and global mode of robot control.

Discussion:

Interesting extension to HMM for rejecting invalid gestures. Are gestures better than joystick controls while interacting with robots?

Comments: Drew, Franck

Gestures without libraries, toolkits or training: a $1 recognizer for user interface prototypes

Authors:

Jacob O. Wobbrock

Andrew D. Wilson

Yang Li

Summary:

Goal :

1. be resilient to variations in sampling due to movement speed or sensing;

2. support optional and configurable rotation, scale, and position invariance;

3. require no advanced mathematical techniques (e.g., matrix inversions, derivatives, integrals);

4. be easily written in few lines of code;

5. be fast enough for interactive purposes (no lag);

6. allow developers and application end-users to “teach” it new gestures with only one example;

7. return an N-best list with sensible [0..1] scores that are independent of the number of input points;

8. provide recognition rates that are competitive with more complex algorithms previously used in HCI to recognize the types of gestures

A class library contains templates . A template/ gesture is a sequence of points. A user entered gesture is first resampled, rotated to indicative angle and scaled to match the template. A MSE score for the best angle is then calculated for each class template. N - best list is then generated based on the MSE score.

User study : Data was collected from 10 users - 4800 gestures. 16 gestures were asked to draw in slow, medium and fast speeds.

The recognizer provides 97% accuracy on 1 training sample / class and 99.5% accuracy for 3+ training sample/ class.

Limitations: $1 cannot distinguish gestures whose identities depend on specific orientations, aspect ratios, or locations. 1D gestures like horizontal and vertical lines cannot be recognized - will be affected by non uniform scaling. The gestures cannot be differentiated based on speed.

Discussion:

A pretty extensive user study which analyzes effect number of training sample/ template, execution speed, accuracy,... I do not understand how qualitative data can be collected for a classifier. But the authors have collected qualitative data on the gestures that users liked. Analysis of the limitations of the recognizer was interesting. The authors also provide solution for removing some limitations.

Comments: Peschel, Franck

The $3 recognizer: simple 3D gesture recognition on mobile devices

Authors:

Sven Kratz

Michael Rohs

Summary:

Goal: build a simple 3D gesture recognition system - simple to implement and require less training data

A gesture recognizer was build for 3D gestures extended Wobbrock's 1$ recognizer for 2D touch gestures. It is easy to implement, requires as less as 5 samples for good recognition result. The system uses raw acceleration data. The change in acceleration is calculated. A sequence of acceleration delta is used to represent a gesture trace. A class library contains a list of gesture traces. The user entered gesture is compared to the class libraries to generate a score table. A heuristic is then applied to recognize the gesture.Resampling, rotation to indicative angle and rescaling are used to normalize the user performed gesture. MSE at best angle if found for each class and a score table is generated. To reduce false positives, a heuristics is used. A threshold value is set. The score table is sorted based on the scores. A heuristic to select class is defined based on threshold. If the threshold dont hold the gesture is recognized as unclassified.

Discussion:
Very similar to the $1 recognizer. easy to implement and easy to train. Providing accuracy data would have been more compelling in defending the classifier. introducing "unclassified" tag is an improvement over the $1 recognizer.

Comment: Franck