Whack gestures: inexact and inattentive interaction with mobile devices

Authors:

Scott E. Hudson

Chris Harrison

Beverly L. Harrison

Anthony LaMarca

Summary:

Goal: Inexact & inattentive interaction. interaction that does not require visual attention and does not disrupt the current activity.

Whack gestures - a set of gestures which are combinations of whack/ tap and wiggle/ shake. An adhoc recognition engine is set up to recognize the gestures. A user study with 11 people was performed to check the accuracy of the recognition.

Discussion:

I was totally taken back by the evaluation. I was expecting qualitative evaluation of the system as such user experience evaluation. I did not expect a recognizer accuracy evaluation. It would have been more interesting if they had evaluated the user experience given a certain scenario.

Comments: Murat

Gameplay issues in the design of spatial 3D gestures for video games

Authors:

John Payne

Paul Keir

Jocelyn Elgoyhen

Mairghread McLundie

Martin Naef

Martyn Horner

Paul Anderson

Summary:

effective implementation in 2D, 3D spatial gestures present their own problems in relation to: how to present 3D gesture feedback, user performance differences,how to instruct/learn user gestures,what are familiar semiotics for 3D gestures. The authors have created a device called 3motion and created several games to evaluate the useful ness of gestures in gaming scenarios. They identified intuitive gestures, clear user feedback and effective semiotics are important.

Discussion:

I did not realize that this was a paper before Wii was in the market. elaborate research on user feedback and recognizing features of feedback would be helpful. In fact, each topic identified as important needs an elaborate study.

Comments: Drew, franck

Wiizards: 3D gesture recognition for game play input

Authors:

Louis Kratz

Matthew Smith

Frank J. Lee

Summary:

Goal : Gesture recognition from 3D accelorometer data from Wiimote. Fluid tolerance and more immersive experience in gaming.

Paper evaluated a HMM model for recognizing Wii accelerometer gestures. The effect number of training data and number of states in HMM on the recognition speed and accuracy has been analyzed. The average correctness of HMM without training data is also measured which is around 50%. The HMM performed well at 15 states and after 10 samples per gesture yielded 80% accuracy and after 20 samples per gesture training 95% accuracy. The results showed that more states in HMM meant more training time and slower recognition speed.

Discussion:

I do not see what is novel about this system. Almost all the classifiers need some form of training so why not think about making the training process interesting or making the training part of the game.

Comments: Franck, Kevin

Device agnostic 3D gesture recognition using hidden Markov models

Authors:

Anthony Whitehead

Kaitlyn Fox

Summary:

Goal: test the accuracy of the 3D gestures by dividing the 3D space into subspaces.

The 3D space is divided into 27, 64 and 127 cubes. The data collected is then used to train HMM and tested on a gesture set. The number of training examples needed, recognition speed are measured. 27 state HMM acheived 800 recognition per second while a single recognition took seconds for 125 state HMM. The number of training data needed to train 25 state HMM was lesser. More states in the HMM meant more training data to train.

Discussion:

I am confused between the states and the segmentation of the space. By states do they mean the actual number of states in the HMM or the number of segmentations. The interesting part though is dividing the 3d space into cubes.

Comments: Franck,

Sensory Hand

Author:

Vernon b Mountcastle

Summary:

* Movement is the facilitating agent for complex tactile experiences.

the ability of the brain to integrate successive patterns of input to create a perceptual whole. The intra-cortical processing times of 80-100 ms for each pattern. Mechanical oscillations can be delivered to the skin. Humans can sense over frequency range from about 5 - 600 Hz. Frequencies in the range of 5 -50 Hz evoke a localized stimulus site. Increasing the frequencies changes the sensory experience gradually to the deep, spreading and poorly localized hum.

* Weber Law - Ratio of the perceivable change in tactile stimuli and the magnitude of stimuli is constant.

* Fechner law - Extending the Weber's law, Fechner quantified the sensation as logarithmic function of the magnitude of the stimulus.

Chapter 11

Localization error is minimum at finger pad and gradually increase towards proximal and palm.

(Schady & Torebjork, 1983) - point localization and Wheat, et al - tactile localization experiment on hand - separation threshold to distinguish a point from sphere of radius to the object of same radius. The human receptiveness increases with time. Sensitivity to stimuli increased in 300ms.

Movement and direction - all the stimuli reaching hand are sensed by scanning movements combine lateral movement and skin stretch.

Flutter vibration - experiments on frequency change detections and change detection thresholds.

An architecture for gesture-based control of mobile robots

Authors:

Iba, S.

Weghe, J. M. V.

Paredis, C. J. J.

Khosla, P. K.

Summary:

Goal: Gesture recognition system to interact with Robots. More appropriately gesture spotting with HMM.

Hardware used : Cyberglove + 6DOF location sensor.

Feature set: 18 sensor data is reduced to 10 dimension vector and each dimension taken with it derivatives increases the dimension of the vector to 20. This 20 dimension vector is coded on to a 32 bit integer. This codeword is then sent into HMM for recognition. HMM is trained with 5000 postures from full hand posture space. Restricting the observation sequence to better quantification. Wait state is included to provide a method to reject invalid gestures.

Gesture set : opening, opened, closing, pointing, waving left and waving right. These gestures carry different semantics in local mode and global mode of robot control.

Discussion:

Interesting extension to HMM for rejecting invalid gestures. Are gestures better than joystick controls while interacting with robots?

Comments: Drew, Franck

Human-centered interaction with documents

Authors:

Andreas Dengel

Stefan Agne

Bertin Klein

Achim Ebert

Matthias Deller

Summary:

Goal: Combining 3D visualization techniques and hand gesture recognition to improve interaction with documents.

Visualization:

Well designed virtual reality like graphical document explorer in 3D. The documents are represented in the form of books in the book case. Search bar is invoked using a gesture and the documents related to the query are displayed with higher semantic zoom and greater detail than book case. The system provides 2 modes of visualization. Plane mode - matrix of documents. Color coding - yellowness for the oldness of the document, animating pulsing behavior to state the importance, thickness to show the size of the document and the thumbnail shows the first page of the document.

Cluster Mode - 3d visualization of the related documents. The relations are visualized by the colored flashing lines connecting the documents.

Gesture recognition - a gesture recognition system which would allow definition of gestures, take in training data and provide recognition results. The system contains 2 threads - data collection and gesture manager. Data collection thread generates events based on the glove movements - glove move, posture changed,... . Gesture manager receives the events from the data collection thread and is responsible for the gesture recognition.

Discussion:

The document moving gesture and the pointing gesture were intuitive. Other gestures for opening the document and returning the document to the grid did not seem to be intuitive. The paper claims to have several ways of interacting with a document but did not mention it how/ what those are. There is no user study. I could not understand some parts of the visualization concepts used in the system like relation visualization, occlusion due to the links and green box to solve occlusion. A systematic user study would have helped in seeing what the original problems are.

I thought i would get some ideas on the gestures users tend to use in 3D environments in the paper.

Comments: Franck