Distant freehand pointing and clicking on very large, high resolution displays

Authors:

Daniel Vogel

Ravin Balakrishnan

Summary:

In a large display, we cannot assume the fixed spatial relationship between the user and the screen. The desirable design characteristics involve accuracy in pointing, pointer acquisition speed, pointing and selection speed, comfortable usage and smooth transition across interaction distances. Clicking tenchniques evaluated - index finger "Airtap" and thumb "in and out" gestures. Airtap technique is similar to the index finger movement in mouse click. Two challenges involved in the airtap are determining the up and down position of the click and the different styles involved in clicking by different users. The absolute position of the fingers cannot be used to determine the start and stop of click events since there start position of the finger is not fixed. So in addition to the position of the index finger, the velocity and the acceleration of the finger is used to detect the click events. A simple calibration was used to distinguish the click from the involuntary finger movements and reduce the effect of different user styles. 5 seconds of finger movement is recorded, the principle direction is calculated and distance moved in 200ms is compared with threshold distances to find the click events. Visual and auditory feedback is used to replace the lost kinesthetic feedback of button click.

Thumbtrigger has the advantage of having the kinesthetic feedback ( a click happens when the thumb touches the side of the palm). But this proved to be difficult for the users.

Pointing techniques - raycasting with absolute pointing, relative pointing with clutching and combination of both. Finger raycasting was used to point to object on screen. The index finger pointing direction was used. Jittery cursor movement is the disadvantage of the method. The jitters are caused due to fatigue. Palm as pointer, kalman filter, dynamic recursive low pass filters are some techniques used to reduce the jitters. The safe hand posture is used to point and clutching gesture is used to select a object in relative pointing technique. In the hybrid technique, the index finger is used to point to an approximate area (absolute pointing, cursor - circle) then the palm is used to move the cursor relatively to the correct position (cursor - cross). Clutching gesture is then used to select objects.

Results:

A transitive task and a sequence task were used in the user study to analyze the pointing techniques. Using Raycasting, small corrective movements to select small objects were difficult. So it consumed more time than the other relative pointing technique. Higher error rates were found in selecting large targets using raycasting. Recalibration time in relative techniques - Users performed short and quick recalibration by clutching independent of the distance to the object. In case of raytorelative, users performed very less recalibration since the recalibration involved large overhead. Longer recalibration time did not affect the overall selection time in raytorelative pointing technique.

Discussion:

Fatigue plays a significant role in this big screen interaction technique. Modeling the systems where such big screen interactions are involved would help in understanding the length of interactions. This would help understand the different actions performed with the interface.

I think the pointing gestures used in RaytoRelative is counter intuitive. Index finger is usually used to point accurate regions and hand wave and clutch to mark approximate regions in big space. The interface seems to have the opposite roles for the gestures.

Comments:

1. Kevin Kwan's Blog

2. Josh Peschel 's blog