Human-centered interaction with documents

Authors:

Andreas Dengel

Stefan Agne

Bertin Klein

Achim Ebert

Matthias Deller

Summary:

Goal: Combining 3D visualization techniques and hand gesture recognition to improve interaction with documents.

Visualization:

Well designed virtual reality like graphical document explorer in 3D. The documents are represented in the form of books in the book case. Search bar is invoked using a gesture and the documents related to the query are displayed with higher semantic zoom and greater detail than book case. The system provides 2 modes of visualization. Plane mode - matrix of documents. Color coding - yellowness for the oldness of the document, animating pulsing behavior to state the importance, thickness to show the size of the document and the thumbnail shows the first page of the document.

Cluster Mode - 3d visualization of the related documents. The relations are visualized by the colored flashing lines connecting the documents.

Gesture recognition - a gesture recognition system which would allow definition of gestures, take in training data and provide recognition results. The system contains 2 threads - data collection and gesture manager. Data collection thread generates events based on the glove movements - glove move, posture changed,... . Gesture manager receives the events from the data collection thread and is responsible for the gesture recognition.

Discussion:

The document moving gesture and the pointing gesture were intuitive. Other gestures for opening the document and returning the document to the grid did not seem to be intuitive. The paper claims to have several ways of interacting with a document but did not mention it how/ what those are. There is no user study. I could not understand some parts of the visualization concepts used in the system like relation visualization, occlusion due to the links and green box to solve occlusion. A systematic user study would have helped in seeing what the original problems are.

I thought i would get some ideas on the gestures users tend to use in 3D environments in the paper.

Comments: Franck