Visual Similarity of Pen Gestures

Bibliographical Information:
Long, Landay, Rowe, Michiels. "Visual Similarity of Pen Gestures." CHI '00 Proceedings of the SIGCHI conference on Human Factors in Computing Systems. Pages 360-367. ACM New York, NY, USA

URL:
http://dl.acm.org.lib-ezproxy.tamu.edu:2048/citation.cfm?id=332458

This paper provides somewhat of an early look into computational interpretation of sketches. Specifically, the authors gear this research toward one-stroke gestures intended to be used as commands, as was common the year this paper was published. PDA devices with smaller screens relied on one-stroke gestures to perform commands, and while it proved convenient, many felt frustration and confusion when the system failed to interpret their gestures as intended. The paper presents two experiments intended to provide better insight into how humans perceive "similarity" between two gestures. Indeed, one of the more common complaints when a user is presented with a gesture recognition failure is "but my gesture really does look like a triangle!" Providing insight into why and how a human can perceive similarity is what leads to the paper's main motivation, that being the identification of what the paper calls "perceptual similarity", and whether this similarity can be computed empirically. Both experiments involve users being presented with similar shapes, with the user selecting the shape in the set that is the least similar. Computationally, the authors generated a set of gesture features that played the most vital role in determining how something can be perceived as "similar" or "different" based on these responses. The first experiment involved the user picking between a set of three gestures, called triads. All triads were seen exactly once. The second experiment involved three new gesture sets of nine gestures each, with those gestures being the point of comparison. The paper presented a computable model to determine this perceptual similarity, which they found to coordinate with reported similarity with a factor of 0.56. The authors found these results to be encouraging and significant, as a computational model was now possible to help determine what could previously only be a subjective analysis.

The paper provides a good foundation for the concept of perception and what would eventually become geometric recognition. I did find some issues with the fact that none of these gestures compared were drawn by users, but rather picked as shapes from a menu of choices. While the significant reduction in experiment time was important, the task being observed here is one of simple image comparison, which is very different from the actual practice of a user's gesture and how a computer would perceive it. Additionally, even though subjectivity is important to capture, relying on pure subjective analysis between individual users to then generate a computational formula to replicate it would only reflect the differences in perceptions of that particular group at best. However, this paper still did provide a significant first step toward computationally determining similarity as it would be reported by a human being.