Minimizing modes for smart selection in sketching/drawing interfaces

User interface modes are ubiquitous in both mouse-keyboard and pen-based user interfaces, but the requirement for prior setting of a user interface mode before performing an action imposes a persistent drag on system usability. This chapter reviews our research in approaches to avoiding prior deliberate mode setting while still allowing overloading of fundamental tap and gesture operations. We analyze the human-machine dynamics of UI protocols through a graphical notation called the Interaction Flow Diagram. Our framework offers a pyramid of methods ranging from simple UI design techniques, through recognition of gestures and canvas content, to modeling of user knowledge and goals. These are represented in four methods: Overloaded Loop Selection to infer rectangle versus lasso selection mode; the Inferred Mode Protocol for Inferring Draw/Select Mode; the Sloppy Selection method for inferring intended content of an ambiguous selection; and the Cycle Tap Selection Method for exploiting structure recognition.

Citation

Saund, E.; Lank, E. Minimizing modes for smart selection. Chapter in Sketch-based Interfaces and Modeling, edited by Joaquim Jorge and Faramarz Samavati. London: Springer; 2011; 55-80.