Snap-Crackle-Pop: Tactile Feedback for Mobile Touch Screens
`
`Topi Kaaresoja'
`Nokia Research Center
`Helsinki, Finland
`
`Lorna M. Brown?
`University of Glasgow
`Scotland, UK
`
`Jukka Linjama®
`Nokia Technology Platforms
`Helsinki, Finland
`
`ABSTRACT
`
`This paper describes a mobile device demonstrating tactile
`feedback for touch screens. The demonstrator includes four
`different applications: keypad, text selection, scrolling, and drag
`and drop.
`
`Keywords: mobile devices,tactile feedback, touch screen
`
`1
`
`INTRODUCTION
`
`The small screens on mobile devices and the fact that they are
`used in dynamic mobile contexts mean that visual feedback
`alone is often insufficient. Current touch screen devices often
`feature audio feedback in response to user actions, but tactile
`feedback is rarely included. Adding tactile feedback to touch
`screens has the potential
`to improve usability by providing
`additional feedback to the user, e.g. confirming that a button has
`been pressed. This may be particularly beneficial
`in noisy
`environments, where audio feedback would not be heard.
`In
`addition, tactile feedback could improve the user experience by
`makingvirtual widgets feel morelike real physical widgets.
`This paper presents a mobile device with a touch screen
`enhanced with
`tactile
`feedback,
`and
`discusses
`four
`demonstration applications — keypad, text selection, scrolling,
`and drag and drop.
`
`2 RELATED WORK
`
`serial cable) for transferring the data related to the touch
`screen. The applications are written in Java.
`Off-the-shelf piezo actuators are located undera resistive
`touch screen, and providetactile feedback to a stylus or finger
`using the device and to the holding hand. The details of the
`technology of this demonstrator are beyond the scope ofthis
`paper, which focuses on the user interface elements of the
`system.
`feedback has been added to four different
`Tactile
`applications: numerical keypad, text selection, scrolling, and
`drag and drop. Based on Poupyrev’s finding that
`tactile
`feedback is most useful when used to enhance gestural
`interactions, three out of these four applications use gestural
`interaction; only the keypad application uses a simple tapping
`interaction. Within each application it is possible to switch the
`different types of tactile feedback on or off, and to customize
`the strength ofthe tactile feedback.
`The applications presented in this paper are based on the
`previous work, but also extend the scope of what has
`previously been done. For example, the text selection demo is
`based on the text selection design by Poupyrev, but features
`tactile feedback for line selection in addition to character
`selection. The scrolling demo is based on the speed dependent
`scrolling application by Poupyrev, but the interaction uses
`touch screen gestures instead oftilting. Finally an entirely new
`application of tactile feedback is presented, with the addition
`of tactile feedback to the different phases of the gestural drag
`
`and drop interaction.
`
`Research on tactile feedback for touch screens has previously
`been carried out by a numberof researchers. Fukumoto [1] used
`a voice coil actuator mounted in the body of a device to provide
`single pulse tactile feedback to the finger when buttons on a
`numerical keypad are pressed. Nashel and Razzaque[2] used a
`vibration motor from a tactile mouse to provide information to
`users about the buttons on a PDA screen. Poupyrevetal. [3, 4]
`have carried out extensive research in this area. They added
`proprietary piezo actuators between the touch screen and the
`resistive touch panel. They addedtactile feedback to multiple
`GUI elements and introduced a structure for gestures in touch
`screen. They also introduced speed dependenttactile feedback
`for scrolling a document by tilting a device. Although the
`continuation of their work focuses on desktop touch screens[5],
`it is still relevant for mobile devices. They conducted a tapping
`study with a stylus, and introducedatext selection feature
`enhanced with tactile clicks for each character. They also found
`that tactile feedback is most effective when combined with an
`active gesture, such as dragging, rather than just being used to
`enhance a simple tap.
`
`Figure 1 The demonstrator device, displaying the keypad demonstration
`application.
`
`4 KEYPAD DEMONSTRATION
`
`The keypad demonstration application shownin Figure 1 is a
`virtual ITU-T number keypad with the device in portrait
`
`APPLE-1028
`
`3 DEMONSTRATION
`
`The demonstrator is a proprietary mobile device, called “Snap-
`Crackle-Pop”, which is connected to a laptop via two cables,
`one for transferring the display to the device and the other (a
`
`e-mail: topi.kaaresoja@nokia.com
`e-mail: lorna@dcs.gla.ac.uk
`e-mail: jukka.linjama@nokia.com
`
`123
`
`1
`
`APPLE-1028
`
`

`

`4.3
`
`Drag and Drop Demonstration
`
`4.1
`
`Text selection Demonstration
`
`mode. The buttons are designed to be pressed with a thumb or a
`stylus. When a button is pressed the color of the button changes
`Tactile feedback was added to drag and drop operations in
`andatactile click is presented; when it is released the color
`order to improve targeting, enhance the feel of the “dragging”
`changesbackto the original and secondtactile click is presented
`action, and help to confirm useractions by providing different
`to the user. This tactile feedback is designed to simulate the real
`tactile feedback when a documentis hit to or dropped onto
`tactile feedback experienced whenpressing a physical button.
`different items.
`In the demonstration there are three graphical
`icons on a desktop in this demonstration application. One of
`them, a text document, is movable and the user can pick it up
`by holding the stylus on it for a while in the similar manner
`than entering the scrolling mode in previous demonstration.
`The user can drag the icon by moving the stylus while holding
`it on the screen.
`Seven different types of tactile feedback are presented to the
`user in this application for the following actions (Figure 4):
`
`In the text selection demonstration a text documentis displayed
`on the screen (Figure 2) and tactile feedback is presented to the
`user whentext is selected. The device is used in landscape mode
`for this application. When the user selects the text by dragging
`horizontally, a gentle tactile click is presented as each character
`is selected. If the user drags vertically to select line by line, a
`stronger tactile click is presented for every line indicating that
`more text has been selected. The addition oftactile feedback to
`the text selection task is intended to improve targeting and
`enable moreprecise textselection.
`
`corper suscipit lobortis nisl
`having beenlighted indeed since my father’s funeral and when
`
`Lorem ipsum dolor sit amet,
`feleetr-ctetuer adipiscing elit,
`ised diam nonummy nibh euismod
`tincidunt ut laoreet dolore
`
`magna aliquam ecrajpaenkthsyl ammmels
`wisi enim ad minim veniam, quis
`nostrud exerci tation ullam-
`
`Figure 2 Text selection demonstration with some text selected.
`
`went.
`‘Mrs. David Copperfield, | think,sald Miss Betsey, the emphasis
`referring, perhaps, to my mother's mourning weeds, and her
`condition
`"Yes,said my mother,faintly.
`‘Miss Trotwood,’ said thevisitor.
`
`‘You have heard of her, | dare
`
`My mother answered she had had that pleasure. And she had a
`disagreeable consciousnessof not appearing toimply that it had
`been an overpowering pleasure
`‘Nowyou see her, said Miss Betsey. My mother bent her head, and
`begged herto walkin
`Theywentinto the parlour my mother had comefrom, the fire in the
`best roomon theother side ofthe passage not being lighted - not
`
`NAYSeb
`
`Whenthe user picks up the text documenticon.
`Whenthe user drags the object on the desktop.
`Whenthe userhits the folder object
`Whenthe userhits the word processor object
`Whenthe user drops the icon onto the desktop
`Whenthe user drops the icon into the folder
`When the user drops the icon onto the word
`processor application and the application opens the
`text document.
`
`5 CONCLUSIONS
`
`device
`“Snap-Crackle-Pop”
`the
`describes
`paper
`This
`demonstrating tactile feedback in touch screen applications for
`mobile devices. The addition of tactile feedback to mobile
`device touch screens has the potential to improve both the
`usability and user experience of such devices. Future work
`will
`investigate the effect of this tactile feedback on user
`performance and experience in the different tasks featured in
`
`this demonstration.
`
`©
`—=6)oe‘older.In!
`=i _———— ©
`@
`doc.doc.InkNC wy
`2)
`Microsoft Word.Ink
`
`Figure 3 The scrolling application. The path of the stylus is shown with
`dashedline. The speed ofthe scroll is propotional to the distance (d).
`
`Figure 4 The cases whenthe usergets different tactile feedback in
`drag & drop application.
`
`4.2
`
`Scrolling Demonstration
`
`In the scrolling demonstration speed dependent scrolling is
`enhanced with tactile feedback in order to indicate the current
`scrolling speed to the user non-visually. In the demonstrator
`application a large document is displayed on the screen. The
`user can enter the scrolling mode by pressing the stylus onto the
`centre of the screen and holdingit still until tactile feedback is
`presented. This tactile feedback indicates that the user has
`entered the scrolling mode. The user can then scroll up and
`down through the document by movingthe stylus up and down,
`respectively, while holding the stylus on the screen. The
`scrolling speed is propotional to the distance of the stylus from
`the initial point (Figure 3). As the user scrolls through the
`document tactile clicks are presented, with the frequency of
`occurenceofthe clicks being proportionalto the scrolling speed.
`The user can leave the scrolling mode by simply lifting the
`stylus off the screen. When the scrolling mode is not engaged
`this application features the tactile feedback from the text
`selection demo.
`
`ACKNOWLEDGEMENTS
`
`The authors would like to thank Jukka Jalkanen and Risto
`Suitiala for programming the demonstrations, and Juhani
`Tuovinen for building the hardware.
`
`REFERENCES
`
`[3]
`
`[4]
`
`[1] M. Fukumoto and T. Sugimura, "Active Click: Tactile Feedback for
`Touch Panels," presented at CHI2001, 2001.
`[2] A. Nashel and S. Razzaque, "Tactile Virtual Buttons for Mobile
`Devices," presented at CHI2003, Ft. Lauderdale, Florida, USA, 2003.
`I. Poupyrev and S. Maruyama,"Tactile interfaces for small touch
`screens," presented at UIST 2003, 2003.
`I. Poupyrev, S. Maruyama,and J. Rekimoto, "Ambient Touch:
`Designingtactile interfaces for handheld devices," presented at UIST
`2002, 2002.
`I. Poupyrev, M. Okabe, and S. Maruyama, "Haptic Feedback for Pen
`Computing: Directions and Strategies," presented at CHI 2004,
`Vienna, Austria, 2004.
`
`[5]
`
`2
`
`