Tactile Feedback for Mobile Interactions
`
`Stephen Brewster, Faraz Chohan and Lorna Brown
`Glasgow Interactive Systems Group, Department of Computing Science
`University of Glasgow, Glasgow, G12 8QQ, UK
`stephen@dcs.gla.ac.uk www.dcs.gla.ac.uk/~stephen
`
`
`ABSTRACT
`We present a study investigating the use of vibrotactile
`feedback for touch-screen keyboards on PDAs. Such key-
`boards are hard to use when mobile as keys are very small.
`We conducted a laboratory study comparing standard but-
`tons to ones with tactile feedback added. Results showed
`that with tactile feedback users entered significantly more
`text, made fewer errors and corrected more of the errors
`they did make. We ran the study again with users seated on
`an underground train to see if the positive effects trans-
`ferred to realistic use. There were fewer beneficial effects,
`with only the number of errors corrected significantly im-
`proved by the tactile feedback. However, we found strong
`subjective feedback in favour of the tactile display. The
`results suggest that tactile feedback has a key role to play in
`improving interactions with touch screens.
`
`Author Keywords
`Tactile icons, Tactons, touch-screen buttons, mobility.
`
`ACM Classification Keywords
`H5.2. [User Interfaces]: Haptic I/O.
`
`INTRODUCTION
`This paper presents a study into the use of tactile feedback
`for an on-screen PDA keyboard where a stylus (or finger) is
`used to press the keys. Entering text on such keyboards is
`problematic as the keys are small (less than 1cm2 on a PDA
`such as in Figure 1). Trying to do this whilst mobile makes
`interaction even harder as the PDA and stylus are both
`moving. Particularly difficult situations are on buses or
`trains, which can be very bumpy, yet these are situations
`where people often want to read/send email, browse Web
`sites, etc. on their way to work. The aim of our work was to
`look at the effects of tactile feedback from key presses with
`a stylus to see if performance could be improved.
`
`BACKGROUND
`There have been some good examples of the use of tactile
`displays to improve desktop interfaces. For example,
`Mackenzie and others have successfully shown that tactile
`
`
`Permission to make digital or hard copies of all or part of this work for
`personal or classroom use is granted without fee provided that copies are
`not made or distributed for profit or commercial advantage and that copies
`bear this notice and the full citation on the first page. To copy otherwise,
`or republish, to post on servers or to redistribute to lists, requires prior
`specific permission and/or a fee.
`CHI 2007, April 28–May 3, 2007, San Jose, California, USA.
`Copyright 2007 ACM 978-1-59593-593-9/07/0004...$5.00.
`
`
`feedback can improve pointing interactions [1] when using
`a mouse. Tactile cues can aid users in hitting targets such as
`buttons faster and more accurately.
`
`
`
`Figure 1: A typical on-screen keyboard on an HP iPAQ PDA.
`
`However, most research in the area has focused on the de-
`sign of tactile actuators; until recently there were few tactile
`actuators routinely available and they were often designed
`for use in different domains (e.g. sensory substitution sys-
`tems). Lee et al. [7] developed a tactile stylus to use on
`touch screens and PDA’s. Poupyrev et al. and Luk et al. [8,
`9] have designed sophisticated tactile displays for handheld
`computers. Luk et al. have begun to look at interactions
`using their devices but none have been formally studied so
`there is little evidence that tactile displays are beneficial in
`practical situations.
`
`Brewster and King [3] designed a tactile progress bar that
`indicated the progress of a download via the time difference
`between two tactile pulses; as the pulses got closer together
`the download got closer to completion. They found that
`users performed better with tactile progress bars than stan-
`dard visual ones when involved in a visual typing task. Us-
`ers were able to attend to the tactile feedback and type at
`the same time. In their experiment, the tactile actuator was
`on the user’s wrist, but users were not mobile.
`
`Brewster also looked at the benefits from adding sound to
`buttons for mobile interactions [2]. The aim was to over-
`come problems of contention for visual attention, where
`users must look where they are going when walking and
`cannot devote so much attention to the visual display. He
`found that sounds increased the amount of data people
`could enter on a PDA whilst walking and reduced subjec-
`tive workload. We based the design of our vibrations on
`these sounds.
`
`There has been very little work on the use of tactile displays
`in mobile settings. Many mobile devices already have vi-
`
`1
`
`APPLE-1029
`
`

`

`
`
`brotactile actuators built-in but they are little used in most
`interactions. The vibrations that accompany a mobile phone
`ringing are useful and alert the user to the call even if
`he/she cannot hear the audio. One problem can occur if
`users cannot feel the feedback (their phone maybe in a bag
`for example) but this is not the case with keyboard interac-
`tions as the user will be holding the device and so be in
`contact with the vibration.
`
`The aim of our work is to see if tactile displays can offer
`other benefits for touch-screen devices. This paper presents
`two studies, one seated and one mobile, to investigate text
`entry on touch-screen displays. Text entry is a common
`activity and is based on button pressing, one of the most
`basic interaction techniques of all.
`
`REQUIREMENTS GATHERING
`The first stage of our work was to find out what kinds of
`errors people make when entering text on a PDA. We chose
`to investigate the interaction on an underground train as
`people use PDAs and phones on trains and buses every day
`whilst commuting. The underground is a good platform for
`testing as noise levels are very dynamic, being quiet when
`stopped at a station, but very noisy when the train is in mo-
`tion. Light levels again vary dramatically. Vibration and
`movement are also very changeable. When the train is
`stopped there is little vibration. However, when it acceler-
`ates and decelerates people are subjected to lots of forces
`and vibration from the engine and general movement. An-
`other important factor is that it is a safe environment as
`participants can be seated. Others have looked at testing
`whilst walking [2]. There are ethical issues as participants
`can trip or fall whilst taking part. All of these characteristics
`make it an interesting and realistic place to run studies.
`
`We designed a simple interface to allow users to enter text
`using an on-screen keyboard on an iPAQ (copying the dis-
`play seen in Figure 1). We gave participants short messages
`to type in and logged the kinds of errors and slips made. We
`encouraged them to be as fast and accurate as possible.
`There were six participants, all male students from the Uni-
`versity (aged 19-26) and familiar with mobile devices. The
`study took around 10 minutes per person.
`
`Results and Discussion
`We saw a range of different errors occurring. One of the
`reasons was the competing attention demands of looking at
`the keys, the text entry window and the text to be entered.
`Analysis of keystroke logs showed a variety of problems:
`
`Wrong letters: There were many examples of the wrong
`letters entered. Not all of these were caused by train move-
`ments, some were just mistakes. These are hard to detect
`and support with extra feedback as users may just tap the
`wrong key by mistake.
`
`Slips: We noticed a number of slips, where the user put the
`stylus down on one letter and lifted it on another (the effect
`
`being no character is entered). Most slips occurred off the
`bottom or to the left of a key.
`
`Double taps: We found that accidental double taps were
`common. This is due to the movements of the train, PDA
`and stylus, with the stylus bumping into the screen.
`
`Many of the errors made were not corrected, again perhaps
`due to the attentional demands of the different parts of the
`display and the interference from the environment.
`
`Our aim was to see if tactile feedback could overcome these
`problems. Audio feedback would be difficult in such an
`environment as it is very noisy; an earpiece would have to
`be worn. Visual feedback would be difficult as there is al-
`ready much to look at and the screen is small. The tactile
`actuators in many PDAs and phones are not used unless a
`call is being received, so could provide extra feedback.
`
`EXPERIMENT 1: TACTILE DISPLAY IN THE LAB
`A first study investigated the part that tactile feedback
`might play in stationary interactions, which we tested in a
`laboratory. This would allow us to set a baseline of per-
`formance that we could then compare to a mobile setting.
`
`The iPAQ we used for the study did not include a vibrotac-
`tile actuator so we added an external EAI C2 tactor
`(www.eaiinfo.com). This was connected via the headphone
`jack (see Figure 2). We placed it at the top right corner of
`the iPAQ so that the index finger of a right-handed user
`would rest on it. Other locations are possible, but for this
`study we were most interested in whether tactile feedback
`aided interaction, rather than studying actuator placement.
`
`
`
`
`
`Figure 2: The vibrotactile actuator on the back of an iPAQ.
`
`We used simple Tactons (tactile icons) [4] to represent dif-
`ferent aspects of the button interaction. We kept the tactile
`design as simple as possible as keyboard interactions are
`fast and we needed our feedback to keep pace. We used two
`stimuli: one to indicate a successful button press and one to
`indicate an error. The success Tacton was played when a
`button was correctly pressed and then released. The error
`Tacton was played when a slip or double tap error occurred.
`
`The design of the feedback was based on audio feedback
`added to buttons by Brewster [2]. We used an 800 ms.
`250Hz sine wave success cue, and a rough (amplitude
`modulated) sine wave for the error cue. 250Hz is in the
`range of greatest sensitivity of the skin [6] and the fre-
`quency at which the device resonates, giving the greatest
`power output. Brown et al. [4] showed that amplitude
`modulation is felt as ‘roughness’ and can provide a cue that
`
`2
`
`

`

`is recognizably different to a ‘smooth’ sine wave, without
`taking any longer to play. These cues played as sound files
`through the tactile actuator.
`
`We used 12 right-handed participants, all students from the
`University with no experience of touch screens. The study
`took place in a usability lab with participants seated, hold-
`ing the iPAQ in their left hand. We used a within subjects
`design, with participants using both standard, visual buttons
`(Standard condition) and buttons with tactile feedback
`added (Tactile condition) in a counterbalanced order for 10
`minutes each. A brief training phase preceded each condi-
`tion to familiarise participants with the interface. Partici-
`pants were given poems to type in and were asked to enter
`the text as fast and as accurately as possible. The software
`was similar to that used in the requirements capture, with a
`soft keyboard at the bottom of the screen and a text area at
`the top. Dependent variables were the amount of text en-
`tered, the total number of errors made (characters that were
`not in the poem) and the number of errors that were uncor-
`rected by users.
`
`Results and Discussion for the Laboratory Study
`The results are shown in Figure 3. A T-test showed that
`there was a significant difference in the number of lines of
`text entered (T11=6.28, p<0.001) with more entered in the
`Tactile condition. Significantly more errors were made in
`the Standard condition (T11=2.66, p=0.02) and significantly
`more were corrected (T11=4.10, p=0.001) in the Tactile.
`
`Figure 3: Results from the laboratory and mobile studies (with
`standard error bars shown).
`
`The results show that with tactile feedback participants
`were generally performing much better: entering more text,
`making fewer errors and noticing more of the ones they did
`make. We suggest that the tactile feedback generally in-
`creased their awareness of mis-hit keys so that they could
`go back and correct them. There is room for improvement
`as they still missed some errors and better tactile feedback
`might bring this number down (although some of these are
`likely to be ‘wrong letter’ errors that we could not give ex-
`tra feedback to support). The vibrations from the tactile
`feedback did not affect typing in a negative way as partici-
`pants entered more text in the Tactile condition.
`
`EXPERIMENT 2: TACTILE DISPLAYS ON THE MOVE
`We ran the same experiment again but this time users were
`seated on a train on the Glasgow city underground. This
`allowed us to assess the effects of tactile feedback in a more
`realistic scenario, and if the benefits observed in the labora-
`tory would carry over to the real world. We again used a
`within-subjects design to compare standard keyboard but-
`tons to ones which we added tactile cues. The procedure
`and stimuli used in the experiment were as before to allow a
`comparison of the results. We used six new participants,
`again students from the University.
`
`Participants sat in a seat on the underground train next to
`the experimenter who held the poem sheets (Figure 4). This
`time we also administered NASA TLX workload sheets
`after each condition [5]. We added an extra category of
`Annoyance to see how people felt about the extra feedback
`they received and whether the vibrations bothered them.
`
`
`
`Figure 4: The experimental setup on the underground train.
`
`Results and Discussion for the Mobile Study
`Formal statistical analysis is limited due to the small num-
`ber of participants, but gives some indication of where ef-
`fects lie. The number of lines of text entered was not sig-
`nificantly different between the two conditions (T5=0.34,
`p=0.74), neither was the total number of errors made
`(T5=1.54, p=0.18). There was, however, a significant dif-
`ference in the number of uncorrected errors (T5=3.06,
`p=0.02), with more being corrected in the Tactile condition
`(as in the lab study). Figure 3 shows the results.
`
`
`
`Results show that tactile feedback was less beneficial when
`users were mobile. The variations introduced by the envi-
`ronment (the underground generates much vibration)
`masked small benefits found in the lab (and the small num-
`ber of participants will have caused more variance in the
`data). We did still see an effect for the number of uncor-
`rected errors; more mistakes were still missed in the visual
`condition. This suggests that the extra feedback was still
`useful as correcting errors made is critical.
`
`Figure 5 shows the results of the TLX questionnaires.
`Overall workload was significantly reduced (T5=5.14,
`p=0.003). A more detailed analysis showed significant re-
`ductions in workload in the Tactile condition for Mental
`Demand, Physical Demand, Effort Expended and Frustra-
`tion (all with p<0.01). There was a significant increase in
`
`Standard (lab)
`Tactile (lab)
`Standard (train)
`Tactile (train)
`
`Number of lines entered
`
`Total errors
`
`Number of errors
`uncorrected
`
`60
`
`50
`
`40
`
`30
`
`20
`
`10
`
`0
`
`Average Score
`
`3
`
`

`

`
`
`perceived Performance Level (p<0.001) for the Tactile con-
`dition. There was no difference in terms of Time Pressure
`(p=0.2). This is unsurprising as there were no differences
`between the two conditions in this respect. Annoyance was
`also found to be significantly reduced in the Tactile condi-
`tion (p=0.006).
`
`Figure 5: NASA TLX results for the mobile study.
`
`The qualitative workload results show participants were
`strongly in favour of the Tactile condition, reducing almost
`all of the workload factors. This combined with the quanti-
`tative results shows that tactile feedback for touch-screen
`displays is beneficial in real mobile settings.
`
`Comparing the results here to the laboratory study we can
`that the shapes of the graphs are broadly similar. There
`were 22.4% more errors made in the standard condition
`than the tactile when in the lab and 25.7% more when mo-
`bile. However, 48.3% more errors corrected in the tactile
`condition than the standard when in the lab but 66.9% more
`were corrected when mobile (the mean number of uncor-
`rected errors in the tactile condition in the lab was 17.3 but
`only 7.1 when mobile). This suggests that the tactile feed-
`back was even more beneficial for error correction in the
`mobile situation, giving participants useful information
`amongst all of the noise and vibration of the train. It is im-
`portant that errors that are made are corrected; ideally fewer
`errors would occur, but if they do occur then it is crucial
`that the user notices and corrects them otherwise incorrect
`messages could be sent.
`
`The results for the mobile study match some of those Brew-
`ster found with sonically-enhanced buttons when tested on
`the move (in that case with users walking) [2]. For example,
`he also found more data was entered when extra feedback
`was given. Another similarity was a large reduction in
`workload with the extra feedback when users were mobile.
`This suggests that touch-screen buttons are hard to use in
`mobile settings and users benefit when they are given extra
`assistance. The advantage of tactile over sound is that it can
`be given even in noisy environments. A future study will
`directly compare audio, tactile and a combination of the two
`feedback types to see which is most beneficial.
`
`
`
`View publication stats
`
`CONCLUSIONS
`The studies presented here have shown that tactile feedback
`provides significant benefits for keyboard interactions on
`touch-screen devices, both in static situations and more
`dynamic, mobile ones. Such feedback is likely to help all
`button interactions on touch-screens, not just text entry,
`which would be a considerable benefit as buttons are very
`common. Giving tactile feedback via the device rather than
`the stylus also means that users would get the benefits even
`if using a finger to press the buttons.
`
`Brewster [2] showed that sonic enhancement of buttons
`could improve performance. The downside of his solution
`was that sounds could be intrusive or not heard in noisy
`environments. Tactile feedback is an effective alternative
`and does not suffer the same drawbacks. A key recommen-
`dation from this work is for PDA and smart phone design-
`ers to use tactile feedback in more of the interactions with
`their devices as an easy way to improve usability.
`
`
`
`ACKNOWLEDGMENTS
`This work was funded EPSRC Advanced Research Fellow-
`ship GR/S53244.
`
`REFERENCES
`1. Akamatsu, M., MacKenzie, I.S. and Hasbrouq, T. A
`comparison of tactile, auditory, and visual feedback in
`a pointing task using a mouse-type device. Ergonom-
`ics, 38. 816-827.
`2. Brewster, S.A. Overcoming the Lack of Screen Space
`on Mobile Computers. Personal and Ubiquitous Com-
`puting, 6 (3). 188-205.
`3. Brewster, S.A. and King, A.J., An Investigation into
`the Use of Tactons to Present Progress Information. In
`Proc. of Interact 2005, Springer, 6-17.
`4. Brown, L.M., Brewster, S.A. and Purchase, H.C., A
`First Investigation into the Effectiveness of Tactons. In
`Proc. of Worldhaptics 2005, IEEE Press, 167-176.
`5. Hart, S.G. and Wickens, C. Workload assessment and
`prediction. In Booher, H.R. ed. MANPRINT, an ap-
`proach to systems integration, Van Nostrand Reinhold,
`New York, 1990, 257-296.
`6. Klatzky, R. and Lederman, S. Chapter 6: Touch. In
`Healy, A. and Proctor, R. eds. Handbook of Psychol-
`ogy Volume 4: Experimental Psychology, John Wiley
`and Sons, 2003, 147-176.
`7. Lee, J.C., Dietz, P., Leigh, D., Yerazunis, W. and Hud-
`son, S.E., Haptic Pen: A Tactile Feedback Stylus for
`Touch Screens. In Proc. of ACM UIST 2004, ACM
`Press Addison-Wesley, 291-294.
`8. Luk, J., Pasquero, J., Little, S., MacLean, K.,
`Levesque, V. and Hayward, V. A role for haptics in
`mobile interaction. In Proc. ACM CHI 2006, ACM
`Press Addison Wesley, 171-180.
`9. Poupyrev, I. and Maruyama, S., Tactile Interfaces for
`Small Touch Screens. In Proc. of UIST 2003, ACM
`Press, 217-220.
`
`Standard (train)
`Tactile (train)
`
`20
`
`15
`
`10
`
`Effort Expended
`Time Pressure
`
`Performance Level
`Frustration
`
`Annoyance
`
`05
`
`Mental Demand
`Physical Demand
`
`Average Score
`
`4
`
`