as) United States
`a2) Patent Application Publication 10) Pub. No.: US 2012/0319959 Al
`
` Saponaset al. (43) Pub. Date: Dec. 20, 2012
`
`
`US 20120319959A1
`
`(54) DEVICE INTERACTION THROUGH
`BARRIER
`
`(75)
`
`Inventors:
`
`T. Scott Saponas, Woodinville, WA
`(US); Christopher Harrison,
`Pittsburgh, PA (US); Hrvoje Benko,
`Seattle, WA (US)
`°
`
`(73) Assignee:
`
`MICROSOFT CORPORATION,
`Redmond, WA (US)
`
`(21) Appl. No.:
`
`13/159,441
`
`(22)
`
`Filed:
`
`Jun. 14, 2011
`
`Publication Classification
`
`(51)
`
`Int. Cl.
`(2006.01)
`G06F 3/041
`(52) U.S. C1. cc ceeeseseeesersereeceeceeceeceeceeeneenes 345/173
`(57)
`ABSTRACT
`.
`.
`.
`.
`.
`.
`There is provided an electronic device having a touch-sensing
`element configured for sensing touches on a surface thereof.
`A baseline sensitivity setting determines a sensitivity of the
`touch-sensing element. The touch-sensing elementis config-
`ured to register a touch that meets or exceeds the baseline
`sensitivity setting, and to ignore a touch that does not meet the
`baseline sensitivity setting. The device further includes a
`sensor that senses an operating condition of the device. A
`memory ofthe device includes code executable by the device
`and configured to adjust the baseline sensitivity setting based
`upon the sensed operating condition.
`
`Interpreting Group Touches
`
`Determining a Use Position
`
`Adjusting the Sensitivity
`
`Detecting Touches
`
`Excluding Invalid Touches
`
`Pre-Processing Valid Touches
`
`Grouping Sequential Touches
`
`402
`
`404
`
`406
`
`408
`
`410
`
`412
`
`414
`
`fs
`
`iO
`
`1
`
`APPLE-1046
`
`1
`
`APPLE-1046
`
`

`

`Patent Application Publication
`
`Dec. 20,2012 Sheet 1 of 4
`
`US 2012/0319959 Al
`
`160
`
`Controller
`
`Processor
`
`150
`
`2
`
`

`

`Patent Application Publication
`
`Dec. 20,2012 Sheet 2 of 4
`
`US 2012/0319959 Al
`
`112
`
`120
`
`Touch Surface Display
`
`160
`
`100
`FIG. 2
`
`
`
`Controller
`
`Processor
`
`150
`
`3
`
`

`

`Patent Application Publication
`
`Dec. 20,2012 Sheet 3 of 4
`
`US 2012/0319959 Al
`
`HT 302
`
`
`304
`
`J —
`
`312
`
`FIG. 3A
`
`FIG. 3B
`
`320
`
`Fil|ftilt
`
`FIG. 3C
`
`FIG. 3D
`
`4
`
`

`

`Patent Application Publication
`
`Dec. 20,2012 Sheet 4 of 4
`
`US 2012/0319959 Al
`
`Determining a Use Position
`
`Adjusting the Sensitivity
`
`Detecting Touches
`
`Excluding Invalid Touches
`
`Interpreting Group Touches
`
`402
`
`404
`
`406
`
`408
`
`410
`
`412
`
`414
`
`Pre-Processing Valid Touches
`
`Grouping Sequential Touches
`
`—~ © iO
`
`5
`
`

`

`US 2012/0319959 Al
`
`Dec. 20, 2012
`
`DEVICE INTERACTION THROUGH
`BARRIER
`
`BACKGROUND
`
`[0001] Modern mobile electronic devices, such as cellular
`phones, smart phones, laptops, andthelike are sophisticated
`computing platformsthat allow users to, for example, make
`phonecalls, listen to music, surf the Web, send and receive
`emails and text messages, and perform various other tasks.
`These devices are often stored in pockets, bags, backpacks,
`their own carrying cases, purses, or other similar carrying
`locations. Thus, in order to interact with the device a user
`typically removes the device from its carrying location in
`order to access even the basic functionality of the device.
`Removing such devices from their carrying locations may be
`inconvenient or even difficult under certain conditions, such
`as whentheuserofthe device is jogging, moving through an
`airport with luggage,or eating. Further, removing the device
`from its carrying location and interacting with the device may
`under certain circumstances be considered impolite or even
`rude.
`
`The method includes determining a use position ofthe elec-
`tronic device, and adjusting the sensitivity level of a touch-
`sensitive surface of the device dependentat least in part upon
`the determined use position. The method further includes
`detecting touches and strokes that occur on the touch-sensi-
`tive surface and which meet or exceed the sensitivity level.
`The method further includes excluding invalid detected
`touches and strokes. The method groups together the valid
`sequential touches and strokes, and interprets the grouped
`valid sequential touches and strokes as an alphanumeric
`string.
`[0006] Yet another embodimentofthe claimed subject mat-
`ter relates to a non-transitory computer-readable storage
`medium that includes modules of instructions that, when
`executed by a processor of an electronic device, cause the
`electronic device to determine an operating condition, and to
`adjust a sensitivity level of a touch-sensing element of the
`device dependent upon the sensed operating condition. The
`instructions further cause the device to detect touches upon
`the touch-sensing element that meet or exceedthe sensitivity
`level, and to exclude invalid detected touches. The instruc-
`tions still further cause the device to group togetherthe valid
`sequential
`touches, and to recognize the grouped valid
`sequential touches as a word or alphanumericstring.
`
`[0002] Moreover, interfacing with a typical mobile elec-
`tronic device to perform evenrelatively simple tasks, such as
`sending a text or email messageor ignoring an incomingcall,
`not only requires removal of the device from its carrying
`location but also may require a fair amount of cognitive and
`manipulative effort. For example, to ignore an incomingcall
`[0007] FIG.1isa functional block diagram ofan electronic
`
`on a typical device, a user may be obliged to remove the
`device that includes one embodimentof a system for device
`device from its carrying location, unlock or wake the device,
`interaction according to the subject innovation;
`look at the screen, select the “ignore” option, and return the
`[0008]
`FIG. 2 isa functional block diagram ofan electronic
`device to its carrying location. Similarly, in order to send a
`device that includes another embodiment of a system for
`brief text message in reply to a received text message a user
`device interaction according to the subject innovation;
`may be obliged to remove the device from its carrying loca-
`[0009] FIGS.3A-3D are diagramsthat show the exemplary
`tion, unlock or wake the device, look at the screen, select or
`processing of certain sequential strokes according to one
`navigate a menu to the “reply” option, enter the reply text,
`embodimentof the subject innovation; and
`select or navigate a menuto the “send” command, and then
`[0010]
`FIG. 4 is a process flow diagram that shows one
`return the deviceto its carrying case. These and many other
`embodiment of a method for interacting with an electronic
`actions require the user to visually examine and physically
`device according to the subject innovation.
`manipulate the device to a non-trivial degree.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`SUMMARY
`
`DETAILED DESCRIPTION
`
`[0003] The followingis a brief summary of subject matter
`that is described in greater detail herein. This summary is not
`intended to be limiting as to the scope of the claims. It is
`intendedto neither identify key elements of the claimed sub-
`ject matter nor delineate the scope of the subject innovation.
`Its sole purpose is to present some concepts of the claimed
`subject matter in a simplified form as a prelude to the more
`detailed description that is presentedlater.
`[0004] The claimed subject matter generally provides an
`electronic device having a touch-sensing element configured
`for sensing touches ona surface thereof. A baseline sensitivity
`setting determinesa sensitivity of the touch-sensing element.
`The touch-sensing element is configured to register a touch
`that meets or exceeds the baseline sensitivity setting, and to
`ignore a touch that does not meet the baseline sensitivity
`setting. The device further includes a sensor that senses an
`operating condition of the device. A memory of the device
`includes code executable by the device and configured to
`adjust the baseline sensitivity setting based upon the sensed
`operating condition.
`[0005] Another embodimentof the claimed subject matter
`relates to a method of interacting with an electronic device.
`
`[0011] The claimed subject matter is described with refer-
`ence to the drawings, wherein like reference numerals are
`used to refer to like elements throughout. In the following
`description, for purposes of explanation, numerousspecific
`details are set forth in order to provide a thorough understand-
`ing of the subject innovation.It may be evident, however,that
`the claimed subject matter may be practiced without these
`specific details. In otherinstances, well-knownstructures and
`devices are shownin block diagram form in orderto facilitate
`describing the subject innovation.
`[0012] As utilized herein, terms “component,” “system,”
`“client” and the like are intended to refer to a computer-
`related entity, either hardware, software (e.g., in execution),
`and/or firmware, or a combination thereof. For example, a
`componentcan be a process running on a processor, an object,
`an executable, a program, a function, a library, a subroutine,
`and/or a computer or a combination of software and hard-
`ware.
`
`[0013] By wayofillustration, both an application running
`on a server and the server can be a component. One or more
`components can reside within a process and a component can
`be localized on one computer and/or distributed between two
`
`6
`
`6
`
`

`

`US 2012/0319959 Al
`
`Dec. 20, 2012
`
`or more computers. The term “processor”is generally under-
`stoodto refer to a hardware component, such as a processing
`unit of a computer system.
`[0014]
`Furthermore, the claimed subject matter may be
`implemented as a method, apparatus, or article of manufac-
`ture using standard programming and/or engineering tech-
`niquesto produce software, firmware, hardware, or any com-
`bination thereof to control a computer to implement the
`disclosed subject matter. The term “article ofmanufacture”as
`used herein is intended to encompass a computer program
`accessible
`from any non-transitory computer-readable
`device, or media.
`[0015] Non-transitory computer-readable storage media
`can include but are not limited to magnetic storage devices
`(e.g., hard disk, floppy disk, and magnetic strips, among
`others), optical disks (e.g., compact disk (CD), and digital
`versatile disk (DVD), among others), smart cards, and flash
`memory devices(e.g., card, stick, and key drive, among oth-
`ers). In contrast, computer-readable media generally(1.e., not
`necessarily storage media) may additionally include commu-
`nication media such as transmission media for wireless sig-
`nals andthe like.
`
`[0016] Of course, those skilled in the art will recognize
`many modifications may be madeto this configuration with-
`out departing from the scope orspirit of the claimed subject
`matter. Moreover, the word “exemplary” is used herein to
`mean serving as an example, instance, or illustration. Any
`aspect or design described herein as “exemplary”is not nec-
`essarily to be construed as preferred or advantageous over
`other aspects or designs.
`[0017] With reference now to FIG. 1, a block diagram an
`electronic device 100 having one embodimentof a system for
`device interaction accordingto the subject innovationis illus-
`trated. The electronic device 100 includes a display screen
`110, a sensor 120, a controller 130, a memory 140, processor
`150 and a signal bus 160.
`[0018] The electronic device 100 may be configured as
`virtually any type of electronic device, such as a smart phone
`or other mobile phone, a laptop computer, a tablet computer,
`an mp3 player, a gaming system, a voice or video recorder, a
`camera, an e-reader or e-book reader, or other electronic
`device that may sometimesbe stored or placed in a carrying
`case, pocket, backpack, purse or other similar carrying or
`storage location.
`[0019] The display screen 110 may, in one embodiment, be
`a touch-sensitive screen having an integral touch element 112
`that overlies or is otherwise associated with a surface of the
`
`display screen 110. The touch element 112 issues a touch
`signal 114 indicative ofthe occurrence of a touch event on the
`touch element 112. In one embodiment, the display screen
`110 may be configured as a capacitive-sensing touch screen
`that includes an integral touch element 112. However,it is to
`be understoodthat the display screen 110 maybe alternately
`configured as, for example, as a display screen having an
`integral touch element 112 configured as a resistive-sensing
`surface, a surface acoustic wave sensing surface, an infrared
`sensing surface, an acoustic pulse sensing surface or other
`suitable type of touch-sensitive surface. In another embodi-
`ment, as shown in FIG. 2, the touch element 112 may be
`separate from and non-integral with display screen 110 and
`can be, for example, disposed on one or moreof the surfaces,
`such as a back, side or top surface, of the electronic device
`100, and be configured as any of the foregoing sensing sur-
`faces. In yet another embodiment, the touch element 112 may
`
`be disposed over or be integral with a substantial portion or
`the entire case or body ofelectronic device 100.
`[0020] The sensor 120 may include one or more sensors
`configured to detect when a barrier such as one or more layers
`of fabric or other material may be in close proximity to or
`overlying the touch element 112. As such, the sensor 120 is
`configured to detect when the electronic device 100 may have
`been placedin its carrying case or other carrying location. In
`one embodiment, the sensor 120 may be configuredto sense,
`for example, an ambientlight level, proximity, or other appro-
`priate parameter or combination of parameters indicative of
`one or morelayers of fabric or other material being disposed
`in close proximity to or overlying the touch element 112. In
`another embodiment, the sensor 120 may be configured to
`sense the self-capacitance of one or moreareas or portions of
`the touch element 112 to thereby detect whether one or more
`layers of fabric or other material is in close proximity to the
`touch element 112. The sensor 120 may issue a sense signal
`122 indicative ofwhether one or morelayers offabric or other
`material may be in close proximity to or overlying the touch
`element 112.
`
`For the sake ofclarity, the terms carrying location,
`[0021]
`carrying case and storage location, as used herein, shall mean
`apocket, bag, backpack, carrying case, purse, or other similar
`location in whichthe electronic device 100 maybe placed and
`by which placement one or more layers of fabric or other
`material may be in close proximity to or overlying the touch
`element 112.
`
`[0022] The controller 130 may be configured as a touch
`screen or touch surface controller that can detect and register
`the location ofa touch occurring on the touch element 112. In
`the embodiment shown, the touch element 112 may be con-
`figured as a capacitive touch screen, and the controller 130
`maycorrespondingly be configured to detect and register the
`location of a conductive element touching the touch element
`112 based at least in part upon a change in the measured
`capacitance of the touch element 112 as indicated,at least in
`part, by the touch signal 114 whichis received by or otherwise
`provided to the controller 130.
`[0023] The memory 140 mayinclude one or more non-
`transitory computer-readable storage media, such as random
`access memory, read only memory, or any combination ofthe
`foregoing in any suitable configuration, including removable
`and non-removable, volatile and non-volatile, flash, disk
`drive, or any other type and configuration ofmemory suitable
`for use in electronic device 100. The memory 140 may
`include various instruction modules 142, including an oper-
`ating system 144, firmware 146, and a recognition module
`148, as well as other modules 142 that enable the operation
`and various functions of electronic device 100. The modules
`
`142 are computer- or processor-executable sets of instruc-
`tions that, when executed by the processor 150, cause the
`electronic device 100 to perform certain corresponding func-
`tions.
`
`[0024] The processor 150 may be configured as a micro-
`processorthat executes the modules 142 to facilitate the sub-
`ject innovation, as will be moreparticularly described, as well
`as executing other modules 142 to control and/or enable the
`operation of the electronic device 100.
`[0025] Each ofthe display screen 110, the sensor 120, the
`controller 130, the memory 140 and the processor 150 can be
`communicatively coupled to the bus 160 over which elec-
`tronic signals and data, including the touch signal 114 and the
`sense signal 122, are exchanged. The bus 160 maybe config-
`
`7
`
`

`

`US 2012/0319959 Al
`
`Dec. 20, 2012
`
`ured as a universal serial bus (USB) or other type of bus
`suitable for use in the electronic device 100.
`
`[0026] The firmware module 146 can be executed by the
`processor 150 to apply baseline sensitivity settings 170 that
`determine the sensitivity level of the touch element 112,
`dependent at least in part upon the sense signal 122. More
`particularly, the processor 150 executes firmware module 146
`to continuously, periodically, or on demand, adjust the base-
`line sensitivity settings 170 to thereby adjust the sensitivity
`level ofthe touch element 112, dependentat least in part upon
`the sense signal 122. The baseline sensitivity settings 170
`determine, at least in part, which touches will and which
`touches will not be detected by touch element 112.
`[0027]
`For the sake ofclarity, it is noted that the term valid
`touch as used herein shall mean a direct or indirect touch that
`is detected and meets, whereas an invalid touch is defined as
`a touch that is detected but does not meet, the characteristics
`of a valid touch, as will be more particularly described here-
`inafter. The terms contact, contacts, touch, touches, as used
`herein shall, unless otherwise indicated, encompass various
`types of valid and invalid contacts upon the touch element
`112. Various types of touches may occur, including a tap
`(whichis defined as a single touchlasting less than or equal to
`a predetermined time), a press (which is defined as a touch
`thatpersists forat least a predetermined minimum amountof
`time), a stroke or moving contact/touch (a touch that in a
`substantially uninterrupted mannertraverses at least a prede-
`termined minimum portion of the touch element 112) and the
`like.
`
`[0028] When the sense 122 signal indicates that the elec-
`tronic device 100 is disposed in a normal-use position, such
`as, for example, on a table or being held by a user, the pro-
`cessor 150 executing the firmware module 146 causes the
`controller 130 to set or adjust the baseline sensitivity settings
`170 to a corresponding and appropriate level for the cur-
`rently-sensed operating condition, 1.e., the normal-use oper-
`ating condition. The baseline sensitivity settings 170 for such
`a normal-use operating condition may bereferred to as the
`default or normal-use baseline sensitivity level, and are well-
`suited for the controller 130 to detect and register direct
`touches or contacts by a user on the touch element 112, all of
`which touches are represented by the touch signal 114, and to
`reject or ignore touches that are weakerthan or fall below the
`normal-use sensitivity settings.
`[0029]
`Similarly, when the sense signal 122 indicates that
`the electronic device 100 is in a storage or carrying location,
`such as in a pocket, and thus that one or morelayersoffabric
`or other material are in close proximity to or overlying the
`touch element 112, the processor 150 executing the firmware
`module 146 causes controller 130 to adjust the baseline sen-
`sitivity settings 170 to a corresponding and appropriate level
`for the currently-sensed operating condition,i.e., the storage
`or carrying-location operating condition. The baseline sensi-
`tivity settings 170 for such a carrying-location operating con-
`dition maybereferred to as the carrying-location sensitivity
`level, and are well-suited for the controller 130 to detect and
`register indirect touches on the touch element 112, all of
`which touches are represented by the touch signal 114, and
`reject or ignore touches that are weakerthan or fall below the
`carrying-location baseline sensitivity level.
`[0030] Any indirect touch upon touch element 112, such as
`a touch that occurs through one or more layers of fabric or
`material, will be attenuated to a degree that dependsat least in
`part upon the characteristics ofthe fabric or material disposed
`
`between a touching member, such as a finger, and the touch
`element 112. Accordingly, in one embodimenthereof, adjust-
`ing the baseline sensitivity settings 170 may includeincreas-
`ing the sensitivity level of the touch element 112 relative to
`the normal-use baseline sensitivity settings when the carry-
`ing-location baseline sensitivity level is applied. In such an
`embodiment, the touch element 112 maybe less sensitive
`whenthe baseline sensitivity level is applied than when the
`carrying-location baseline sensitivity level is applied. Thus,
`whenthe carrying-location baseline sensitivity is applied the
`controller 130 will detect and register relatively light or soft
`direct and indirect touches on the touch element 112, all of
`which touches are represented by the touch signal 114, and
`will reject or ignore touches that are weakerthan orfall below
`the carrying-location baseline sensitivity level.
`[0031] However, in the embodiment wherein the baseline
`sensitivity settings 170 are adjusted to the carrying-location
`baseline sensitivity level, and thus to a more sensitive level
`relative to the normal-use baseline sensitivity level, and with
`the electronic device 100 stored in a storage location where
`one or morelayers of fabric or other material are disposed in
`close proximity to or overlaid upon the touch element 112,
`there is increased likelihood for false or invalid touches to
`occur. These false or invalid touches will be detected and
`
`registered by the touch element 112, and may be included
`within or otherwise reflected by the touch signal 114. These
`false or invalid touches mayalso be referred to as noise. For
`example, when the electronic device 100 is ina pocket and the
`fabric ofthe pocketor another object in the pocket comesinto
`contact with the touch element 112 invalid touches or noise
`
`may be generated, and such noise may be included in the
`touch signal 114 received or otherwise provided to controller
`130.
`
`[0032] The controller 130, in one embodiment, is config-
`ured to distinguish between such noise and valid touches. In
`such an embodiment, the processor 150 may execute the
`firmware module 146 to cause the controller 130 to filter,
`exclude, or otherwise remove invalid touches or noise that
`maybe included in or represented by the touch signal 114.
`The filtering may be based on one or moreattributes or
`parameters of the touch as reflected by the touch signal 114.
`[0033]
`In another embodiment, the processor 150 execut-
`ing the firmware module 146 can cause the controller 130 to
`ignore or reject any touches that have a duration that is less
`than a predetermined minimum amountoftime, to ignore or
`reject touches that do not traverse a predetermined minimum
`portion of the touch element 112, to ignore or reject touches
`that do not traverse a predeterminedtarget portion of or loca-
`tion upon the touch element 112, to ignore or reject touches
`that occur across more than a predetermined portion of the
`touch element 112 (e.g., the object touching the touch ele-
`ment 112 is detected as exceeding an anticipated size of a
`touching member), or to apply othercriteria and parameters
`to distinguish between valid and invalid touchesor noise. In
`this embodiment, a second sensor, such as an accelerometer
`or other sensor, may be used to distinguish between valid and
`invalid touches. Moreparticularly, the processor 150 execut-
`ing the firmware 146 may, in addition to touch 114 signal, also
`considersignals or other indications from other sensors ofthe
`electronic device 100 in order to distinguish between valid
`and invalid touches.
`
`the processor 150
`In yet another embodiment,
`[0034]
`executing the firmware 146, rather than the controller, per-
`formsthe filtering of the noise or otherwise invalid signals
`
`8
`
`

`

`US 2012/0319959 Al
`
`Dec. 20, 2012
`
`that may be present in the touch signal 114. In yet another
`embodiment, the controller 130 may be configured as or
`include a processor and mayitself execute the firmware 146
`to therebyfilter the noise or otherwise invalid signals that may
`be present in touch 114 signal. In a still further embodiment,
`the firmware 146 module can be included within a memory of
`the controller 130.
`
`Thefiltering of noise and otherwise invalid signals
`[0035]
`occurring upon the touch element 112 and that may be present
`in touch signal 114is also facilitated at least in part through
`adjustmentofthe baseline sensitivity settings 170 ofthe touch
`element 112. As described above, the processor 150 execut-
`ing the firmware module 146 causes the controller 130 to
`adjust the baseline sensitivity settings 170 to a corresponding
`and appropriate level for the current operating condition as
`indicated atleast in part by sensor 120 and sense signal 122.
`In one embodiment, the baseline sensitivity settings 170 of
`the touch element 112 are periodically adjusted. More par-
`ticularly, the firmware 146 module when executed by the
`processor 150 may be configured to periodically read sense
`122 signal and, based at least in part thereon, adjust the
`baseline sensitivity settings 170 of the touch element 112. In
`this embodiment, the baseline sensitivity settings 170 of the
`touch element 112 may be adjusted in a substantially-con-
`tinuous or other periodic manner, for example every 100
`milliseconds or over any other suitable time period.It should
`be further noted that, in this embodiment, the baseline sensi-
`tivity settings 170 are not necessarily limited to discrete val-
`uesor levels of sensitivity for a detected operating condition,
`but rather are also substantially-continuously variable in
`termsofthe value or level of sensitivity.
`[0036] The touches that are not ignored, rejected or other-
`wise filtered by the execution of firmware 146 may be con-
`sidered valid touches. In one embodiment, execution of the
`firmware module 146 may be configured to pre-process the
`valid touches to, for example, smooth or merge together
`strokes having irregularities such as gaps or discontinuities in
`an intended single unitary stroke, such as where the intended
`single stroke has a discontinuity whereina first portion of the
`intended single stroke has an end that is temporally and spa-
`tially near a beginning portion of a second portion of the
`intended single stroke.
`[0037] As noted above, various types of valid touches may
`occur upon touch element 112, such as a tap, stroke or press.
`The various types of valid touches or combinations thereof
`maybe processedby electronic device 100 to, in one embodi-
`ment, invoke certain functions, to cause certain functions to
`be performed, or be interpreted as textual or characterinput.
`As noted above, the processor 150 executes the firmware
`module 146 to cause the controller 130 to filter, exclude, or
`otherwise remove invalid touches or noise that may be
`included in or represented by touch signal 114.
`[0038] The processor 150 executing recognition module
`148 processes the valid touches. The recognition module 148
`as executed by processor 150 is configured, in one embodi-
`ment, to interpret the certain types of valid touches as corre-
`sponding to alphanumeric characters, gestures or commands.
`For example, in one embodiment, a tap anywhere on the touch
`element 112 may be recognized by the recognition module
`148 or applied as a commandto place the electronic device
`100 in a silent or muted modeof operation, whereas a press
`anywhere on touch element 112 may be recognized by the
`
`recognition module 148 as acommandto place the electronic
`device 100 in a designated mode of operation, such as a
`telephone mode.
`[0039]
`In yet another embodiment, one or more predeter-
`mined strokes on touch element 112, such as a stroke in the
`shapeofthe capital letter “L” or two parallel strokes “|”, may
`be recognized by recognition 148 module as a command or
`stroke by which the coordinate system for inputting subse-
`quent touchesorstrokes via touch element112 is established.
`The use of such predetermined strokes enables, at least in
`part, the electronic device 100 to determine, via recognition
`148 module, an orientation of the electronic device 100 rela-
`tive to the user, and thereby indicates, for example, the top,
`bottom andsides ofthe touch element 112 relative to the user.
`
`The same one or more predetermined strokes may also be
`recognized by recognition 148 module as corresponding to
`one or more additional commands, such as a command to
`unlock and/or wakethe electronic device 100 from a stand-by
`or sleep mode and as a commandto adjust the baseline sen-
`sitivity of the touch element 112, and thus the predetermined
`special strokes may serve two or more purposes making
`operation ofand interface with the electronic device 100 more
`efficient.
`
`[0040] Due to the small screen sizes found on manyelec-
`tronic devices, separate touches or strokes may occur within
`the samespace or location on the touch element 112 and yet
`occurat different times or sequentially. Accordingly,the rec-
`ognition module 148 mayfurther be configured to distinguish
`between touches or strokes that occur in the same space of
`touch element 112, and thus overlap and/or overlie each other
`but are separatedin time, to thereby form groups of sequential
`strokes. Groups of sequential strokes may be delineated by a
`predetermined touch orstroke, such as a stroke on the touch
`element 112 that correspondsto the “>” symbol(the “greater
`than” symbol), by a tap or double tap upon the touch element
`112, or by a simple pause in stroke or touch entry. The rec-
`ognition module 148 may be configured to group together
`sequential strokes that occur between or precede the delin-
`eating characters, and to configure those groups of sequential
`strokes as though they were written on a continuousline. The
`recognition module 148 is further configured to convert the
`groups of sequential strokes into words by, for example,
`applying a suitable text recognition methodthat incorporates
`a language model
`for word recognition,
`such as the
`Microsoft® Ink application programminginterface or similar
`text and word recognition method.
`[0041] The recognition module 148 is further configured to
`combine and interpret together certain sequential strokes
`rather thaninterpreting those strokes as separate strokes. Cer-
`tain characters, such as small-case letters including “k’”, “t”
`and “x”, are formed from separate and sequential strokes that
`are separated temporally yet indicate a single character or
`letter. The separate strokes used to form these certain charac-
`ters or letters contain a set of predetermined secondstrokes,
`including strokes corresponding to the symbols “<”, “-”, “\”’
`or “/”(1.e., the less than symbol, the dash symbol, the back-
`ward slash symbol, and the forward slash symbol, respec-
`tively). The recognition module 148 is configured to combine
`a memberoftheset of predetermined secondstrokes with the
`precedingstroke.
`[0042] With reference now to FIGS. 3A-3D,the processing
`of certain sequential strokes on the touch element 112 is
`illustrated. An exemplary collection of strokes 300 that
`occurred upon and were detected by the touch element 112
`
`9
`
`

`

`US 2012/0319959 Al
`
`Dec. 20, 2012
`
`are illustrated in FIG. 3A. Althoughthe strokes 300 may be
`temporally separated, the strokes 300 nonetheless occur upon
`and traverse commonor overlapping portions of the touch
`element 112. The recognition module 148 as executed by the
`processor 150 is configuredto distinguish between touches or
`strokes that occur in the samespaceof the touch element 112
`but are separated in time, and thereby form groups of sequen-
`tial strokes. The strokes 300 as temporally separated into
`strokes 302, 304, 306, 308, 310 and 312 by the recognition
`module 148are illustrated in FIG. 3B.
`
`[0043] As discussed above, the recognition module 148
`may be configured to combine together certain sequential
`strokes rather than treating those strokes as individual and
`separate strokes. As shown in FIG.3C, the recognition mod-
`ule 148 as executed by the processor 150 combines together
`strokes 302 and 304, and combines together strokes 310 and
`312, each ofwhich were madeat different times and thus each
`constitutes separate but sequential touchesor stroke events on
`the touch element 112. The combined strokes form a text
`
`input 320 that is further processed by the recognition module
`148 to interpret or convert the text input 320 into a word, such
`as by applying a suitable text recognition methodthat incor-
`porates a language model for word recognition. As shown in
`FIG.3D,the resulting word or group of alphanumeric char-
`acters 330 is produced.
`[0044] With reference now to FIG. 4, a method 400 of
`device interaction of the subject innovationis illustrated. The
`method 400 includes determining