as) United States
`a2) Patent Application Publication (0) Pub. No.: US 2012/0304107 Al
` Nanet al. (43) Pub. Date: Nov. 29, 2012
`
`
`
`US 20120304107A1
`
`(54) EDGE GESTURE
`
`(22)
`
`Filed:
`
`May27, 2011
`
`(76)
`
`Inventors:
`
`Jennifer Nan, Seattle, WA (US);
`Jesse Clay Satterfield, Seattle, WA
`(US); David A. Matthews, San
`Francisco, CA (US); ThomasP.
`Russo,Seattle,WA (US); Robert J.
`Jarrett, Snohomish, WA (US);
`Weidong Zhao, Redmond, WA
`(US); Jensen Harris, Bellevue, WA
`(US); Chattanya Dev Sareen,
`Seattle, WA (US)
`
`(21) Appl. No.:
`
`13/118,221
`
`Publication Classification
`
`(51)
`
`Int. Cl.
`(2006.01)
`GO6F 3/01
`(2006.01)
`GO6F 3/048
`(52) US. Ch. cece 715/781; 715/863; 715/810
`
`(57)
`
`ABSTRACT
`
`techniques and apparatuses
`This document describes
`enabling an edge gesture. In some embodiments, these tech-
`niques and apparatuses enable selection ofa user interface not
`currently exposed on a display through an edge gesturethatis
`easy-to-use and remember.
`
` Adorns ~-
`BhReork af
`“eg.
`Mome
`
`ed
`poe
`i.
`,
`
`».
`
`STL
`
`Satelite Atay Crash To Banh
`os
`afhe National Weather Service bas
`just inarned thet a waether selelite
`has been damaged by a frietear and
`is in a decevina orbit.
`
`bitur: the sex? tac
`Fotvis vill likely
`
`ator.
`
`
`we pdioe
`
`1
`
`APPLE 1104
`
`APPLE 1104
`
`1
`
`

`

`Patent Application Publication
`
`Nov. 29, 2012 Sheet 1 of 14
`
`US 2012/0304107 Al
`
`108 —~,
`*%
`
`Computing Davies 12
`
`Processor]!
`
`Madia 118
`
`Onerating Bystenr
`420
`
`Application UA
`
`rae
`i
`
`i;
`
`i
`
`i
`
`Windows-Based Mode Module
`$22
`
`ey
`.
`;
`immersive Mode Mrriube
`i24
`
`Systerr-irderisce Module
`
`Gesture Hancler
`43
`
`Applications} 138
`
`2
`
`

`

`Patent Application Publication
`
`Nov. 29, 2012 Sheet 2 of 14
`
`US 2012/0304107 Al
`
`
`
`Receive Gesture
`
`—Baiereevee wheterStart Point of *
`
`Gesture b atidae
`met
`.
`
`'
`
`Yas
`
`Nis
`
`26
`
`“Batermine Wher Line frorn Sint
`eon io Leer Point is Approximaioly
`Perpendicular
`
`~
`
`o-
`
`No
`
`nay
`
`
`
`au.
`Pass Gestureto
`Exposed LA
`°
`
`YES |
`
`End
`
`Than Exposed LH
`
`Pease Gesture to Entity Other
`
`Fig. 2
`
`3
`
`

`

`Patent Application Publication
`
`Nov. 29, 2012 Sheet 3 of 14
`
`US 2012/0304107 Al
`
`we
`ra)
`dre National Weatier Service has
`just loarnect thet e wealher satelite
`has been darned by 4 meteor and
`ig ic a decaying orbit.
`
`Bei
`Bef
`Your
`
`
`
`Hosnat
`
`
`
`
`he currandyuncertain of
`(ne of reentry or
`nig if dabria, the
`3
`gicated that
`yehin the reext ied
`pebris will Healy
`BIOL,
`
` 7 i i j i
`
`Fig. 3
`
`4
`
`

`

`Patent Application Publication
`
`Nov. 29, 2012 Sheet 4 of 14
`
`US 2012/0304107 Al
`
`
`
`Fig. 4
`
`5
`
`

`

`Patent Application Publication
`
`Nov. 29, 2012 Sheet 5 of 14
`
`US 2012/0304107 Al
`
`
`
`6
`
`

`

`Patent Application Publication
`
`Nov. 29, 2012 Sheet 6 of 14
`
`US 2012/0304107 Al
`
`
`
`7
`
`

`

`Patent Application Publication
`
`Nov. 29, 2012 Sheet 7 of 14
`
`US 2012/0304107 Al
`
`fhe Mational Weather Gurvice has
`
`rat learned that a weelher saleiite
`
`
`Home!
`
`in a decaying orbit.
`
`
`
`
`rendyurnomriain of
`ae of reentry or
`A
`
`
`:
`dir of dabris, the
`gy
`
`ALYsannanny,,}porrnnnnnnnnngnnnoninnnainiiinnnmnnisonmnminniisenniit oe golf
`PUR won be re *
`wy dicated that
`vouir
`
`me within the mek? two
`baat cletris will Healy
`
`heequator,
`
`
`8
`
`

`

`Patent Application Publication
`
`Nov. 29, 2012 Sheet 8 of 14
`
`US 2012/0304107 Al
`
`B00my
`
`
`
`
`
`Boe
`
`Detarmine thal a Gesture Made vara User intariace
`hes a Start Point Within an Edge of the User Interface
`ara @ Loter Point Not WiHhin the Edge
`
`
`
`
`
`804
`Determine Which Liser Interface to Present
`Based on Factor of Gesture
`
`Present Determined User interface
`
`
`
`9
`
`

`

`Patent Application Publication
`
`Nov. 29, 2012 Sheet 9 of 14
`
`US 2012/0304107 Al
`
`GOD wom,
`
`Receive o Successive Pont of the festure
`
`wn
`904
`ie
`|
`;
`Reversal Ue" Determine, Based on Successive
`“i
`Pond, Whether Reversal or dealtior
`Extansion or Metther_
`:
`
`Exterior
`
`Second User interface
`
`Cease hiPrasent
`Second User
`inferiace
`
`|
`
`Present Third User
`interface or Expand
`
`10
`
`10
`
`

`

`Patent Application Publication
`
`Nov. 29,2012 Sheet 10 of 14
`
`US 2012/0304107 Al
`
`
`
`
`(Prony Sue Brown
`Han
`on
`: Email Subjeck Lal's Meet. |
`
`yon 1004
`om AGIQ = 1006
`
`
`
`Ze
`
`oNpws
`
`ig,
`‘eolher Salatie May
`Ri Grash To Sard
`
`The conference call funotion
`
`
`
`onmy ofios phone seamsip
`"og
`.
`.
`:
`
`#4
`~The National Weather
`be outst onder ~canwemeet
`
`
`| Serving has just
`in Negla’s office? She has the
`
`new le meaiing soltvare cn
`| nomad that 3 weather
`
`
`her computer,
`i saigiite has beer
`
`VAL my
`j darmaged by 4 meteor
`“Tyarent
`| aral is in a decaying
`fA?
`
`:
`| Sue
`| Orbit.
`
`
` canadecacaansaanaccccccceaccinsanscscccccsscscccsasssssg Mat
`
`
`
`
`
`Fig. 10
`
`11
`
`11
`
`

`

`Patent Application Publication
`
`Nov. 29, 2012 Sheet 11 of 14
`
`US 2012/0304107 Al
`
`1 4
`
`,
`EU338
`;a
`
`
`
`i From Sue Brown
` Subjeck Lets Meat. 3
`
`
`
`E| The conference call Liction
`Bi on my offies phone agemne i
`i
`he cul of ordercan we mee
`n Navia's office? She has the
`3B Dew lee rmetingsodmens orp PP
`Bi her commuter,
`1
`TOEmy AS
`pi Thankel
`1110 -——QOMR LS {Ht
`
`wn
`
`Fig. 17
`
`12
`
`12
`
`

`

`Patent Application Publication
`
`Nov. 29, 2012 Sheet 12 of 14
`
`US 2012/0304107 Al
`
`sone $004
`f
`
`pow 1008
`f
`
`her noiteday /
`
`The conferees call lunciion
`
`be cut of order ~ car we amet
`in Mazio’s affion? She hes the
`new live meeting sofhears on
`
`ee asews
`vYeather Saieliite May
`Crash Toa Earth
`
`” TheMeational Weather
`Service has just
`leamed that a weather
`saiedite has bean
`damaged by a meteor
`and ig ina decaying
`orbit.
`
`{DOE ——"
`
`Fig. 12
`
`13
`
`13
`
`

`

`Patent Application Publication
`
`Nov. 29, 2012 Sheet 13 of 14
`
`US 2012/0304107 Al
`
`pt
`
`| News
`
`The conferenns call function
`ny
`Boe phone seems to
`
`vreather Safaive May
`a
`ei Crash To Garth
`*
`
`The National Weather
`| Service has just
`i learned that a weather
`i satellite has beer
`
`| damaged by a meteor
`Land is ins decaying
`+ orbit.
`
`Fig. 13
`
`14
`
`14
`
`

`

`Patent Application Publication
`
`Nov. 29, 2012 Sheet 14 of 14
`
`US 2012/0304107 Al
`
`Device
`Applications
`
`Madue
`ded
`
`Sesture
`Harter
`428
`
`| Application(s}
`
` | Systern-interlace |
`
`Proceasing
`& Control
`4432
`
`Communication
`Device(s}
`
`Devine
`Dalia
`+404
`
`eieefeeiieiieeieniieiseeieiniieiieeieninieiienienineeiseienineieing,
`
`Chater
`inpulle)
`4456
`
`CONTInCation
`inferface(s}
`
`Storage
`Mecha
`4478
`
`Fig. 14
`
`15
`
`

`

`US 2012/0304107 Al
`
`Nov. 29, 2012
`
`EDGE GESTURE
`
`FIG.4 illustrates the example immersive interface
`[0010]
`of FIG. 3 along with example edges.
`BACKGROUND
`
`[0011] FIG.5illustrates the example immersive interface
`of FIGS. 3 and 4 along with angular variance lines from a
`perpendicular line and a line from a start pointto a later point
`ofa gesture.
`[0012]
`FIG.6 illustrates the edges of the immersive inter-
`face shownin FIG.4 along with two regions inthe right edge.
`[0013]
`FIG.7 illustrates an application-selection interface
`presented by a system-interface module in response to an
`edge gesture and over the immersive interface and webpage
`of FIG.3.
`[0014]
`FIG.8 illustrates an example method for enabling
`edge gestures including determining an interface to present
`based on somefactor of the gesture.
`[0015]
`FIG. 9 illustrates an example method enabling
`expansion of, or ceasing presentation of, a user interface
`presented in response to an edge gesture or presentation of
`another user interface.
`
`[0001] Conventional techniques for selecting a user inter-
`face that is not currently exposed on a display are often
`confusing, take up valuable display space, cannot be univer-
`sally applied across different devices, or provide a poor user
`experience.
`[0002]
`Some conventional techniques, for example, enable
`selection of a user interface through onscreen controls in a
`task bar, within a floating window, or on a window frame.
`These onscreen controls, however, take up valuable display
`real estate and can annoyusers by requiring usersto find and
`select the correct control.
`[0003]
`Someother conventional techniques enable selec-
`tion ofa user interface through hardware, such as hot keys and
`buttons. At best these techniques require users to remember
`what key, key combination, or hardwarebutton to select. Even
`in this best case users often accidentally select keys or but-
`tons. Further, in many cases hardware-selection techniques
`cannot be universally applied, as hardware on computing
`devices can vary by device model, generation, vendor, or
`manufacturer. In such cases either the techniques will not
`work or work differently across different computing devices.
`This exacerbates the problem of users needing to remember
`the correct hardware, as many users have multiple devices,
`and so may need to rememberdifferent hardware selections
`for different devices. Further still, for many computing
`devices hardware selection forces users to engage a comput-
`ing device outside the user’s normal flow ofinteraction, such
`as when a touch-screen device requires a user to changehis or
`her mental and physical orientation from display-basedinter-
`actions to hardware-basedinteractions.
`
`SUMMARY
`
`[0004] This document describes techniques and appara-
`tuses enabling an edge gesture. In some embodiments, these
`techniques and apparatuses enable selection of a userinter-
`face not currently exposed on a display through an edge
`gesture that is easy-to-use and remember.
`[0005] This summary is provided to introduce simplified
`concepts for enabling an edge gesture that are further
`described below in the Detailed Description. This summary is
`not intended to identify essential features of the claimed
`subject matter, nor is it intended for use in determining the
`scopeof the claimed subject matter. Techniques and/or appa-
`ratuses enabling an edge gesture are also referred to herein
`separately or in conjunction as the “techniques”as permitted
`by the context.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`are
`enabling an edge gesture
`[0006] Embodiments
`described with referenceto the following drawings. The same
`numbersare used throughout the drawings to reference like
`features and components:
`[0007]
`FIG.1 illustrates an example system in which tech-
`niques enabling an edge gesture can be implemented.
`[0008]
`FIG.2 illustrates an example method for enabling
`edge gestures based on the edge gesture being approximately
`perpendicular to an edge in which the gesture begins.
`[0009]
`FIG. 3 illustrates an example tablet computing
`device having a touch-sensitive display presenting an immer-
`sive interface.
`
`FIG. 10 illustrates a laptop computer having a
`[0016]
`touch-sensitive display having a windows-based email inter-
`face and two immersive interfaces.
`[0017]
`FIG. 11 illustrates the interfaces of FIG. 10 along
`with two gestures having a start point, later points, and one or
`more successive points.
`[0018]
`FIG. 12 illustrates the windows-based email inter-
`face of FIGS. 10 and 11 along with an email handling inter-
`face presented in response to an edge gesture.
`[0019]
`FIG. 13 illustrates the interfaces of FIG. 12 along
`with an additional-email-options interface presented in
`response to a gesture determined to have a successive point a
`preset distance from the edge.
`[0020]
`FIG. 14 illustrates an example device in which tech-
`niques enabling edge gestures can be implemented.
`
`DETAILED DESCRIPTION
`
`[0021] Overview
`[0022] This document describes techniques and appara-
`tuses enabling an edge gesture. These techniques enable a
`user to quickly and easily select an interface not currently
`exposed on the user’s device, as well as other operations.
`[0023] Consider a case where a user is watching a movie on
`a tablet computing device. Assumethat the movie is playing
`on an immersive interface occupying all of the display and
`that the user would like to check her social-networking
`webpage without stopping the movie. The described tech-
`niques and apparatuses enable herto select other interfaces
`through a simple swipe gesture started at an edge of her
`display. She may swipe from one edgeofherdisplay and drag
`out a user interface enabling her to select her social network-
`ing website. Or instead, assumethat she wouldlike to interact
`with the media application playing the movie in a mannernot
`permitted by the immersive interface, such as to display a
`menu enabling subtitles or a director’s commentary. She may
`swipe from another edgeofher tablet’s display and drag out
`a control menu for the immersive interface and select items
`and/or commandsfrom this menu quicklyandeasily.
`[0024]
`In both of these cases valuable real estate used to
`play the movie wasnot taken up with on-screen controls, nor
`wasthe user required to rememberandfind a hardware but-
`ton. Furtherstill, no gesture, other than onestarting from an
`edge,
`is used by the techniques in this example, thereby
`permitting the immersive interface to use nearly all com-
`monly-available gestures. Additionally, by considering edge
`
`16
`
`16
`
`

`

`US 2012/0304107 Al
`
`Nov. 29, 2012
`
`gestures or portions thereof, the techniques do not affect
`performanceof a gesture or touch input system as the edge
`gestures can be processed before the entire gesture is com-
`plete avoiding latency associated with processing entire ges-
`tures started elsewhere.
`
`System-interface module 126 provides one or more
`[0033]
`interfaces through which interaction with operating system
`120 is enabled, such as an application-launchinginterface, a
`start menu, or a system tools or options menu, to namejust a
`few.
`
`[0025] These are but two examples of the many ways in
`which the techniques enable and use edge gestures, others of
`which are described below.
`
`[0026]
`Example System
`FIG. 1 illustrates an example system 100 in which
`[0027]
`techniques enabling an edge gesture can be embodied. Sys-
`tem 100 includes a computing device 102, which is illustrated
`with six examples: a laptop computer 104, a tablet computer
`106, a smart phone 108, a set-top box 110, a desktop com-
`puter 112, and a gaming device 114, though other computing
`devices and systems, such as servers and netbooks, mayalso
`be used.
`
`[0028] Computing device 102 includes computer processor
`(s) 116 and computer-readable storage media 118 (media
`118). Media 118 includes an operating system 120, windows-
`based mode module 122, immersive mode module 124, sys-
`tem-interface module 126, gesture handler 128, and one or
`moreapplications 130, each having one or moreapplication
`user interfaces 132.
`
`[0029] Computing device 102 also includesor hasaccess to
`one or more displays 134 and input mechanisms 136. Four
`example displays are illustrated in FIG. 1. Input mechanisms
`136 mayinclude gesture-sensitive sensors and devices, such
`as touch-based sensors and movement-tracking sensors(e.g.,
`camera-based), as well as mice (free-standingor integral with
`a keyboard), track pads, and microphones with accompany-
`ing voice recognition software, to name a few. Input mecha-
`nisms 136 may beseparate or integral with displays 134;
`integral examples include gesture-sensitive displays with
`integrated touch-sensitive or motion-sensitive sensors.
`[0030] Windows-based mode module 122 presents appli-
`cation user interfaces 132 through windows having frames.
`These frames may provide controls through whichto interact
`with an application and/or controls enabling a user to move
`and resize the window.
`
`Immersive mode module 124 provides an environ-
`[0031]
`ment by which a user may view andinteract with one or more
`of applications 130 through application user interfaces 132.
`In some embodiments, this environment presents content of,
`and enables interaction with, applications with little or no
`window frame and/or without a need for a user to manage a
`window frame’s layout or primacyrelative to other windows
`(e.g., which window is active or up front) or manually size or
`position application user interfaces 132.
`[0032] This environment can be, but is not required to be,
`hosted and/or surfaced without use of a windows-based desk-
`top environment. Thus, in some cases immersive mode mod-
`ule 124 presents an immersive environment that is not a
`window (even one without a substantial frame) and precludes
`usage of desktop-like displays (e.g., a taskbar). Furtherstill,
`in some embodiments this immersive environmentis similar
`to an operating system in that it is not closeable or capable of
`being un-installed. While not required, in some cases this
`immersive environmentenablesuse ofall or nearly all of the
`pixels of a display by applications. Examples of immersive
`environments are provided below as part of describing the
`techniques, though they are not exhaustive or intended to
`limit the techniques described herein.
`
`[0034] Operating system 120, modules 122, 124, and 126,
`as well as gesture handler 128 can be separate from each other
`or combinedor integrated in any suitable form.
`[0035] Example Methods
`[0036]
`FIG. 2 depicts a method 200 for enabling edge ges-
`tures based on the edge gesture being approximately perpen-
`dicular to an edge in which the gesture begins. In portions of
`the following discussion reference may be made to system
`100 of FIG. 1, reference to which is made for example only.
`[0037] Block 202 receives a gesture. This gesture may be
`receivedat variousparts of a display, such as over a windows-
`based interface, an immersive interface, or no interface. Fur-
`ther, this gesture may be madeandreceived in various man-
`ners, such as a pointer tracking a movementreceived through
`a touch pad, mouse,or roller ball or a physical movement
`made with arm(s), finger(s), or a stylus received through a
`motion-sensitive or touch-sensitive mechanism. In some
`
`cases, the gesture is received off of or proximate to a physical
`edge ofthe display (e.g., as a finger or stylus encounters the
`edge of the display) by a touch digitizer, a capacitive touch
`screen, or a capacitive sensor, just to name a few.
`[0038] By way of example consider FIG. 3, which illus-
`trates a tablet computing device 106. Tablet 106 includes a
`touch-sensitive display 302 shown displaying an immersive
`interface 304 that includes a webpage 306. As part of an
`ongoing example, at block 202 gesture handler 128 receives
`gesture 308 as shownin FIG.3.
`[0039] Block 204 determines whethera start point of the
`gesture is at an edge. As noted above,the edge in question can
`be an edge of a user interface, whether immersive or win-
`dows-based, and/or of a display. In somecases, of course, an
`edge of a user interface is also an edge of a display. The size
`ofthe edge can vary based on variousfactors aboutthe display
`or interface. A small display or interface may have a smaller
`size in absolute orpixel terms than a large display or interface.
`Ahighly sensitive input mechanism permits a smaller edge as
`well. In some instances, an edge may extend beyond an edge
`of the display or a screen when an input mechanism is able to
`receive a gesture portion beyond the display or screen.
`Example edges are rectangular and vary between one and
`twenty pixels in one dimension andan interface limit of the
`interface ordisplay in the other dimension,though othersizes
`and shapes, including convex and concave edges may instead
`be used.
`
`[0040] Continuing the ongoing example, consider FIG.4,
`whichillustrates immersive interface 304 and gesture 308 of
`FIG. 3 as well as left edge 402, top edge 404,right edge 406,
`and bottom edge 408. For visual clarity webpage 306 is not
`shown.In this example the dimensions of the interface and
`display are of a moderate size, between that of smart phones
`and that of many laptop and desktop displays. Edges 402,
`404, 406, and 408 have a small dimensionof twenty pixels or
`about 10-15 mm in absolute terms, an area of each shown
`bounded by dashedlines at twenty pixels from the display
`limit at edge limit 410, 412, 414, and 416, respectively.
`[0041] Gesture handler 128 determines that gesture 308 has
`a start point 418 and thatthis start point 418 is within left edge
`402. Gesture handler 128 determines the start point in this
`case by receiving data indicating [X,Y] coordinatesin pixels
`
`17
`
`17
`
`

`

`US 2012/0304107 Al
`
`Nov. 29, 2012
`
`at which gesture 308 begins and comparingthefirst of these
`coordinates to those pixels contained within each edge 402-
`408. Gesture handler 128 often can determinethe start point
`and whetherit is in an edge faster than a samplerate, thereby
`causing little or no performance downgrade from techniques
`that simply pass gestures directly to an exposedinterface over
`which a gesture is made.
`[0042] Returning to method 200 generally, if block 204
`determinesthatthe start point of the gesture is not at an edge,
`method 200 proceeds along a “No”path to block 206. Block
`206 passes the gestures to an exposeduserinterface, such as
`an underlying interface over which the gesture was received.
`Altering the ongoing example, assumethat gesture 308 was
`determinednot to have a start point within an edge. In sucha
`case gesture handler 128 passes buffered data for gesture 308
`to immersive user interface 304. After passing the gesture,
`method 200 ends.
`
`qualifies the gesture. Examples include whena laterpointis
`within the edge, such as dueto a hover, tap, press-and-hold,or
`up-and-downgesture (e.g., to scroll content in the user inter-
`face), when the gesture is set to be a single-input gesture and
`a secondinputis received (e.g., a first finger starts at an edge
`but a second finger then lands anywhere), or if a tap event
`occurs during or prior to the gesture (e.g., a finger is already
`making contact elsewhere or contact is received elsewhere
`during the gesture).
`[0049]
`If block 208 determines that the line is approxi-
`mately perpendicular based a later point outside the edge,
`method 200 proceeds along a “Yes” path to block 210.
`[0050] Block 210 respondsto the positive determination of
`block 208 by passing the gesture to an entity other than the
`exposeduser interface. This entity is not a user interface over
`which the gesture was received, assuming it was received
`over a user interfaceat all. Block 210 may determine to which
`If block 204 determines that the start point of the
`[0043]
`entity to pass the gesture as well, such as based on an edge or
`gesture is in an edge, method 200 proceeds along a “Yes”path
`region of an edge in whichthe start point of the gesture is
`to block 208. Optionally, block 204 may determinealength of
`received. Consider FIG. 6, for example, which illustrates
`a portion ofthe gesture before method proceeds to block 208.
`immersive interface 304 and edges 402, 404, 406, and 408 of
`In some cases, determining the length of the portion of the
`FIG.4 but adds top region 602 and bottom region 604to right
`gesture allows the determinationofthe start point to be made
`edge 406. A start point in top region 602 can result in a
`prior to acompletion ofthe gesture. Block 208 respondsto the
`different entity (or even a same entity but a different user
`positive determination of block 204 by determining whether
`interface provided in response) than a start point received to
`a line from the start point to a later point of the gesture is
`bottom region 604. Likewise, a start pointin top edge 404 can
`approximately perpendicular from the edge.
`result in a different entity or interface than left edge 402 or
`[0044] Block 208, in some embodiments, determines the
`bottom edge 408.
`later point used. Gesture handler 128, for example, can deter-
`[0051]
`In some cases, this entity is an application associ-
`mine the later point of the gesture based on the later point
`ated withtheuserinterface. In such a case, passing the gesture
`being received a preset distance from the edge orthe start
`to the entity can be effective to cause the application to
`point, such as past edge limit 410 for edge 402 or twenty
`present a second user interface enabling interaction with the
`pixels from start point 418, all of FIG. 4. In some other
`application. In the movie example above, the entity can be the
`embodiments, gesture handler 128 determinesthe later point
`media player playing the movie but not the immersiveinter-
`based on it being received a preset time after receipt of the
`face displaying the movie. The media player can then present
`start point, such an amountoftimeslightly greater than used
`a second user interface enabling selection of subtitles or a
`generally by computing device 102 to determine that a ges-
`director’s commentary rather than selections enabled by the
`ture is a tap-and-hold or hovergesture.
`interface displaying the movie, such as “pause,” “play,” and
`[0045]
`For the ongoing embodiment, gesture handler 128
`“stop.” This capability is permitted in FIG. 1, where one of
`uses a later-received point of gesture 308 received outside of
`applications 130 can includeor be capable ofpresenting more
`edge 402 so long as that later-received pointis received within
`than one application user interface 132. Thus, block 210 can
`a preset time.Ifno point is received outside ofthe edge within
`pass the gesture to system-interface module 126, the one of
`that preset time, gesture handler 128 proceeds to block 206
`applications 130 currently presenting the user interface, or
`and passes gesture 308 to immersive interface 304.
`another of applications 130, to namejust three possibilities.
`[0046] Using thestart point, block 208 determines whether
`[0052] Concluding the ongoing embodiment, at block 210
`a line from the start point to the later point of the gesture is
`gesture handler 128 passes gesture 308 to system-interface
`approximately perpendicular to the edge. Various angles of
`module 126. System-interface module 126 receives the buff-
`variance can be usedin this determination by block 208, such
`ered portion of gesture 308 and continuesto receive the rest of
`as five, ten, twenty, or thirty degrees.
`gesture 308 as it is made by the user. FIG. 7 illustrates a
`possible response upon receiving gesture 308, showing an
`[0047] By way of example, consideran angle ofvariance of
`application-selection interface 702 presented by system-in-
`thirty degrees from perpendicular. FIG. 5 illustrates this
`terface module 126 and over immersive interface 304 and
`example variance, showing immersive interface 304, gesture
`webpage 306 from FIG. 3. Application-selection interface
`308, left edge 402, left edge limit 410, and start point 418 of
`702 enables selection of various other applications and their
`FIGS. 3 and 4 along with a thirty-degree variance lines 502
`respective interfaces at selectable application tiles 704, 706,
`from perpendicular line 504. Thus, gesture handler 128 deter-
`708, and 710.
`mines that line 506 from start point 418 to later point 508
`(which is at about twenty degrees from perpendicular) is
`[0053] The example application-selection interface 702 is
`approximately perpendicular based on being within the
`an immersiveuser interface presented using immersive mode
`example thirty-degree variance line 502.
`module 124, thoughthis is not required. Presented interfaces,
`ora list thereof, may instead be windows-basedandpresented
`[0048] Generally, if block 208 determines that the line is
`using windows-based module 122. Both ofthese modules are
`not approximately perpendicular to the edge, method 200
`illustrated in FIG.1.
`proceeds along a “No” path to block 206 (e.g., a path of a
`finger is curved). As noted in part above, block 208 mayalso
`determine that a later point or other aspect of a gesture dis-
`
`[0054] Block 210 mayalso or instead determineto pass the
`gesture to different entities and/or interfaces based on other
`
`18
`
`18
`
`

`

`US 2012/0304107 Al
`
`Nov. 29, 2012
`
`lar to manners described in other methods, such as with a
`progressive display of application-selection user interface
`702 of FIG.7.
`
`the gesture received. Example factors are
`factors about
`described in greater detail in method 800 below.
`[0055] Note that method 200 and other methods described
`Following method 200 and/or method 800 in whole
`[0062]
`hereafter can be performed in real-time, such as while a
`or in part, the techniques may proceed to perform method 900
`gesture is being made and received. This permits, among
`of FIG. 9. Method 900 enables expansionof a user interface,
`other things, a user interface presented in responseto a ges-
`presentation of another interface, or ceasing presentation of
`ture to be presented prior to completion of the gesture. Fur-
`the userinterface presented in response to an edge gesture.
`ther, the user interface can be presented progressively as the
`gesture is received. This permits a user experience of drag-
`[0063] Block 902 receives a successive pointof the gesture
`ging out the user interface from the edge as the gesture is
`and after presentation of at least some portion of the second
`performed with the user interface appearing to “stick”to the
`user interface. As noted in part above, methods 200 and/or
`gesture (e.g., to a mouse point or person’s finger making the
`800 are able to present or cause to be presented a second user
`gesture).
`interface, such as a seconduserinterface for the same appli-
`cation associated with a current user interface, a different
`FIG. 8 depicts a method 800 for enabling edge ges-
`[0056]
`application, or a system userinterface.
`tures including determining an interface to present based on
`some factor of the gesture. In portions of the following dis-
`[0064] By way of example, consider FIG. 10, which illus-
`cussion reference may be made to system 100 of FIG. 1,
`trates a laptop computer 104 having a touch-sensitive display
`reference to which is made for example only. Method 800
`1002 displaying a windows-based email interface 1004 and
`two immersive interfaces 1006 and 1008. Windows-based
`may act wholly or partly separate from, or in conjunction
`with, other methods described herein.
`email interface 1004 is associated with an application that
`[0057] Block 802 determines that a gesture made over a
`manages email, which can be remote or local to laptop com-
`
`userinterface hasastart point at an edgeofthe user interface puter 104. FIG. 10 also illustrates two gestures, 1010 and
`and a later point not within the edge. Block 802 may operate 1012. Gesture 1010 proceeds inastraight line while gesture
`
`similarly to or use aspects ofmethod 200, such as determining
`1012 reverses back (shown with two arrows to show two
`a later point on which to base block 802’s determination.
`directions).
`Block 802 mayact differently as well.
`[0065]
`FIG. 11 illustrates gesture 1010 havinga start point
`[0058]
`In one case, for example, block 802 determinesthat
`1102, a later point 1104, and a successive point 1106, and
`a gesture is a single-finger swipe gesture starting at an edge of
`gesture 1012 having a samestart point 1102, a later point
`
`an exposed immersive user interface and havinga later point 1108, andafirst successive point 1110, and a second succes-
`not at the edge butnot based on an angle ofthe gesture. Based
`sive point 1112. FIG. 11 also shows a bottom edge 1114, a
`on this determination, block 802 proceeds to block 804 rather
`later-point area 1116, and an interface-addition area 1118.
`thanpassthe gestureto the exposed immersiveuserinterface.
`[0066] Block 904 determines, based on the successive
`[0059] Block 804 determines which interface to present
`point, whether the gesture includes a reversal, an extension, or
`based on one or morefactors ofthe gesture. Block 804 may do
`neither. Block 904 may determinea reversal in the direction
`so based on a final or intermediate length of the gesture,
`of the gesture by determiningthat a successive pointis at the
`whetherthe gesture is single or multi-point (e.g., single-finger
`edgeoris closer to the edge than a prior point of the gesture.
`or multi-finger), or a speed of the gesture. In somecases, two
`Block 904 may determinethat the gesture extends based on
`or more factors of a gesture determine which interface to
`the successive point being a preset distance from the edge or
`present such as a drag-and-hold gesture having a drag length
`the later point. If neither of these is determined to be true,
`and hold timeor a drag-and-drop gesture having a drag length
`method 900 may repeat blocks 902 and 904 to receive and
`and drop position. Thus, block 804 may determineto present
`analyze additional successive points until the gesture ends. If
`block 904 determines that there is a reversal, method 900
`a start menu in response to a multi-finger gesture, an appli-
`cation-selection interface in response to a relatively short
`proceeds along “Reversal” path to block 906. If block 904
`single-finger gesture, or a system-control interface permitting
`determinesthat the gesture is extended, method 900 proceeds
`selection to shut down computing device 102 in response to
`along an “Extension”path to block 908.
`relatively long single-finger gesture, for example. To do so,
`[0067]
`In the context of the present example, assume that
`gesture handler 128 may determinethe length of the gesture,
`gesture handler 128 receives first successive point 1110 of
`speed, or a numberof inputs (e.g., fingers).
`gesture 1012. Gesture handler 128 then d