as) United States
`a2) Patent Application Publication co) Pub. No.: US 2008/0168404 Al
`(43) Pub. Date: Jul. 10, 2008
`
`Ording
`
`US 20080168404A1
`
`(54)
`
`LIST SCROLLING AND DOCUMENT
`TRANSLATION, SCALING, AND ROTATION
`ON A TOUCH-SCREEN DISPLAY
`
`cation No. 60/946,971, filed on Jun. 28, 2007, provi-
`sional application No. 60/937,993, filed on Jun. 29,
`2007.
`
`(75)
`
`Inventor:
`
`Bas Ording, San Francisco, CA
`(US)
`
`Correspondence Address:
`MORGAN LEWIS & BOCKIUS LLP/ APPLE
`INC.
`2 PALO ALTO SQUARE, 3000 EL CAMINO
`REAL
`PALO ALTO, CA 94306
`
`(73)
`
`Assignee:
`
`Apple Inc., Cupertino, CA (US)
`
`Publication Classification
`
`(51)
`
`Int. Cl.
`G06F 3/033
`
`(2006.01)
`
`(52) U.S. C1. cc ceeeseseeesersereeceeceeceeceeceeeneenes 715/863
`
`(57)
`
`ABSTRACT
`
`Related U.S. Application Data
`
`In accordance with some embodiments, a computer-imple-
`mented method for use in conjunction with a device with a
`touch screen display is disclosed. In the method, a movement
`of an object on or near the touch screen display is detected. In
`11/956,969
`Appl. No.:
`(21)
`response to detecting the movement, an electronic document
`displayed on the touch screen display is translated inafirst
`Filed:
`Dec. 14, 2007
`(22)
`direction. If an edge of the electronic document is reached
`while translating the electronic documentinthefirst direction
`while the object is still detected on or near the touch screen
`display, an area beyond the edge ofthe documentis displayed.
`After the object is no longer detected on or near the touch
`screen display, the documentis translated in a second direc-
`tion until the area beyond the edge of the documentis no
`longer displayed.
`
`(60)
`
`Provisional application No. 60/879,253, filed on Jan.
`7, 2007, provisional application No. 60/883,801, filed
`on Jan. 7, 2007, provisional application No. 60/879,
`469, filed on Jan. 8, 2007, provisional application No.
`60/945,858,filed on Jun. 22, 2007, provisional appli-
`
`\ees 7
`
`APPLE-1017
`
`Portable Multifunction Device
`
`Speaker 111
`
`Optical
`Sensor 164
`
`Proximity
`Sensor 166
`
`
`
`Touch Screen 112
`
`=p
`
`oe)
`204
`
`113.
`
`1
`
`APPLE-1017
`
`

`

`Patent Application Publication
`
`Jul. 10, 2008 Sheet 1 of 38
`
`US 2008/0168404 Al
`
`Portable Multifunction Device
`100
`126 Applications (continued)
`128
`130
`132
`
`136
`
`148
`
`149
`149-1
`
`134
`Stocks Widget
`135
`-
`149-30Wega Me
`
`
`
`a
`
`earch
`
`Created Widaet
`
`Module
`
`149-2
`
`149.5
`
`149-6
`
`tt
`
`Memory
`102
`
`Operating System
`Communication Module
`
`Contact/Motion Module
`
`Graphics Module
`
`Text Input Module
`GPS Module
`
`ontacts Module
`
`elephone Module
`
`
`
`-mail Client Module
`
`nstant Messaging Module
`
`Camera Module
`
`mage Management Module
`Video Player Module
`Music Player Module
`Browsing Module
`
`‘a7
`
`ns
`139
`
`a4
`442
`143
`144
`145
`146
`147
`
`
`
`ha RF Circuitry-~198
`103 110
`Speaker
`103
`111
`
`>Dl)s|m|</4]/Oeoe©°Ssa©°==OooO2©€5DooO©© = °oS £ DO
`I| Controller
`
`
`
`Power
`System
`
`External
`Port
`
`162
`
`124
`
`103
`
`cutieFe
`
`Circuitr
`
`103
`
`103
`
`118
`
`Peripherals
`
`Interface
`
`y =)
`—_
`Microphone
`— Proximity
`13
`A103
`Sensor
`166
`
`
`158
`/O Subsystem
`
`Other Input
`Optical
`
`
`
`Display
`Controller(s)
`sensor(s)
`
`Controller
`
`Controller
`
`
`160
`
`103
`
`
`
`Touch-Sensitive
`Display System
`
`Optical
`Sensor(s)
`164
`
`Figure 1
`
`Devices
`
`
`103
`
`Other Input
`Control
`
`116
`
`104
`
`122
`
`120
`
`106
`
`156
`
`112
`
`2
`
`

`

`Patent Application Publication
`
`Jul. 10,2008 Sheet 2 of 38
`
`US 2008/0168404 Al
`
`
`
`Portable Multifunction Device
`
`Speaker 111
`
`Optical
`Sensor 164
`
`Proximity
`Sensor 166
`
`J. 204 |
`
`Microphone
`13°
`
`Touch Screen 11
`
`( Home \
`
`Figure 2
`
`3
`
`

`

`Patent Application Publication
`
`Jul. 10,2008 Sheet 3 of 38
`
`US 2008/0168404 Al
`
`Portable Multifunction Device
`
`
`
`Current Time »» 308
`310 Day and Date 312
`
`. 204 |4
`
`Wallpaper image
`314
`
`Slide to unlock
`
`Touch Screen 11
`
`Microphone
`113
`
`f Home
`
`Figure 3
`
`4
`
`

`

`Patent Application Publication
`
`Jul. 10, 2008 Sheet 4 of 38
`
`US 2008/0168404 Al
`
`Portable Multifunction Device
`100
`
`
`
`
`
`
`| C)
`
`
`Se
`
`
`
`
`| 204 |
`
`
`
`ROS)
`
`Videos
`Camera
`Photos
`145
`143
`144
`
`
`~~
`(Jan |
`15
`
`
`Weather
`149-1
`
`Stocks
`149-2
`
`Calendar
`148
`
`
`
`“ User-
`Created
`.. Widget _/
`Alarm
`Widget
`Dictionary
`149-6
`149-4
`149-5
`
`Calculator
`149-3
`
`
`
`Music
`Browser
`146
`147
`
`
`Touch Screen 11
`
`Microphone
`
`113
`
`fo “\
`[ Home |
`
`Figure 4
`
`5
`
`

`

`Patent Application Publication
`
`Jul. 10,2008 Sheet 5 of 38
`
`US 2008/0168404 Al
`
`wnioiO
`
`Detect a movementof an object (e.g., a finger) on or near a touch screen
`display of a device.
`
`Scroll a list of items displayed on the touch screen displayin a first
`direction (e.g., vertical or horizontal).
`
`Scroll the list at a speed corresponding to a speed of movementof
`the object.
`
`‘Scroll thelist in accordance with a simulation of an equation of motion:
`
`512
`
`having friction.
`screen display.
`
`e819
`a Isa a
`
`terminus of the
`oS
`Lo
`a list reached while scrolling the NN No
`list in the first direction while the object
`/
`
`is still detected on or near
`the touch screen
`display?
`
`Process
`Complete
`
`514
`
`516
`
`518
`
`520
`
`522
`
`524
`
`After the object is no longer detected on or near the touch screen display,
`scroll the list in a second direction opposite the first direction until the area
`beyond the terminus ofthe list is no longer displayed.
`
`Makethe terminus ofthe list appear to be elastically attached to an
`‘edge of the touch screen display or to an edge displayed on the touch
`
`Figure 5
`
`6
`
`

`

`Patent Application Publication
`
`Jul. 10, 2008 Sheet 6 of 38
`
`US 2008/0168404 Al
`
`Portable Multifunction Device
`
`Current Time 404
`
`@b 406
`
`Mailboxes 7
`“January 30,008"
`ee ‘Bruce Wa
`ker?”
`Lunch meeting today°
`3530
`3510-3
`Kim Brook*~
`January 30, 2006~
`Draft agreement
`
`113
`
`
`
`co
`
`.
`
`*\
`
`63502
`
`Inbox
`
`3508-3
`
`Bob Adams
`January 29, 2006
`Project Orion
`3514
`6 Darin Adler
`January 29, 2006
`fantasy football
`
`3512-2
`
`Aaron Jones
`
`January 29, 2006
`Fwd: status report
`
`Kim Brook
`Re: proposal
`3532
`
`January 28, 2006
`
`Microphone
`
`|
`
`/ Home \
`\ 204 /
`
`Figure 6A
`
`7
`
`

`

`Patent Application Publication
`
`Jul. 10,2008 Sheet 7 of 38
`
`US 2008/0168404 Al
`
`Portable Multifunction Device
`
`3500A
`
`Speaker 111
`
`Optical
`
`Sensor 164
`
`Proximity
`
`Sensor 166
`
`
`
`| 402 @> 406 Current Time 404
`
`
`
`on iw¢3504
`Mailboxes 7
`Inbox
`6°
`
`\ 3
`
`. 204 |
`
`January 30, 2006"
`Aaron Jones’.
`~
`3508-1
`3534
`Re: Project Orion
`
`Bruce Walker’
`January 30, 2006
`Lunch meeting today"?
`3530
`Kim Brook! January 30, 2006"
`.
`-
`3510-3
`Draft agreement
`13514
`Bob Adams-~
`=
`January 29, 2006
`Project Orion
`V
`“Darin Adler
`fantasy football
`
`506-1
`
`3510-1
`
`3512-1
`
`3506-2
`
`3510-2
`
`3508-3!
`
`6
`
`
`
`12-2
`
`January 29, 2006
`
`January 29, 2006
`Aaron Jones
`Fwd: status report
`
`Microphone
`
`113
`
`( Home \
`
`Figure 6B
`
`8
`
`

`

`Patent Application Publication
`
`Jul. 10,2008 Sheet 8 of 38
`
`US 2008/0168404 Al
`
`100
`
`Optical
`
`Proximity
`
`3500A
`
`,
`| 402
`
`'
`Current Time 404
`
`a O
`
`Q
`“
`
`
`
`a! || Mailboxes *~Inbox
`
`eea3502
`
`3506-1
`
`3310-1
`
`
`
`le
`Aaron Jones’,January 30, 2006”
`Re: Project Orion’
`3534
`Bruce Walker”
`January 30, 20062"°”
`Lunch meeting todays*°°°?
`3530
`
`Kim Brook’! January 30, 2006"
`:
`38
`“960
`S
`Draftagreemént=859514
`
`0
`Bob Adams:
`Project Orion
`V
`3512-2@ Darin Adler
`fantasy football
`
`
`Portable Multifunction Device
`
`3512-1
`
`3506-2
`
`_
`
`3510-3
`
`January 29, 2006
`
`January 29, 2006
`
`
`
`Figure 6C
`
`9
`
`

`

`\ 204 |
`
`Patent Application Publication
`
`Jul. 10,2008 Sheet 9 of 38
`
`US 2008/0168404 Al
`
`Current Time 404
`
`@> 406
`
`
`
` -
`
`(eaoa\ ¢3502
`Sor\ 3504
`
`Mailboxes 7"~Inbox 6%
`Aaron Jones* January 30, 2006"
`.
`.
`3508
`Re: Project Orion®
`3534
`‘Bruce Walker?
`January 30, 20062
`Lunch meeting today°°°>?
`3530
`Kim Brook*” von SaNUAry 30, 2006"
`Draft agreement~
`
`3510-1
`
`-
`
`506-2
`
`.
`
`3506-3
`
`3510-2
`
`3510-3
`
`January 29, 2006
`
`Bob Adams
`Project Orion
`
`‘@ Darin Adler
`fantasy football
`
`
`January 29, 2006
`
`January 29, 2006
`Aaron Jones
`Fwd: status report
`|
`3518
`
`
`
`Microphone
`113
`
`( Home \
`
`Figure 6D
`
`10
`
`10
`
`

`

`Patent Application Publication
`
`Jul. 10,2008 Sheet 10 of 38
`
`US 2008/0168404 Al
`
`~ iSiS
`
`Detect a movementof an object (e.g., a finger) on or near a touch screen
`display.
`
`Translate an electronic document displayed on the touch screen display in
`a first direction (e.g., vertical, horizontal, or diagonal).
`
`Translate the electronic document at a speed oftranslation
`corresponding to a speed of movementof the object.
`
`Translate the electronic documentin accordance with a simulation of
`
`
`
`an equation of motion havingfriction.
`displayed on the touch screen display.
`
`isanTO
`a 712
`edgeofthe =
`_-
`_--~ electronic document reached ~~
`__-while translatingthe electronic document inN°|Process
`oN _the first direction while the objectis still a Complete
`detected on or near aa
`oN
`“s.
`the touch screen
`display? 0-7
`
`
`
`
`
`714
`
`716
`
`718
`
`720
`
`722
`
`724
`
`After the object is no longer detected on or near the touch screen display,
`translate the document in a second direction (¢.g., opposite the first
`direction) until the area beyond the edge of the documentis no longer
`displayed.
`
`Make the edgeof the electronic document appear to be elastically
`attached to an edge of the touch screendisplay or to an edge
`
`Figure 7
`
`11
`
`11
`
`

`

`
`
`
`
`
`
`
`Patent Application Publication
`
`Jul. 10,2008 Sheet 11 of 38
`
`US 2008/0168404 Al
`
`Portable Multifunction Device
`00
`
`Optical
`164
`
`S
`
`Proximity
`1
`
`Current Time 404 co}
`
`Welcome 3904
`
`
`http:/Awww.company.com/start 3908
`
`
`
`Block 3
`
`3914-3
`
`
`
`Block 1
`
`3914-1
`
`$928
`
`
`Block 4
`3914-4
`
`
`
`Block 5
`
`3914-5
`
`Block 2
`
`3914-2
`
`
`
`
`Block 6
`
`3914-6
`
`~
`
`\7)\
`3916
`
`Microphone
`113
`
`™,
`
`/ Home \
`\ 204 ;
`SN7
`
`Figure 8A
`
`12
`
`12
`
`

`

`Patent Application Publication
`
`Jul. 10,2008 Sheet 12 of 38
`
`US 2008/0168404 Al
`
`206
`
`Portable Multifunction Device
`100
`
`Current Time 404
`
`
`
`Welcome 3904
`
`http:/Awww.company.com/start 3908
`
`( J
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Block 1
`
`3914-1
`
`Block 2
`
`3914-2
`
`Block 4
`
`3914-4
`
`Block 5
`3925 3914-5
`wy,
`
`*
`
`.
`
`s
`
`®
`
`e
`Block 6
`
`3914-6
`
`Web page
`3912
`
`Block 7
`
`3914-7
`3914-7
`
`Block 8
`
`3914-8
`
`Block 9
`3914-9
`
`
`
`Microphone
`113
`
`/ Home \
`\ 204 |
`
`Figure 8B
`
`13
`
`13
`
`

`

`Patent Application Publication
`
`Jul. 10,2008 Sheet 13 of 38
`
`US 2008/0168404 Al
`
`Portable Multifunction Device
`
`Current Time 404
`
`Welcome 3904
`
`
`3906
`
`http://www.company.com/start 3908 Cj
`
`3914-1
`
`Block 4
`3914-4
`
`Block 7
`3914-7
`
`
`
`Block 5
`
`3914-5
`
`wer
`
`®
`
`
`
`
`
`\ 204 |
`
`®
`
`“
`
`.
`
`BlocRs6
`3914-6
`
`Web page
`
`3012.
`
`Block 9
`
`3914-9
`
`Microphone)/ Home \
`413
`
`Figure 8C
`
`14
`
`14
`
`

`

`Patent Application Publication
`
`Jul. 10,2008 Sheet 14 of 38
`
`US 2008/0168404 Al
`
`206
`
`Portable Multifunction Device
`100
`
`Current Time 404
`
`
`
`Welcome 3904
`
`
`
`http:/Awww.company.com/start 3908
`
`
`
`
`
`Block 1
`
`3914-1
`
`Block7
`Block 4
`sgitr
`3914-4
`
`
`
`Block 5
`
`3914-5
`
`Block 2
`
`3914-2
`
`
`
`
`204 |
`
`
`
`
`
`Block 6
`
`3914-6
`
`Web page
`
`Block 9
`3914-9
`
`Microphone
`
`113
`
`\.
`
`f Home |
`
`Figure 8D
`
`15
`
`15
`
`

`

`Patent Application Publication
`
`Jul. 10,2008 Sheet 15 of 38
`
`US 2008/0168404 Al
`
`900
`
`Display an electronic documentata first magnification on a touch screen
`display. The electronic document has a document length and a document
`width.
`
`a pinching gesture) on or near the touch screen
`Detect a gesture (€.g.,
`display corresponding to a command to zoom out by a user-specified
`amount.
`
`In response to detecting the gesture, display the electronic documentat a
`magnification less than the first magnification.
`
`902
`
`904
`
`906
`
`
`
`
`the electronic document are no longer displayed.
`
`— ro’
`/
`cr
`a Is the a
`we
`a documentlength or
`Process
`_~ documentwidth entirely displayed while ~No
`S anal: .
`
`
`
`“the gesture is still detected on or near ~~
`Complete
`oN
`the touch screen
`ee
`a display? a
`
`
`
`
`
`Display the electronic documentat a magnification wherein areas beyond
`opposite edges of the electronic documentare displayed.
`
`Upon detecting termination of the gesture, display the electronic
`documentat a magnification wherein the areas beyond opposite edges of
`
`912
`
`914
`
`Figure 9
`
`16
`
`16
`
`

`

`Patent Application Publication
`
`Jul. 10,2008 Sheet 16 of 38
`
`US 2008/0168404 Al
`
`206
`
`Portable Multifunction Device
`100
`
`Current Time 404
`
`Welcome 390
`
`
`
`
`
`http:/Awww.company.com/start 3908
`
`
`
`Block 3
`
`3914-3
`
`Block 1
`
`3914-1
`
`
`
`
`
`\ 204 |
`
`
`
`2eablock 4
`039513
`Soe 3914-4
`
`
`
`59531
`+0?
`
`Block 5
`3914-5
`
`Block 2
`
`3914-2
`
`
`
`
`Block 6
`
`3914-6
`
`Microphone
`113
`
`( Home \
`
`Figure 10A
`
`17
`
`
`
`
`
`
`17
`
`

`

`
`
`3955
`
`
`
`Patent Application Publication
`
`Jul. 10,2008 Sheet 17 of 38
`
`US 2008/0168404 Al
`
`206
`
`Portable Multifunction Device
`100
`
`Current Time
`
`Welcome 3904
`
`
`
`Block 3
`3914-3
`
`Web page
`3912
`
`
`
`Block 7
`
`3955
`
` http:/Awww.company.com/start 39038
`
`
`
`
`
`
`
`@0
`
`3951339144
`
`
`
`
`Block 5
`3914-5
`
`oe
`Block 28395:
`3914-2
`
`Block 9
`3914-9
`
`Microphone
`113
`
`/ Home \
`\ 204 ;
`/
`4
`
`Figure 10B
`
`18
`
`18
`
`

`

`Patent Application Publication
`
`Jul. 10,2008 Sheet 18 of 38
`
`US 2008/0168404 Al
`
`Portable Multifunction Device
`
`Current Time 404
`
`Welcome 3904
`
`
`http:/Awww.company.com/start 3908
`
`
`
`
`
`
`
`
`
`
`
`
`
`\ 204 /4
`
`Block 3
`3914-3
`
`Block 4
`3914-4
`
`
`Block 5
`3914-5
`
`Block 2
`3914-2
`
`
`
`Microphone)/ Home \
`113
`
`Figure 10C
`
`19
`
`19
`
`

`

`Patent Application Publication
`
`Jul. 10,2008 Sheet 19 of 38
`
`US 2008/0168404 Al
`
`Display at least a first portion of an electronic documentata first
`magnification on a touch screen display.
`
`Detect a gesture (€.g., a de-pinching gesture) on or near the touch screen
`display corresponding to a command to zoom in by a user-specified
`amount.
`
`In responseto detecting the gesture, display decreasing portions of the
`
`electronic documentat increasing magnifications. ,
`
`1102
`
`1104
`
`1106
`
`1110
`
`“1108a
`a Upon detecting
`NN
`termination of the gesture, does the
`magnification exceed a predefined
`oN
`magnification?
`
`_~
`“\-
`
`oa
`
`Process
`Complete
`
`_
`
`Display a respective portion of the electronic documentat the predefined
`
`magnification.
`
`1112
`
`Figure 11
`
`20
`
`20
`
`

`

`Patent Application Publication
`
`Jul. 10,2008 Sheet 20 of 38
`
`US 2008/0168404 Al
`
`Portable Multifunction Device
`
`
`
`Current Time 404
`
`906
`3904
`Welcome
`a
`
`
`http:/Awww.company.com/start 3908
`
`
`
`
`
`( i
`
`. 204 |
`
`
`
`
`
`Block 3
`
`3914-3
`
`
`
`Block 1037
`3914-1
`A-~
`
`Block 4
`iene
`
`13933) 3914-4
`
`
`
`Block 5
`
`3914-5
`
`Block 6
`
`3914-6
`
`Microphone
`
`113
`
`( Home \
`
`\
`
`Figure 12A
`
`21
`
`21
`
`

`

`Patent Application Publication
`
`Jul. 10,2008 Sheet 21 of 38
`
`US 2008/0168404 Al
`
`206
`
`Portable Multifunction Device
`100
`
`Current Time 404
`
`
`
`
`
`Welcome 3904
`
`
`
`
`http:/Awww.company.com/start 3908
`
`
`
`
`
`
`
`|
`
`Web page
`3912
`
`03933)
`s
`@
`
`Block 1
`
`3914-1
`
`ON
`
`f Home \
`Microphone
`13). 204
`
`Figure 12B
`
`22
`
`
`
`22
`
`

`

`Patent Application Publication
`
`Jul. 10,2008 Sheet 22 of 38
`
`US 2008/0168404 Al
`
`206
`
`Portable Multifunction Device
`100
`
`Current Time 404
`
`
`
`Welcome 3904
`
`
`
`
`http:/Awww.company.com/start 3908
`
`
`
`| 204 |
`
`Web page
`3912
`
`
`
`
`
`Microphone
`
`13)
`
`fo
`
`fl Home \
`
`Figure 12C
`
`23
`
`23
`
`

`

`Patent Application Publication
`
`Jul. 10,2008 Sheet 23 of 38
`
`US 2008/0168404 Al
`
`
`
`’ToFigure13B
`
`BOL
`(s)da}oWOIa|B90V
`
`0
`WOH
`
`CLE
`suoydoai/
`
`(Frie¥P2¢s
`
`99} Josuas
`
`FOL 1osuag
`Tonio)
`
`—_
`
`24
`
`
`
`Figure13A
`
`24
`
`

`

`Patent Application Publication
`
`Jul. 10,2008 Sheet 24 of 38
`
`US 2008/0168404 Al
`
`
`
`ToFigure13C
`
`
`
`
`
`
`
`B91
`(s)la}BWOJE[B00V
`
`p02
`aWwoH
`
`ELE
`auoydool||
`
`13B 99T Josues
`
`Figure
`
`POL 1osuas —
`
`25
`
`25
`
`

`

`Patent Application Publication
`
`Jul. 10,2008 Sheet 25 of 38
`
`US 2008/0168404 Al
`
`Bor
`(s)JoJQWOJB|9N9Y
`
`p0e
`WOH
`
`Ebb
`auoydosoiy/|
`
`13C GOr1osues)
`
`Figure
`
`(Poriosuss)
`
`(——
`
`26
`
`26
`
`

`

`Patent Application Publication
`
`Jul. 10,2008 Sheet 26 of 38
`
`US 2008/0168404 Al
`
`1400
`
`
`
`Detect a multifinger twisting gesture on or near a touch screendisplay.
`The multifinger twisting gesture has a corresponding degreeof rotation.
`
`1402
`
`
`
`ro 1408
`
`.
`
`:
`
`oN co
`Does the > -
`;
`Execute a 90°
`_”corresponding degreeof NN Yes
`rotation exceed a > screen rotation
`— predefined degree of
`“oO
`command.
`:
`rotation?
`
`
`
`
`
`“No
`
`Execute a screen rotation commandwith an acute angle ofrotation.
`
`angle.
`
`Upon ceasing to detect the multifinger twisting gesture, executing a
`screen rotation command with an angle of rotation opposite to the acute
`
`1408
`
`1410
`
`Figure 14
`
`27
`
`27
`
`

`

`Patent Application Publication
`
`Jul. 10,2008 Sheet 27 of 38
`
`US 2008/0168404 Al
`
`Speaker 111
`
`Optical
`Sensor 164
`
`Proximity
`Sensor 166
`
`
`
`
`
`To Figure 15B
`
`168
`
`Microphone
`113
`
`Home
`204
`
`Accelerometer(s)
`
`Figure 15A
`
`28
`
`28
`
`

`

`Patent Application Publication
`
`Jul. 10,2008 Sheet 28 of 38
`
`US 2008/0168404 Al
`
`Optical Sensor
`
`Proximity
`
`Accelerometer(s)
`
`.
`
`Home
`
`Figure 15B
`
`29
`
`29
`
`

`

`Patent Application Publication
`
`Jul. 10,2008 Sheet 29 of 38
`
`US 2008/0168404 Al
`
`Speaker 111
`
`Optical
`Sensor 164
`
`Proximity
`Sensor 166
`
`168
`
`Microphone
`113
`
`Accelerometer(s)
`
`Figure 15C
`
`30
`
`To Figure 15D
`
`30
`
`

`

`Patent Application Publication
`
`Jul. 10,2008 Sheet 30 of 38
`
`US 2008/0168404 Al
`
`Speaker 111
`
`Optical
`Sensor 164
`
`Proximity
`Sensor 166
`
`
`
`
`
`To Figure 15E
`
`168
`
`Microphone
`113
`
`Home
`204
`
`Accelerometer(s)
`
`Figure 15D
`
`31
`
`31
`
`

`

`Patent Application Publication
`
`Jul. 10,2008 Sheet 31 of 38
`
`US 2008/0168404 Al
`
`Speaker 111
`
`Optical
`Sensor 164
`
`Proximity
`Sensor 166
`
`168
`
`Microphone
`113
`
`Home
`204
`
`Accelerometer(s)
`
`Figure 15E
`
`32
`
`32
`
`

`

`Patent Application Publication
`
`Jul. 10,2008 Sheet 32 of 38
`
`US 2008/0168404 Al
`
`Optical
`Sensor 164
`
`Proximity
`Sensor 166
`
`1502
`
` Microphone
`
`Speaker 111
`
`113
`
`Figure 16A
`
`33
`
`33
`
`

`

`Patent Application Publication
`
`Jul. 10,2008 Sheet 33 of 38
`
`US 2008/0168404 Al
`
`LED
`
`1604-L P
`
`Figure 16B
`
`34
`
`34
`
`

`

`Patent Application Publication
`
`Jul. 10,2008 Sheet 34 of 38
`
`US 2008/0168404 Al
`
`
`
`>
`
`r
`
`
`
`Figure 16C
`
`35
`
`35
`
`

`

`Patent Application Publication
`
`Jul. 10,2008 Sheet 35 of 38
`
`US 2008/0168404 Al
`
`
` To Figure 16E
`
`
`
`Figure 16D
`
`36
`
`36
`
`

`

`Patent Application Publication
`
`Jul. 10,2008 Sheet 36 of 38
`
`US 2008/0168404 Al
`
`1604-L
`
`Speaker 111
`
`Optical
`Sensor 164
`
`Proximity
`Sensor 166
`
`
`
` YY
`
`To Figure 16F
`
`1604-R
`
`Microphone
`
`113
`
`Home
`
`204
`
`Accelerometer(s)
`
`168
`
`f
`
`Figure 16E
`
`37
`
`37
`
`

`

`Patent Application Publication
`
`Jul. 10,2008 Sheet 37 of 38
`
`US 2008/0168404 Al
`
`Optical Sensor
`
`Proximity
`
`
`
`Microphone
`413
`
`Home
`204
`
`Accelerometer(s)
`168
`
`Figure 16F
`
`38
`
`38
`
`

`

`Patent Application Publication
`
`Jul. 10, 2008 Sheet 38 of 38
`
`US 2008/0168404 Al
`
`Memory 1770
`
`-mail Client Module
`
`
`
`5|m
`
`CPU(s)
`
`1720
`
`Device 1700
`
`\ 1
`
`710
`
`1730
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`126
`
`128
`130
`132
`
`134
`136
`
`137
`
`138
`
`139
`
`140
`
`144
`
`142
`
`143
`
`144
`
`145
`
`146
`
`147
`
`148
`
`149
`
`149-1
`
`149-2
`
`149-3
`
`149-4
`
`149-5
`
`150
`
`151
`
`
`
`
`
`User interface
`
`Touch-Screen
`Display
`
`Keyboard/Mouse
`
`1740
`
`
`
`
`
`Network
`1750
`
`1760|Communications
`fp
`
`Interface
`149-6
`
`Search Module
`
`Figure 17
`
`39
`
`39
`
`

`

`US 2008/0168404 Al
`
`Jul. 10, 2008
`
`LIST SCROLLING AND DOCUMENT
`TRANSLATION, SCALING, AND ROTATION
`ON A TOUCH-SCREEN DISPLAY
`
`RELATED APPLICATIONS
`
`[0001] This application claimspriority to U.S. Provisional
`Patent Application Nos. 60/937,993, “Portable Multifunction
`Device,”filed Jun. 29, 2007; 60/946,971, “List Scrolling and
`Document Translation, Scaling, and Rotation on a Touch-
`Screen Display,” filed Jun. 28, 2007; 60/945,858, “List
`Scrolling and DocumentTranslation on a Touch-Screen Dis-
`play,” filed Jun. 22, 2007; 60/879,469, “Portable Multifunc-
`tion Device,” filed Jan. 8, 2007; 60/883,801, “List Scrolling
`and DocumentTranslation on a Touch-Screen Display,” filed
`Jan. 7, 2007; and 60/879,253, “Portable Multifunction
`Device,” filed Jan. 7, 2007. All of these applications are
`incorporated by reference herein in their entirety.
`[0002] This application is related to the following applica-
`tions: (1) U.S. patent application Ser. No. 10/188, 182, “Touch
`Pad For Handheld Device,” filed on Jul. 1, 2002; (2) U.S.
`patent application Ser. No. 10/722,948, “Touch Pad For
`Handheld Device,” filed on Nov. 25, 2003; (3) U.S. patent
`application Ser. No. 10/643,256, “Movable Touch Pad With
`Added Functionality,” filed on Aug. 18, 2003; (4) U.S. patent
`application Ser. No. 10/654,108, “Ambidextrous Mouse,”
`filed on Sep. 2, 2003; (5) U.S. patent application Ser. No.
`10/840,862, “Multipoint Touchscreen,”filed on May 6, 2004;
`(6) US. patent application Ser. No. 10/903,964, “Gestures
`For Touch Sensitive Input Devices,”filed on Jul. 30, 2004; (7)
`USS. patent application Ser. No. 11/038,590, “Mode-Based
`Graphical User
`Interfaces For Touch Sensitive Input
`Devices”filed on Jan. 18, 2005; (8) U.S. patent application
`Ser. No. 11/057,050, “Display Actuator,” filed on Feb. 11,
`2005; (9) U.S. Provisional Patent Application No. 60/658,
`777, “Multi-Functional Hand-Held Device,” filed Mar. 4,
`2005;
`(10) U.S. patent application Ser. No. 11/367,749,
`“Multi-Functional Hand-Held Device,” filed Mar. 3, 2006;
`and (11) U.S. Provisional Patent Application No. 60/824,769,
`“Portable Multifunction Device,” filed Sep. 6, 2006. All of
`these applications are incorporated by reference herein in
`their entirety.
`
`TECHNICAL FIELD
`
`[0003] The disclosed embodiments relate generally to
`devices with touch-screen displays, and moreparticularly to
`scrolling lists and to translating, rotating, and scaling elec-
`tronic documents on devices with touch-screen displays.
`
`BACKGROUND
`
`[0004] As portable electronic devices become more com-
`pact, and the number of functions performed by a given
`device increases,
`it has becomea significant challenge to
`design a userinterface that allows users to easily interact with
`a multifunction device. This challenge is particularly signifi-
`cant for handheld portable devices, which have much smaller
`screens than desktop or laptop computers. This situation is
`unfortunate becausethe user interface is the gateway through
`which users receive not only content but also responses to
`user actions or behaviors, including user attempts to access a
`device’s features, tools, and functions. Some portable com-
`munication devices (e.g., mobile telephones, sometimes
`called mobile phones, cell phones, cellular telephones, and
`the like) have resorted to adding more pushbuttons, increas-
`
`ing the density of push buttons, overloading the functions of
`pushbuttons, or using complex menu systemsto allow a user
`to access, store and manipulate data. These conventional user
`interfaces often result in complicated key sequences and
`menuhierarchies that must be memorized bythe user.
`[0005] Many conventional user interfaces, such as those
`that include physical pushbuttons, are also inflexible. This is
`unfortunate because it may prevent a user interface from
`being configured and/or adaptedby either an application run-
`ning on the portable device or by users. When coupled with
`the time consuming requirement to memorize multiple key
`sequences and menuhierarchies, and thedifficulty in activat-
`ing a desired pushbutton, such inflexibility is frustrating to
`most users.
`
`[0006] Asa result of the small size of display screens on
`portable electronic devices and the potentially large size of
`electronic files, frequently only a portion of a list or of an
`electronic documentof interest to a user can be displayed on
`the screen at a given time. Users thus will frequently need to
`scroll displayedlists or to translate displayed electronic docu-
`ments. Users also will need to rotate and to scale (i.e., mag-
`nify or de-magnify) displayed electronic documents. How-
`ever, the limitations of conventional user interfaces can cause
`these actions to be awkward to perform.
`[0007]
`Furthermore, scrolling displayed lists and translat-
`ing electronic documents can be awkward on both portable
`and non-portable electronic devices with touch-screen dis-
`plays. A user may becomefrustrated if the scrolling or trans-
`lation does not reflect the user’s intent. Similarly, a user may
`becomefrustrated if rotation and scaling of electronic docu-
`ments doesnotreflect the user’s intent.
`[0008] Accordingly, there is a need for devices with touch-
`screen displays with moretransparentand intuitiveuser inter-
`faces for scrolling lists of items andfor translating, rotating,
`and scaling electronic documentsthatare easyto use, config-
`ure, and/or adapt.
`
`SUMMARY
`
`[0009] The above deficiencies and other problemsassoci-
`ated with userinterfaces for portable devices and devices with
`touch-sensitive displays are reduced or eliminated by the
`disclosed device. In some embodiments, the device has a
`touch-sensitive display (also knownas a “touch screen”) with
`a graphical user interface (GUI), one or more processors,
`memory and one or more modules, programs or sets of
`instructions stored in the memory for performing multiple
`functions. In some embodiments, the user interacts with the
`GUI primarily through finger contacts and gestures on the
`touch-sensitive display. In some embodiments,the functions
`may include telephoning, video conferencing, e-mailing,
`instant messaging, blogging, digital photographing, digital
`videoing, web browsing, digital music playing, and/or digital
`video playing. Instructions for performing these functions
`maybe included in a computer program product configured
`for execution by one or moreprocessors.
`[0010]
`Inaccordance with some embodiments, a computer-
`implemented method for use in conjunction with a device
`with a touch screen display is disclosed. In the method, a
`movement of an object on or near the touch screen display is
`detected. In response to detecting the movement, an elec-
`tronic document displayed on the touch screen display is
`translated in a first direction. If an edge of the electronic
`document is reached while translating the electronic docu-
`mentin the first direction while the objectis still detected on
`
`40
`
`40
`
`

`

`US 2008/0168404 Al
`
`Jul. 10, 2008
`
`edge of the electronic documentif the edge of the electronic
`or near the touch screen display, an area beyond the edge of
`document is reached while translating the electronic docu-
`the document is displayed. After the object is no longer
`mentin the first direction while the objectis still detected on
`detected onor near the touch screen display, the documentis
`or near the touch screen display. The device further comprises
`translated in a seconddirection until the area beyond the edge
`meansfortranslating the documentin a seconddirection until
`of the documentis no longer displayed.
`the area beyond the edge of the documentis no longer dis-
`[0011]
`In accordance with some embodiments, a graphical
`played, after the object is no longer detected on or near the
`user interface on a device with a touch screen display is
`touch screen display.
`disclosed, comprising a portion of an electronic document
`[0015]
`Inaccordance with some embodiments, a computer-
`displayed on the touch screen display and an area beyond an
`implemented method for use in conjunction with a device
`edge of the document. In the graphical user interface, in
`with a touch screen display is disclosed. In the method, a
`response to detecting a movementof an object on or near the
`movement of an object on or near the touch screen display is
`touch screen display, the electronic documentis translated in
`detected. In response to detecting the movement, a list of
`a first direction. If the edge of the electronic documentis
`items displayed on the touch screen display is scrolled in a
`reached while translating the electronic documentin thefirst
`first direction. Ifa terminusofthe list is reached whilescroll-
`direction while the object is still detected on or near the touch
`ing the list in the first direction while the object is still
`screen display, the area beyond the edge of the documentis
`detected on or near the touch screen display, an area beyond
`displayed. After the object is no longer detected on or nearthe
`the terminus ofthe list is displayed. After the object is no
`touch screen display, the documentis translated in a second
`longer detected on or near the touch screen display, the list is
`direction until the area beyondthe edge ofthe documentis no
`scrolled in a second direction opposite the first direction until
`longer displayed.
`the area beyondthe terminusofthe list is no longer displayed.
`[0012]
`In accordance with some embodiments, a device is
`[0016]
`In accordance with some embodiments, a graphical
`disclosed, comprising a touch screen display, one or more
`user interface on a device with a touch screen display is
`processors, memory, and one or more programs. The one or
`disclosed, comprising a portion of a list of items displayed on
`more programsare stored in the memory and configured to be
`the touch screen display and an area beyonda terminusofthe
`executed by the one or more processors. The one or more
`list. In response to detecting a movementof an object on or
`programsincludeinstructions for detecting a movement of an
`
`objecton or near the touch screen display andinstructions for near the touch screen display, the list is scrolled inafirst
`translating an electronic document displayed on the touch
`direction. If the terminusofthe list is reached whilescrolling
`screen display in a first direction, in response to detecting the
`thelist in thefirst direction while the objectis still detected on
`movement. The one or more programsalso include instruc-
`or near the touch screen display, the area beyondthe terminus
`tions for displaying an area beyondan edgeofthe electronic
`of the list is displayed. After the object is no longer detected
`documentif the edge of the electronic documentis reached
`on or near the touch screen display, the list is scrolled in a
`while translating the electronic documentinthefirst direction
`second direction opposite the first direction until the area
`while the object is still detected on or near the touch screen
`beyondthe terminusofthe list is no longer displayed.
`display. The one or more programs further include instruc-
`[0017]
`In accordance with some embodiments, a device is
`tions for translating the documentin a seconddirection until
`disclosed, comprising a touch screen display, one or more
`the area beyond the edge of the documentis no longer dis-
`processors, memory, and one or more programs. The one or
`played, after the object is no longer detected on or near the
`more programsare stored in the memory and configured to be
`touch screen display.
`executed by the one or more processors. The one or more
`[0013]
`Inaccordance with some embodiments, a computer-
`programsincludeinstructions for detecting a movement ofan
`program product is disclosed, comprising a computer read-
`object on or near the touch screen display and instructions for
`able storage medium and a computer program mechanism
`scrolling a list of items displayed on the touch screen display
`embedded therein. The computer program mechanism com-
`ina first direction in response to detecting the movement. The
`prises instructions, which when executed by a device with a
`one or more programsalso includeinstructions for displaying
`touch screen display, cause the device to detect a movement
`an area beyond a terminusofthelist if the terminusofthe list
`of an object on or near the touch screen display andto trans-
`is reached while scrolling thelist in thefirst direction while
`late an electronic document displayed on the touch screen
`the objectis still detected on or near the touch screen display.
`display in a first direction, in response to detecting the move-
`The one or more programs further include instructions for
`ment. Theinstructions also cause the device to display an area
`scrolling thelist in a second direction opposite thefirst direc-
`beyond an edge of the electronic documentif the edge of the
`tion until the area beyondthe terminusofthelist is no longer
`electronic document is reached while translating the elec-
`displayed, after the object is no longer detected on or near the
`tronic documentin the first direction while the object is still
`touch screen display.
`detected on or near the touch screen display. The instructions
`[0018]
`Inaccordance with some embodiments, a computer-
`further cause the device to translate the documentin a second
`program product is disclosed, comprising a computer read-
`direction until the area beyondthe edge ofthe documentis no
`able storage medium and a computer program mechanism
`longer displayed, after the object is no longer detected on or
`embeddedtherein. The computer program mechanism com-
`near the touch screen display.
`prises instructions, which when executed by a device with a
`[0014]
`In accordance with some embodiments, a device
`touch screen display, cause the device to detect a movement
`with a touch screen display is disclosed. The device com-
`of an object on or near the touch screen display and to scroll
`prises meansfor detecting a movementof an object on or near
`a list of items displayed on the touch screen display in a first
`the touch screen display and meansfor translating an elec-
`direction in response to detecting the movement. Theinstruc-
`tronic document displayed on the touch screen display in a
`tions also cause the device to display an area beyond a termi-
`nus of the list if the terminus of the list is reached while
`first dire