US 8,493,344 B2
`(10) Patent No.:
`a2) United States Patent
`Fleizachetal.
`(45) Date of Patent:
`Jul, 23, 2013
`
`
`US008493344B2
`
`(75)
`
`(54) DEVICES, METHODS, AND GRAPHICAL
`USER INTERFACES FOR ACCESSIBILITY
`USING A TOUCH-SENSITIVE SURFACE
`Inventors: Christopher Brian Fleizach, Santa
`Clara, CA (US); Eric Taylor Seymour,
`San Jose, CA (US)
`(73) Assignee: Apple Inc., Cupertino, CA (US)
`
`(*) Notice:
`
`Subject to any disclaimer, the term ofthis
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 740 days.
`
`(21) Appl. No.: 12/565,745
`
`(22)
`
`Filed:
`
`Sep. 23, 2009
`
`(65)
`
`(51)
`
`(56)
`
`Prior Publication Data
`US 2010/0309147 Al
`Dec. 9, 2010
`Related U.S. Application Data
`(60) Provisional application No. 61/184,825, filed on Jun.
`7, 2009.
`Int. Cl.
`GO6F 3/041
`(52) US. Cl.
`USPC ciecceecceccstccestteeees 345/173; 715/863; 715/864
`(58) Field of Classification Search
`None
`See application file for complete search history.
`.
`References Cited
`U.S. PATENT DOCUMENTS
`eae “
`oyool MeAvinncy
`;
`;
`ornettet
`5502803 A
`3/1996 Yoshida et al.
`5.761.485 A
`6/1998 Munyan
`5,832,528 A
`11/1998 Kwatinetz et al.
`5,943,043 A
`8/1999 Furuhataet al.
`
`(2006.01)
`
`al.
`
`6,046,722 A
`6,088,023 A
`C128007 A
`6,211,856 Bl
`
`4/2000 McKiel, Jr.
`7/2000 Louis etal.
`102000 Seybold
`4/2001 Choietal.
`.
`(Continued)
`FOREIGN PATENT DOCUMENTS
`bute boy AQ
`ioitooe
`7 321889
`12/1995
`WO 92/08183
`5/1992
`OTHER PUBLICATIONS
`International Search Report and Written Opinion dated Jun. 22, 2011,
`received in International Application No. PCT/US2010/034109,
`which corresponds to U.S. Appl. No. 12/565,744.
`
`be
`JP
`WO
`
`(Continued)
`
`Primary Examiner — Lixi C Simpson
`(74) Attorney, Agent, or Firm — Morgan, Lewis & Bockius
`LLP
`ABSTRACT
`(57)
`An accessibility method is performed by an electronic device
`with a display and a touch-sensitive surface. The method
`includes: displaying a plurality ofuser interface elements on
`the display; in response to detecting a first user interface
`navigation gesture by a finger on the touch-sensitive surface,
`navigating in the plurality ofuser interface elements in accor-
`dance with a current navigable unit type;
`in response to
`detecting a first user interface navigation setting gesture on
`the touch-sensitive surface: changing the current navigable
`unit type from the first navigable unit type to a second navi-
`gable unit type; and outputting accessibility information
`about the second navigable unit type; after changing the cur-
`rent navigable unit type, in response to detecting a second
`userinterface navigation gesture by the finger on the touch-
`oa:
`soating
`7
`‘
`‘
`sensitive surface, navigatinginthe platy ofser interface
`elements in accordance wi
`e current navigable unit
`type.
`
`25 Claims, 84 Drawing Sheets
`
`Portable Multifunction Device 100
`Memory
`
`
`
`— 5
`L128 Fepcaons contmaaa
`> p16
`ng
`Mo
`perating System
`1 147
`
`
`Loses 1a|[smarieane—1-18
`
`
`
` 130|[Widget Modules[Contact/Motion Module 4901
`
`[Graphics Module
`[er 132
`[Weather Widget
`vag
`AttachmentEditing Module
`p/~ 133
`[Stocks Widget
`bo
`
`[Text Input Module
`138
`Calculator Widgee
`p48
`
`
`[Alarm Glock WidgetDictlonaryWidget
`
`|”tee1
`
`
`
`
`
`
`
`
`
`
`ele;
`a
`TaerGroated Wiageley Lo we
`[Video
`rence Module
`140
`Widget Creator Module 54
`
`
`omiaseegopaaa 181|Erect Ree I
`
`|Voica Mamo Module
`be 2 Le
`[Camera Module
`Ln 18
`Power }~ 182
`
`
`
`image Management Module [~ “a
`System
`iseFosermeaue
`4 #°o|
`ponf
`118 Ca RFSo Spaeker
`a
`feiss
`
`
`
`
`
`
`
`
`
`
`Circuitry
`bt
`122 —
`
`[stee)aa ue
`ll Controller
`
`L110)
`103
`"inorace’ |le =e
`168
`120-7]
`Processor{s}
`403
`Senear
`
`t
`Accelerometer(s)
`
`1s
`#8
`VO Subsystem
`188
`
`
`
`
`
`Display
`onal )
`Other Input
`HH Controller
`Controller
`Contratier(s) [7
`
`103,
`top
`tes
`vor] someeaie||smares||cone” pH
`
`160
`
`F#
`
`108 4
`156
`
`APPLE 1114
`
`APPLE 1114
`
`1
`
`

`

`US 8,493,344 B2
`
`Page 2
`
`U.S. PATENT DOCUMENTS
`
`* cited by examiner
`
`CNET,“Sony Ericsson W910,”posts, the earliest ofwhich is Oct. 17,
`2007, 4 pages, http://news.cnet/crave/?keyword=Sony+Ericsson+
`6/2001 Zouet al.
`6,246,983 Bl
`W910.
`5/2002 Vanderheiden
`6,384,743 Bl
`Esther, “ GarageBand,” AppleVis, Mar. 11, 2011, http://www.
`5/2002 Segal et al.
`6,396,523 Bl
`applevis.com/app-directory/music/garageband, 4 pages.
`8/2002 Siegel
`6,442,523 Bl
`Immersion, “Haptics:
`Improving the Mobile User Experience
`9/2002 Reynar etal.
`6,446,041 Bl
`through Touch,” Immersion Corporation White Paper, © 2007
`10/2002 Van Ee
`6,466,203 B2
`Immersion Corporation, 12 pages, http://www.immersion.com/docs/
`12/2002 Wongetal.
`6,489,951 Bl
`haptics__mobile-ue_nov07v1.pdf.
`7/2004 Segalet al.
`6,765,557 Bl
`Jaques, R., “HP unveils Pocket PC forblind users,” vnunet.com,Jul.
`8/2005 Martin
`6,926,609 B2
`5, 2004,http://www.vntunet.com/vnunet/news/2125404/hp-unveils-
`6/2006 Kovaleset al.
`7,062,437 B2
`pocket-pe-blind-users, 3 pages.
`3/2007 Chandane
`7,187,394 B2
`Kaneet al., “Slide Rule: Making Mobile Touch Screens Accessible to
`5/2008 Sullivan et al.
`7,376,523 B2
`Blind People Using Multi-Touch Interaction Techniques,” Proceed-
`8/2008 Hinckleyetal.
`7,408,538 B2
`1/2009 Jobset al.
`7,479,949 B2
`ings of ACM SIGACCESS Conference on Computers and. Accessi-
`10/2009 Toyamaetal.
`7,603,621 B2
`bility. Halifax, Nova Scotia, Canada, Oct. 2008, 8 pages.
`7,637,421 Bl=12/2009 Trocme
`Kendrick,D., “The Touch That Means So Much: Training Materials
`8,059,101 B2
`11/2011 Westerman etal.
`for Computer Users Who Are Deaf-Blind,” AFB AccessWorld, Mar.
`8,103,554 B2
`1/2012 Tom
`2005, 2,_http://‘www.afb.org/afbpress/pub.vol. 6, No.
`
`
`
`
`2002/0133350 Al
`9/2002 Cogliano
`asp?DocID=aw060207, 9 pages.
`2003/0046082 Al
`3/2003 Siegel
`Microsoft, “Pocket PC Device for Blind Users Debuts during
`12/2004 Ishigki
`2004/0263491 Al
`6/2005 Chambersetal.
`2005/0134578 Al
`National Disability Employment Awareness Month,” Microsoft.com
`9/2005 Marvitetal. oe. 715/863
`2005/0216867 Al*
`PressPass, Oct. 16, 2002, http://www.microsoft.com/presspass/fea-
`6/2006 Yoonet al.
`2006/0119588 Al
`tures/2002/oct02/10-16ndeam.mspx, 4 pages.
`10/2006 Parsonsetal.
`2006/0230340 Al
`Okadaat al., “CounterVision: A Screen Reader with Multi-Access
`1/2007 Cogliano
`2007/0011011 Al
`Interface for GUI,” Proceedings of Technology And Persons With
`2/2007 Negariet al.
`2007/0033543 Al
`Disabilities Conference, Center on Disabilities, CSU Northridge,
`10/2007 Foss
`2007/0230748 Al
`Mar. 1997,_http:/www.csun.edu/cod/conf/1997/proceedings/
`
`2007/0236475 Al
`10/2007 Wherry
`090,htm, 6 pages.
`2007/0262964 Al
`11/2007 Zotovetal.
`Raman,T,, “Eyes-Free User interaction,” Google Research, Feb. 9,
`2007/0268317 Al
`11/2007 Banay
`2009, http://emacspeak.sf.net/raman, 25 pages.
`2008/0027726 Al
`1/2008 Hansenetal.
`2008/0114566 Al
`5/2008 Slotznicket al.
`tiresias.org, “Touchscreens,” tiresiaeorg, Jul. 15, 2008, http://www.
`2008/0122796 Al
`5/2008 Jobsetal.
`tiresias.org/research/guidelines/touch/htm.
`2008/0140413 Al
`6/2008 Millman etal.
`Touch Usability, “Mobile,” Mar. 12, 2009, http://www.touchus-
`2008/0165141 Al
`7/2008 Christie
`ability.com/mobile/, 9 pages
`2008/0300874 Al
`12/2008 Gavaldaetal.
`Vanderheiden,G., “Use of audio-haptic interface techniquesto allow
`2008/0316183 Al
`12/2008 Westerman etal.
`nonvisual access to touchscreen appliances,” Sep., Oct. 1995, http://
`2009/0303187 Al
`12/2009 Pallakoff
`trace.wise.cdu/docs/touchserceen/chi_conf.htm, 9 pages.
`2009/0313020 Al
`12/2009 Koivunen
`U.S. Appl. No. 10/826,875, filed Apr. 16, 2004, 61 pages (Migos).
`2010/0001953 Al
`1/2010 Yamamotoetal.
`USS. Appl. No. 10/956,720, filed Oct. 1, 2004, 75 pages (Seymour).
`2010/0063880 Al
`3/2010 Atsmonetal.
`U.S. Appl. No. 11/298,977, filed Dec. 9, 2005, 33 pages (Seymour).
`2010/0070281 Al
`3/2010 Conkie etal.
`U.S. Appl. No. 11/643,257, filed Dec. 20, 2006, 44 pages (Seymour).
`2010/0231541 Al
`9/2010 Cruz-Hernandezetal.
`*
`U.S. Appl. No. 11/643,389, filed Dec. 20, 2006, 43 pages (Seymour).
`. 345/173
`2010/0283742 A1*
`11/2010 Lam ......
`*
`2010/0289757 A1*
`11/2010 Budelli on. 345/173
`U.S. Appl. No. 11/686,295, filed Mar. 14, 2007. 40 pages (Seymour).
`2010/0309148 Al
`12/2010 Fleizach etal.
`Office Action dated May 25, 2012, received in U.S. Appl. No.
`2010/0313125 Al
`12/2010 Fleizach etal.
`12/565,744, 16 pages (Fleizach).
`2010/0324903 Al
`12/2010 Kurzweilet al.
`Office Action dated Dec. 21, 2011, received in U.S. Appl. No.
`2011/0050594 Al
`3/2011 Kim et al.
`12/795,633, 9 pages (Fleizach).
`2011/0264452 Al
`10/2011 Venkataramuetal.
`Frantz et al., “Design case history: Speak & Spell learns to talk,”
`2011/0310026 Al
`12/2011 Davis et al.
`IEEE spectrum Feb. 1982, 5 pages.
`Law et al., “Ez Access Strategies for Cross-Disability Access to
`Kiosks,
`‘lelephones and VCRs,’ DINF (Disability Information
`Resources), Feb. 16, 1998. http://www.dinf.ne.jp/doc/english/Us_
`Eu/conf/csun_98/csun98__074 html, 6 pages.
`Vanderheiden, G., “Universal Design and. Assistive Technology in
`Communication and Information Technologies: Alternatives or
`Complements?” Assistive Technology: The Official Journal of
`RESNA,1998, vol. 10, No. 1, 9 pages.
`Vintage, “TSI Speech + & other speaking calculators,” Vintage Cal-
`culators Web Museum, retrieved from the internet May 4, 2012,
`http://www.vintagecalculators.com/html/speech_html, 6 pages.
`Extended Search Report dated Sept. 27, 2012, received in European
`Patent Application No. 12154609,7, which correspondsto U.S. Appl.
`No. 12/565,744, 7 pages (Fleizach).
`Final Office Action dated Dec. 6, 2012, received in U.S. Appl. No.
`12/565,744, 18 pages (Fleizach).
`Office Action dated Nov. 20, 2012, received in European Patent
`Application No. 10719502.6, which corresponds to U.S. Appl. No.
`12/565,744, 5 pages (Fleizach).
`
`European Search Report and Written Opinion dated Jun. 29, 2012,
`received in European Patent Application No. 12154613.9, which
`corresponds to U.S. Appl. No. 12/565,744, 6 pages(Fleizach).
`International Search Report and Written Opinion dated Aug. 30,
`2012, received in International Aopiication No. PCT/US2012/
`040703, which corresponds to U.S. Appl. No. 13/172,479: Ll pages
`(Fleizach).
`Office Action dated Aug. 30, 2012, received in U.S. Appl. No.
`12/795,633, 13 pages (Fleizach).
`American Thermoform Corp., “Touch Screen, Talking Tactile Tab-
`let,’ downloaded Jul. 30, 2008, http://www.americanthermoform.
`com/tactiletablet-htm, 2 pages.
`Apple.com, “VoiceOver,” May 2009, http:/Avwww.apple.com/acces-
`sibility/vocieover, 5 pages.
`Apple Inc., “iPad User Guide,” Apple Inc., © 2010, 154 pages.
`appshopper, “GDial Free—Speed Dial with Gesture.” appshopper.
`com, Mar. 25, 2009, http://appshopper.com/utilities/gdial-free-
`speed-dial-with-gesture, 2 pages.
`
`OTHER PUBLICATIONS
`
`2
`
`

`

`U.S. Patent
`
`Jul. 23, 2013
`
`Sheet 1 of 84
`
`US 8,493,344 B2
`
`Portable Multifunction Device 100
`
`Memory
`102
`136
`
`Applications (continued)
`126
`:
`147
`Operating System
`Browsing Module
`
`
`
`
`
`128
`Communication Module
`148
`Calendar Module
`
`129
`Accessibility Module
`149
`
`
`130
`Contact/Motion Module
`Widget Modules
`149-1
`
`
`
`
`
`
`132
`Graphics Module
`Weather Widget
`149-2
`a
`Attachment Editing Module
`Stocks Widget
`149-3
`
`
`
`
`135
`Text Input Module
`Calculator Widget
`449-4
`136
`GPS Module
`Alarm Clock Widget
`4149-5
`
`
`
`Dictionary Widget
`
`
`Contacts Module re}|LET tag
`
`
`
`
`
`Telephone Module
`User-Created Widget(s)
`139
`450
`
`
`
`Widget Creator Module
`Video Conference Module
`4140
`154
`
`
`
`Search Module
`E-mail Client Module
`4141
`
`;
`Instant Messaging Module
`4142
`>
`162
`Voice Memo Module
`143
`ower
`Camera Module
`144
`System
`Image Management Module
`145
`Video Player Module
`-
`146
`Music Player Module
`
`
`
`
`
`
`
`
`
`
`
`
`137
`
`
`
`
`
`
`124
`
`118
`
`RF Circuitry
`
`External
`Port
`or
`103
`ek
`5 103 Gio |
`
`or
`Circuitry
`110 KI)
`Microphone
`113
`166
`
`404
`
`122
`
`Controller
`
`420
`
`Processor(s)
`
`106
`
`103
`
`103
`
`Sensor
`
`
`|
`
`Peripherals
`Interface
`
`
`| 16
`103
`168
`
`I/O Subsystem
`158
`Optical
`
`
`
`Display
`sensor(s)
`Other Input
`
`
`
`
`
`
`Controller
`Controller(s)
`Controller
`
`
`
`
`
`
`156—"|
`
`103
`
`103
`
`103
`
`Touch-Sensitive
`Display System
`
`112
`
`Sensors)
`
`Oferimpute
`
`ome 164
`Devices
`
`Figure 1A
`
`111
`
`160
`
`116
`
`3
`
`

`

`U.S. Patent
`
`Jul. 23, 2013
`
`Sheet 2 of 84
`
`US 8,493,344 B2
`
`Portable Multifunction Device 100
`
`tsa
`126
`147
`128
`143
`129
`149
`130
`
`132|[wi 149-4
`133
`4149-2
`a
`149.3
`
`|Applications (continued)
`
`icati
`
`ie
`
`ine
`
` Memory 102
`
`140-4
`149.5
`149-6
`139
`
`sap|(--WserCreatedWidget(s)"16,
`141
`151
`155
`
`143
`
`Et
`
`a
`1
`452
`Image Management Module
`we
`veton
`Vi
`d Music PI
`Modul
`
`ideo and 124|SystemMusic Player Module 453 External
`
`
`
`
`Notes Module
`103
`118
`1
`103
`RF Circuitry
`108
`VY 103
`5
`Control
`Audio ||
`
`ontrolier
`Circuitry
`110 +)
`
`
`Peripherals
`Microphone
`
`Interface
`113
`
`166
`
`Processor(s)
`
`-——™
`Accelerometer(s)
`103
`68
`158
`
`/O Subsystem
`
`
`
`Other Input
`Display
`Optical
`
`
`
`sensor(s)
`Controller
`Controller(s)
`i
`Controller
`
`
`
`103
`103
`
`104
`
`122
`
`120
`
`106
`
`156
`
`
`
`403
`
`103
`
`Proximity
`Sensor
`
`1
`
`Speaker
`111
`
`160
`
`Display System
`
`Touch-Sensitive
`
`112
`
`OtherInput
`Control
`Devices
`
`116
`
`Optical
`Sensor(s)
`164
`
`Figure 1B
`
`103
`
`4
`
`

`

`U.S. Patent
`
`Jul. 23, 2013
`
`Sheet 3 of 84
`
`US 8,493,344 B2
`
`Portable Multifunction Device 100
`
`212
`
`Speaker 111
`
`Optical
`Sensor 164
`
`Proximity
`Sensor 166
`
`206
`
`200
`
`208
`
`208
`
`210 is SIM card slot
`212 is headphone jack
`
`Touch Screen 11
`
`Microphone
`113
`
`Home
`204
`
`Accelerometer(s)
`168
`
`External Port 124
`Figure 2
`
`5
`
`

`

`U.S. Patent
`
`Jul. 23, 2013
`
`Sheet 4 of 84
`
`US 8,493,344 B2
`
`.
`
`Device 300
`\
`
`310
`
`CPU(s)
`
`320
`
`>
`
`330
`
`/O interface
`
`Keyboard/Mouse
`
`Touchpad
`
`350
`
`355
`
`149-6
`Sa
`360|Communications User-Created Widget(s) 150
`
`
`Interface
`Widget Creator Module
`154
`ae ——
`380
`rawing
`odule
`382
`Presentation Module
`384
`Word Processing Module
`Website Creation Module
`Disk Authoring Module
`
`Memory 370
`Operating System
`Communication Module
`Accessibility Module
`Contact/Motion Module
`Graphics Module
`AttachmentEditing Module
`Text Input Module
`
`Contacts Module
`Telephone Module
`Video Conference Module
`E-mail Client Module
`Instant Messaging Module
`Voice Memo Module
`Camera Module
`Image Management Module
`Video Player Module
`Music Player Module
`Browsing Module
`Calendar Module
`
`tocks Widget
`Calculator Widget
`larm Clock Widget
`Dictionary Widget
`
`126
`128
`129
`tp
`433
`
`134
`136
`
`137
`
`138
`139
`440
`144
`442
`143
`144
`145
`446
`447
`148
`449
`
`492
`
`149-3
`149-4
`149-5
`
`386
`388
`390
`
`
`
`
`
`
`
`
`
`Spreadsheet Module Figure 3
`
`6
`
`

`

`AeIoae
`
`Portable Multifunction Device 402 E
`
`Text
`141
`
`= 7
`
`5°
`
`Weather
`149-1
`
`Stocks
`149-2
`
`Calculator
`149-3
`
`Browser
`147
`
`Touch Screen 11
`
`Microphone
`113
`
`Home
`204
`
`Accelerometer(s)
`168
`
`Figure 4A
`
`U.S. Patent
`
`Jul. 23, 2013
`
`Sheet 5 of 84
`
`US 8,493,344 B2
`
`Current Time 404
`
`Photos
`144
`
`Camera
`143
`
`Videos
`4145
`
`J
`
`Calendar
`148
`
`
`
`Voice Memo
`142
`
`ABC
`
`Dictionary
`149-5
`
`7
`
`

`

`U.S. Patent
`
`Jul. 23, 2013
`
`Sheet 6 of 84
`
`US 8,493,344 B2
`
`Portable Multifunction Device
`
`> p> ap 406
`405
`
`Jan
`
`Calendar
`148
`
`Photos
`144
`
`Camera
`143
`
`peq) (CS)
`
`100 Current Time 404
`
`fe
`
`Text
`141
`
`A
`
`Online
`Video
`155
`
`lePY
`
`ty
`
`75°
`Weather
`149-1
`
`Stocks
`149-2
`
`Calculator
`149-3
`
`Notes
`153
`
`Settings
`412
`
`Browser
`447
`
`Touch Screen 11
`
`113
`
`Home
`204
`
`Accelerometer(s)
`168
`
`Figure 4B
`
`8
`
`

`

`U.S. Patent
`
`Jul. 23, 2013
`
`Sheet 7 of 84
`
`US 8,493,344 B2
`
`206
`
`Portable Electronic Device
`100
`
`Wallpaper image
`470
`
`Current Time ~» 465
`466 ~ Day and Date~ 467
`
`
`
`|Slidetounlockto unlock
`
`Touch Screen 112
`
`Microphone
`113
`
`Home
`204
`
`Accelerometer(s)
`168
`
`Figure 4C
`
`9
`
`

`

`U.S. Patent
`
`Jul. 23, 2013
`
`Sheet 8 of 84
`
`US 8,493,344 B2
`
`Portable Multifunction Device
`
`502
`
`‘2
`
`Current Time 404
`
`(C35)
`
`Photos
`144
`
`Camera
`143
`
`Videos
`145
`
`Jan
`15
`
`es
`i
`
`Voice Memo
`
`Calendar
`
`ABC
`
`User-
`Created
`Widget
`
`100 Zi 402 E
`
`Text
`441
`
`Weather
`149-1
`
`a
`
`
`
`504-1
`Sy
`+ °4-2 stocks
`:
`
`148 o
`
`Calculator
`149-3
`
`Alarm
`149-4
`
`Dictionary
`149-5
`
`Browser
`147
`
`Touch Screen 11
`
`Microphone
`113
`
`Home
`204
`
`Accelerometer(s)
`168
`
`Figure 5A
`
`10
`
`10
`
`

`

`U.S. Patent
`
`Jul. 23, 2013
`
`Sheet 9 of 84
`
`
`
`100
`
`Photos
`144
`
`Camera
`143
`
`(ee
`oe
`
`Videos
`145
`
`Jan
`15
`
`Weather
`149-1
`
`Stocks
`149-2
`
`Voice Memo
`142
`
`Calendar
`148
`
`Calculator
`149-3
`
`User-
`Created
`
`ABC
`
`Dictionary
`149-5
`
`Browser
`147
`
`Touch Screen 112
`
`US 8,493,344 B2 Portable Multifunction Device
`
`Widget
`
`Microphone
`113
`
`Home
`204
`
`Accelerometer(s)
`168
`
`Figure 5B
`
`11
`
`11
`
`

`

`U.S. Patent
`
`Jul. 23, 2013
`
`Sheet 10 of 84
`
`US 8,493,344 B2
`
`Portable Multifunction Device
`
`144
`
`Camera
`143
`
`Videos
`145
`
`Weather
`149-1
`
`Stocks
`149-2
`
`Voice Memo
`142
`
`Calendar
`148
`
`100 Photos
`
`User-
`
`Created Widget
`
`ABC
`
`Dictionary
`
`Caiculator
`
`506-1-U
`
`Touch Screen 11
`
`Microphone
`113
`
`Home
`204
`
`Accelerometer(s)
`168
`
`Figure 5C
`
`12
`
`12
`
`

`

`Text
`141
`
`
`
`Weather
`149-1
`
`508-1 “~ _
`
`Photos
`144
`
`Camera
`143
`
`S
`oo
`
`.
`
`
`. 508-2 Ss°
`:
`
`tocks
`
`Voice Memo
`
`502
`
`[|
`
`Videos
`145
`
`Jan
`15
`
`Calendar
`148
`
`
`
`
`
`
`] (abe||
`
`
`
`
`
`U.S. Patent
`
`Jul. 23, 2013
`
`Sheet 11 of 84
`
`US 8,493,344 B2
`
`Portable Multifunction Device
`100
`
`Speaker 111
`
`Optical
`Sensor 164
`
`Proximity
`Sensor 166
`
`UI 500D
`§
`
`Z\ 402 E
`
`Current Time 404
`
`a> 406
`
`%
`S|
`
`™
`
`Dictionary
`Alarm
`Calculator
`149-5
`149-4
`149-3
`
`
`Widget
`149-6
`
`Browser
`147
`
`Touch Screen 11
`
`
`
`Microphone
`113
`
`Home
`204
`
`Accelerometer(s)
`168
`
`Figure 5D
`
`13
`
`13
`
`

`

`U.S. Patent
`
`Jul. 23, 2013
`
`Sheet12 of 84
`
`US 8,493,344 B2
`
`Portable Multifunction Device
`
`
`
`“Weather Stocks
`149-1-- +
`149-2
`
`Voice Memo
`142
`
`100 Jan
`
`15
`
`Calendar
`148
`
`Calculator
`149-3
`
`ABC
`
`Dictionary
`149-5
`
`Browser
`147
`
`Touch Screen 112
`
`Microphone
`113
`
`Home
`204
`
`Accelerometer(s)
`168
`
`Figure 5E
`
`14
`
`14
`
`

`

`U.S. Patent
`
`Jul. 23, 2013
`
`Sheet 13 of 84
`
`US 8,493,344 B2
`
`Portable Multifunction Device
`100
`
`Speaker 111
`
`Optical
`Sensor 164
`
`Proximity
`Sensor 166
`
`206
`
`Ul 500F
`
`208
`
`208
`
`
`
`Photos
`
`Camera
`
`Jan
`15
`
`Voice Memo
`142
`
`Calendar
`148
`
`ABC
`
`User-
`
`Created Widget
`
`Calculator
`149-3
`
`Alarm
`149-4 gp.
`
`Dictionary
`14955
`
`Browser
`147
`
`Touch Screen 112
`
`Microphone
`4143
`
`Home
`204
`
`Accelerometer(s)
`168
`
`Figure 5F
`
`15
`
`15
`
`

`

`U.S. Patent
`
`Jul. 23, 2013
`
`Sheet 14 of 84
`
`US 8,493,344 B2
`
`206]
`206
`
`Portable Multifunction Device
`100
`
`Speaker 111
`
`Optical
`Sensor 164
`
`Proximity
`Sensor 166
`
`Ul 500G
`
`ae
`
`Photos
`144
`
`Camera
`143
`
`a x1
`
`
`
`Calculator
`149-3
`
`Dictionary
`149-5
`
`Created
`Widget
`
`Browser
`147
`
`4
`
`Touch Screen 11
`
`Microphone
`413
`
`Home
`204
`
`Accelerometer(s)
`168
`
`Figure 5G
`
`16
`
`16
`
`

`

`U.S. Patent
`
`Jul. 23, 2013
`
`Sheet 15 of 84
`
`US 8,493,344 B2
`
`Portable Multifunction Device
`
`100 [|_|
`
`Photos
`144
`
`Camera
`143
`
`GE
`w
`
`Jan
`15
`
`Stocks
`149-2
`
`Voice Memo
`142
`
`Calendar
`148
`
`User-
`Created
`Widget
`
`ABC
`
`Dictionary
`149-5
`
`75°
`
`Weather
`149-1
`
`Calculator
`149-3
`
`Browser
`147
`
`_
`
`4
`
`Touch Screen 11
`
`Microphone
`113
`
`Home
`204
`
`Accelerometer(s)
`168
`
`Figure 5H
`
`17
`
`17
`
`

`

`U.S. Patent
`
`Jul. 23, 2013
`
`Sheet 16 of 84
`
`US 8,493,344 B2
`
`[206]
`
`Portable Multifunction Device
`100
`
`Current Time 404
`
`ab
`
`© eoea
`
`Text
`141
`
`Photos
`144
`
`Camera
`143
`
`Videos
`4145
`
`Weather
`149-1
`
`'
`
`Calculator
`149-3
`
`Dictionary
`149-5
`
`alendar
`148
`
`User-
`Created
`Widget
`
`
`
`Browser
`147
`
`Touch Screen 112
`
`Microphone
`113
`
`Home
`204
`
`Accelerometer(s)
`168
`
`Figure 5l
`
`18
`
`18
`
`

`

`U.S. Patent
`
`Jul. 23, 2013
`
`Sheet 17 of 84
`
`US 8,493,344 B2
`
`Portable Multifunction Device
`
`412
`
`100 Settings
`
`Browser
`147
`
`Touch Screen 112
`
`113
`
`Home
`204
`
`Accelerometer(s)
`168
`
`Figure 5J
`
`19
`
`19
`
`

`

`U.S. Patent
`
`Jul. 23, 2013
`
`Sheet 18 of 84
`
`US 8,493,344 B2
`
`206
`
`=
`S|
`
`UI 500K
`
`Speaker 111
`
`Optical
`Sensor 164
`
`Proximity
`Sensor 166
`
`Thursday February 26,
`2009
`
` Touch Screen 112
`
`
`
`Accelerometer(s)
`168
`
`Portable Multifunction Device 100
`
`
`
`
`Microphone
`113
`
`Home
`204
`
`Figure 5K
`
`20
`
`20
`
`

`

`U.S. Patent
`
`Jul. 23, 2013
`
`Sheet 19 of 84
`
`US 8,493,344 B2
`
`Portable Multifunction Device 100
`
`Speaker 111
`
`Optical
`Sensor 164
`
`Proximity
`Sensor 166
`
`UI 500L
`
`
`
`Password
`
`© S
`
`elect Language
`
`Thursday February 26,
`2009
`
`Touch Screen 112
`Microphone
`113
`
`Home
`204
`
`Accelerometer(s)
`168
`
`Figure 5L
`
`21
`
`21
`
`

`

`U.S. Patent
`
`Jul. 23, 2013
`
`Sheet 20 of 84
`
`US 8,493,344 B2
`
`Portable Multifunction Device 100
`
`Speaker 114
`
`Optical
`Sensor 164
`
`Proximity
`Sensor 166
`
`UI 500M Password
`
`Select Language
`
`Change Language ©
`Thursday February 26,
`2009
`
`Touch Screen 112:
`Microphone
`113
`
`Home
`204
`
`Accelerometer(s)
`168
`
`Figure 5M
`
`22
`
`22
`
`

`

`U.S. Patent
`
`Jul. 23, 2013
`
`Sheet 21 of 84
`
`US 8,493,344 B2
`
`Portable Multifunction Device 100
`
`Speaker 114
`
`Optical
`Sensor 164
`
`Proximity
`
`Thursday February 26,
`2009
`
`Sensor 166 UI S500N
`
`Touch Screen 112
`Microphone
`113
`
`Home
`204
`
`Accelerometer(s)
`168
`
`Figure 5N
`
`23
`
`23
`
`

`

`U.S. Patent
`
`Jul. 23, 2013
`
`Sheet22 of 84
`
`US 8,493,344 B2
`
`Portable Multifunction Device 100
`
`Speaker 111
`
`Optical
`Sensor 164
`
`Proximity
`Sensor 166
`
`Ul 5000
`
`
`
`User Name
`
`client
`
`Password ~~~ °49
`
`Select Language
`
`Change Language ©
`Thursday February 26,
`2009
`
`Touch Screen 112
`Microphone
`113
`
`Home
`204
`
`Accelerometer(s)
`168
`
`Figure 50
`
`24
`
`24
`
`

`

`U.S. Patent
`
`Jul. 23, 2013
`
`Sheet 23 of 84
`
`US 8,493,344 B2
`
`Portable Multifunction Device
`
`=
`
`Feb 26
`
`10:22
`
`If it were done whe...
`
`
`100 15:00
`
`
`
`
`
`Touch Screen 112
`Microphone
`113
`
`204
`
`Accelerometer(s)
`168
`
`Figure 5P
`
`25
`
`25
`
`

`

`U.S. Patent
`
`Jul. 23, 2013
`
`Sheet 24 of 84
`
`US 8,493,344 B2
`
`206
`
`Portable Multifunction Device
`100
`
`15:00
`
`=
`
`If it were done whe...
`
`Feb 26
`
`10:22
`
`
`
`mi
`
`GC Gr
`
`[Weha
`
`ime
`
`Li me| ‘
`
`ie
`
`:
`
`Of
`
`GO ROTOn vor
`
`
`
`
`ive
`
`a OPK) Drone:
`
`.
`
`Touch Screen 112
`Microphone
`113
`
`204
`
`Accelerometer(s)
`168
`
`Figure 5Q
`
`26
`
`26
`
`€
`

`

`
`
`
`{lus far, so nobly
`hich they have,
`
`‘Carri on. Tt is rather forusto be Here
`
`ich they gave here the last full
`
`Touch Screen 112
`
`Microphone
`113
`
`204
`
`Accelerometer(s)
`168
`
`Figure 5R
`
`27
`
`U.S. Patent
`
`Jul. 23, 2013
`
`Sheet 25 of 84
`
`US 8,493,344 B2
`
`Portable Multifunction Device
`
`iPod &
`
`15:01
`
`=
`
`If it were done whe...
`
`
`
`
`Yesterday 25/4=5575 Feb 26 10:22
`
`
`
`
`
`
`
`Tt is forms, the
`a
`
`|
`
`2 be
`
`ot 8
`
`° w
`
`27
`
`

`

`U.S. Patent
`
`Jul. 23, 2013
`
`Sheet 26 of 84
`
`US 8,493,344 B2
`
`206
`
`Portable Multifunction Device
`100
`
`
`
`I Yesterday
`
`Feb 26
`
`10:22
`
`~~ simi ses se skim ani ah hth kik nnn 4
`dedicated to the great task before us -
`that from these honored dead we take
`
`
`
`
`
`
`
`
`
`new‘birth of fresidom, and that
`overnment of the people, by the
`°
`0 Hle
`i
`
`"ArT
`
`
`
`
`
`Touch Screen 112
`
`Microphone
`113
`
`204
`
`Accelerometer(s)
`168
`
`Figure 5S
`
`28
`
`28
`
`

`

`U.S. Patent
`
`Jul. 23, 2013
`
`Sheet 27 of 84
`
`US 8,493,344 B2
`
`Portable Multifunction Device
`
`iPod ©
`
`15:01
`
`If it were done whe...
`
`
`Feb 26
`
`10:22
`
`
`
`
`
`
`
`
`
`
`
`
`corvied on. It is rather for us to be here
`dedicated to the great task before us -
`that from these honored dead we take
`which they gave here the last ful
`
`measure of devotion - that we here
`
`Touch Screen 112
`Microphone
`413
`
`204
`
`Accelerometer(s)
`168
`
`Figure 5T
`
`29
`
`29
`
`

`

`U.S. Patent
`
`Jul. 23, 2013
`
`Sheet 28 of 84
`
`US 8,493,344 B2
`
`Portable Multifunction Device
`
`100 Touch Screen 112
`
`Microphone
`113
`
`204
`
`Accelerometer(s)
`168
`
`Figure 5U
`
`30
`
`30
`
`

`

`U.S. Patent
`
`Jul. 23, 2013
`
`Sheet 29 of 84
`
`US 8,493,344 B2
`
`Portable Multifunction Device
`
`
`
`
`
`100 Select Language
`
`(crangetanaie5e)g
`
`Thursday February 26,
`2009
`
`Touch Screen 112
`Microphone
`4113
`
`Home
`204
`
`Accelerometer(s)
`168
`
`Figure 5V
`
`31
`
`31
`
`

`

`U.S. Patent
`
`Jul. 23, 2013
`
`Sheet 30 of 84
`
`US 8,493,344 B2
`
`Portable Multifunction Device
`
`client
`
`fii
`
`Password
`
`a)
`
`Select Language
`
`()
`
`Change Language ©
`Thursday February 26,
`2009
`
`100 User Name
`
`Touch Screen 112
`Microphone
`1413
`
`Home
`204
`
`Accelerometer(s)
`168
`
`Figure 5W
`
`32
`
`32
`
`

`

`U.S. Patent
`
`Jul. 23, 2013
`
`Sheet 31 of 84
`
`US 8,493,344 B2
`
`Ca)
`
`Password
`
`Thursday Fehruary 26,
`2009
`
`Portable Multifunction Device User Name
`
`Touch Screen 112
`Microphone
`113
`
`Home
`204
`
`Accelerometer(s)
`168
`
`Figure 5X
`
`33
`
`33
`
`

`

`U.S. Patent
`
`Jul. 23, 2013
`
`Sheet 32 of 84
`
`US 8,493,344 B2
`
`Portable Multifunction Device
`100
`
`Speaker 111
`
`Optical
`Sensor 164
`
`Proximity
`
`Sensor 166 User Name
`
`client (i
`
`Password
`
`Select Language
`
`-
`
`Change Language ©
`Thursday February 26,
`2009
`
`Touch Screen 112
`Microphone
`113
`
`Home
`204
`
`Accelerometer(s)
`168
`
`Figure 5Y
`
`34
`
`34
`
`

`

`
`
`Password
`
`© S
`
`elect Language
`
`Change Language ©
`Thursday February 26,
`2009
`
`Touch Screen 112
`Microphone
`413
`
`Home
`204
`
`Accelerometer(s)
`168
`
`Figure 5Z
`
`35
`
`U.S. Patent
`
`Jul. 23, 2013
`
`Sheet 33 of 84
`
`US 8,493,344 B2
`
`206
`
`Portable Multifunction Device
`100
`
`Speaker 111
`
`Optical
`Sensor 164
`
`Proximity
`Sensor 166
`
`UI 500Z
`
`(ci—__)
`
`User Name
`
`35
`
`

`

`U.S. Patent
`
`Jul. 23, 2013
`
`Sheet 34 of 84
`
`US 8,493,344 B2
`
`Portable Multifunction Device
`100
`
`__
`
`Optical
`164
`
`Ss
`
`Proximity
`
`1 Touch Screen 112
`
`Microphone
`113
`
`Home
`204
`
`Accelerometer(s)
`168
`
`Figure 54A
`
`36
`
`36
`
`

`

`U.S. Patent
`
`Jul. 23, 2013
`
`Sheet 35 of 84
`
`US 8,493,344 B2
`
`Portable Multifunction Device
`
`(#}
`
`Cm) ©
`
`Select Language
`
`Change Language ©
`
`100 16:30
`
`English
`
`Espanol
`
`Touch Screen 112
`Microphone
`113
`
`Home
`204
`
`Accelerometer(s)
`168
`
`Figure 5BB
`
`37
`
`37
`
`

`

`U.S. Patent
`
`Jul. 23, 2013
`
`Sheet 36 of 84
`
`US 8,493,344 B2
`
`Portable Multifunction Device
`
`Speaker 111
`
`Optical
`Sensor 164
`
`Proximity
`Sensor 166
`
`UI S00CC Morgan Lewis Client Internet Service
`
`12.162.29.66/login.p| @ Cy
`
`Login pour Utilisateurs
`Utilisateur
`
`client [li
`
`Mot de passe
`
`en) g
`
`Choisissez la Langue
`
`Touch Screen 112
`Microphone
`413
`
`Home
`204
`
`Accelerometer(s)
`168
`
`Figure 5CC
`
`38
`
`38
`
`

`

`U.S. Patent
`
`Jul. 23, 2013
`
`Sheet 37 of 84
`
`US 8,493,344 B2
`
`206
`
`Portable Multifunction Device
`100
`
`
`
`Morgan Lewis Client Internet Service
`12.162.29.66/login.pl
`ad
`
`Login pour Utilisateurs
`Utilisateur
`
`Touch Screen 112
`Microphone
`113
`
`Home
`204
`
`Accelerometer(s)
`168
`
`Figure 5DD
`
`39
`
`39
`
`

`

`U.S. Patent
`
`Jul. 23, 2013
`
`Sheet 38 of 84
`
`US 8,493,344 B2
`
`Portable Multifunction Device
`100
`
`Speaker 111
`
`ticOptical
`Sensor 164
`
`Proximity
`Sensor 166
`
`Ul SOOEE
`
`iPod =
`
`16:30
`
`Can ©
`
`Select Language Change Language ©
`
`English
`
`Espanol
`
`v Frangais
`
`Touch Screen 112
`Microphone
`113
`
`Home
`204
`
`Accelerometer(s)
`168
`
`Figure 5EE
`
`40
`
`40
`
`

`

`U.S. Patent
`
`Jul. 23, 2013
`
`Sheet 39 of 84
`
`US 8,493,344 B2
`
`Portable Multifunction Device
`100
`
`Speaker 111
`
`UI 500FF
`
`Optical
`Sensor 164
`,
`Current Time 404
`
`Proximity
`sensor 166
`
`@p 406
`
`"
`
`;
`
`(ee*
`
`Photos
`
`[|
`
`Videos
`145
`
`Camera
`143
`
`eI
`
`,
`
`578-1
`
`
`
`Weather
`149-1
`
`149-2
`
`Voice Memo
`142
`
`Calendar
`148
`
`User-
`
`Created Widget
`
`Calculator
`149-3
`
`Dictionary
`149-5
`
`Browser
`147
`
`Touch Screen 11
`
`Microphone
`113
`
`Home
`204
`
`Accelerometer(s)
`168
`
`Figure 5FF
`
`41
`
`41
`
`

`

`U.S. Patent
`
`Jul. 23, 2013
`
`Sheet 40 of 84
`
`US 8,493,344 B2
`
`Portable Multifunction Device
`
`144
`
`Camera
`143.
`
`Videos
`145
`
`Weather
`-
`
`149-1
`
`Stocks
`-
`
`149-2
`
`Voice Memo
`
`Calendar
`148
`
`142 578-3
`
`100 Photos
`
`User-
`ABC|4| Created
`
`
`
`Calculator
`149-3
`
` Widget
`
`- Dictionary- ,
`*., 149-5.”
`
`Browser
`147
`
`408
`
`Touch Screen 14
`
`Microphone
`113
`
`Home
`204
`
`Acceleromeier(s)
`168
`
`Figure 5GG
`
`42
`
`42
`
`

`

`U.S. Patent
`
`Jul. 23, 2013
`
`Sheet 41 of 84
`
`US 8,493,344 B2
`
`206
`
`UI SOOHH
`
`Portable Multifunction Device
`
`Speaker 111
`
`Optical
`Sensor 164
`
`Proximity
`Sensor 166
`
`wall AT&T
`
`KAccessibility
`
`5:39 PM
`
`Zoom
`
`1
`
`
`
`Zoom magnifies the entire screen.
`
`To zoom
`Double-tap with three fingers.
`
`To move around the screen
`Drag three fingers while zoomed.
`
`To change zoom
`Double-tap and drag three fingers
`up or down.
`
`Touch Screen 112
`Microphone
`113
`
`Home
`204
`
`Accelerometer(s)
`168
`
`Figure 5HH
`
`43
`
`43
`
`

`

`U.S. Patent
`
`Jul. 23, 2013
`
`Sheet 42 of 84
`
`US 8,493,344 B2
`
`Portable Multifunction Device
`
`aae
`
`Photos
`144
`
`Camera
`143
`
`Videos
`145
`
`Weather
`149-1
`
`Stocks
`149-2
`
`Voice Memo
`142
`
`Calendar
`148
`
`ABC
`
`oo "+. Dictionary
`
`Calculator
`149-30.
`
`580-1 —~~
`
`100 @p 406
`
`Browser
`147
`
`Touch Screen 11
`
`Microphone
`113
`
`Home
`204
`
`Accelerometer(s)
`168
`
`Figure5il
`
`44
`
`44
`
`

`

`U.S. Patent
`
`Jul. 23, 2013
`
`Sheet 43 of 84
`
`US 8,493,344 B2
`
`Portable Multifunction Device
`
`Current Time 404
`
`100 Z1\ 402 E
`
`Dictionary
`149-5
`
`Touch Screen 112
`
`Microphone
`113
`
`Home
`204
`
`Accelerometer(s)
`168
`
`Figure 5JJ
`
`45
`
`45
`
`

`

`U.S. Patent
`
`Jul. 23, 2013
`
`Sheet 44 of 84
`
`US 8,493,344 B2
`
`Z1 402 E
`
`Current Time 404
`
`Photos
`144
`
`143
`
`Portable Multifunction Device
`
`100
`
`113 Camera
`
`584-5
`
`Stocks
`149-2"
`
`Voice Memo
`142
`
`“584-3
`
`Touch Screen 11
`
`Microphone
`
`Accelerometer(s)
`168
`
`46
`
`

`

`U.S. Patent
`
`Jul. 23, 2013
`
`Sheet 45 of 84
`
`US 8,493,344 B2
`
`Portable Multifunction Device
`
`Current Time 404
`
`100 Z] 402 £
`
`586-6
`
`_
`
`586-2
`
`Voice’Memo
`_ 142
`
`*.7 586-3
`
`. Touch Screen 112
`
`Microphone
`4113
`
`Home
`204
`
`Accelerometer(s)
`168
`
`Figure 5LL
`
`47
`
`47
`
`

`

`U.S. Patent
`
`Jul. 23, 2013
`
`Sheet 46 of 84
`
`US 8,493,344 B2
`
`Portable Multifunction Device
`
`Current Time 404
`
`100 Z1 402 E
`
`Touch Screen 11
`
`Microphone
`113
`
`Home
`204
`
`Accelerometer(s)
`168
`
`Figure 5MM
`
`48
`
`48
`
`

`

`U.S. Patent
`
`Jul. 23, 2013
`
`Sheet 47 of 84
`
`US 8,493,344 B2
`
`Portable Multifunction Device
`
`144
`
`Camera
`143
`
`Bey
`w
`
`Videos
`145
`
`Jan
`15
`
`Voice Memo
`442
`
`Calendar
`148
`
`ABC
`
`Dictionary
`149-5
`
`75°
`
`Weather
`149-1
`
`Calculator
`149-3
`
`100 Photos
`
`Browser
`147
`
`Touch Screen 11
`
`Microphone
`113
`
`Home
`204
`
`Accelerometer(s)
`168
`
`Figure 5NN
`
`49
`
`49
`
`

`

`U.S. Patent
`
`Jul. 23, 2013
`
`Sheet 48 of 84
`
`US 8,493,344 B2
`
`206
`
`Portable Multifunction Device
`100
`
`591-1a
`
`
`
`Touch Screen 112
`Microphone
`113
`
`Home
`204
`
`Accelerometer(s)
`168
`
`Figure 500
`
`50
`
`50
`
`

`

`U.S. Patent
`
`Jul. 23, 2013
`
`Sheet 49 of 84
`
`US 8,493,344 B2
`
`Portable Multifunction Device
`
`100 Touch Screen 112
`
`Microphone
`113
`
`Home
`204
`
`Accelerometer(s)
`168
`
`Figure 5PP
`
`51
`
`51
`
`

`

`U.S. Patent
`
`Jul. 23, 2013
`
`Sheet 50 of 84
`
`US 8,493,344 B2
`
`[206]
`206
`
`Portable Multifunction Device
`100
`
`Proximity
`Optical
`Sensor 166
`Sensor 164
`Speaker 114
`/
`z5
`hn
`7 ’
`.
`6
`j,
`4
`J
`2
`;
`c
`‘
`/
`( £1402E / Current Time 404
`J
`J
`a
`f
`f
`y/
`y é
`i
`i
`vA
`/
`J‘
`J J
`4
`J
`c
`a
`A
`FA
`Zc
`S| sms,—nbs sn a,—ts
`
`7
`
`°
`
`
`
`QN
`
`4
`
`/
`
`——T
`
`599
`
`Camera
`143
`
`Voice Memo
`
`Microphone
`113
`
`Home
`204
`
`Accelerometer(s)
`168
`
`Figure 5QQ
`
`52
`
`UI 500QQ
`
`-
`
`-
`
`-
`
`co
`ja)
`
`S|
`
`142 Touch Screen 112
`
`52
`
`

`

`U.S. Patent
`
`Jul. 23, 2013
`
`Sheet 51 of 84
`
`US 8,493,344 B2
`
`603
`
`610
`
`608
`
`Figure 6A
`
`603
`
`53
`
`53
`
`

`

`U.S. Patent
`
`Jul. 23, 2013
`
`Sheet 52 of 84
`
`US 8,493,344 B2
`
`UI600B
`
`614-1
`
`x 612
`
`on only...
`
`OOO.WindowPO
`
`Apple leads the way in customer
`satisfaction
`April 17, 2009
`When Forrester Research released
`
`the results of its annual survey on
`customersatisfaction, says Jim
`Dalrymple (macworld.com), "Apple
`
`616-4
`
`Figure 6B
`
`54
`
`54
`
`

`

`U.S. Patent
`
`Jul. 23, 2013
`
`Sheet 53 of 84
`
`US 8,493,344 B2
`
`614-2
`
`Round Rect Button
`
`Textured Button
`
`
`
`
`satisfaction
`April 17, 2009
`When Forrester Research released
`the results of its annual survey on
`customersatisfaction, says Jim
`Dalrymple (macworld.com), "Apple
`on only...
`
`Apple leads the way in customer
`satisfaction
`April 17, 2009
`WhenForrester Research released
`the results of its annual survey on
`customersatisfaction, says Jim
`Dalrymple (macworld.com), "Apple
`on only...
`
`Figure 6C
`
`55
`
`

`

`U.S. Patent
`
`Jul. 23, 2013
`
`Sheet 54 of 84
`
`US 8,493,344 B2
`
`Round Rect Button
`
`Textured Button
`
`Apple leads the way in customer
`satisfaction
`April 17, 2009
`WhenForrester Research released
`the results of its annual survey on
`customersatisfaction, says Jim
`Dalrymple (macworld.com}, "Apple
`
`614-3
`
`on only...
`603
`
`Apple leads the way in customer
`satisfaction
`Ail 17, 2009
`When Forrester Research released
`the fesults of its annual survey on
`custolyer satisfaction, says Jim
`Dalrymple (macworld.com), "Apple
`on only...
`
`Figure 6D
`
`56
`
`56
`
`

`

`U.S. Patent
`
`Jul. 23, 2013
`
`Sheet 55 of 84
`
`US 8,493,344 B2
`
`Window
`
`Apple leads the way in customer
`satisfaction
`April 17, 2009
`WhenForrester Research released
`the results of its annual survey on
`customersatisfaction, says Jim
`Dalrymple (macworld.com), “Apple
`
`Round Rect Button
`
`Textured Button
`
`on only...
`
`Figure 6E
`
`603
`
`o7
`
`57
`
`

`

`U.S. Patent
`
`Jul. 23, 2013
`
`Sheet 56 of 84
`
`US 8,493,344 B2
`
`Round Rect Button
`
`Textured Button
`
`on only...
`
`satisfaction
`April 17, 2009
`WhenForrester Research released
`the results of its annual survey on
`customersatisfaction, says Jim
`Dalrymple (macworld.com), "Apple
`
`603
`
`Figure 6F
`
`58
`
`58
`
`

`

`U.S. Patent
`
`Jul. 23, 2013
`
`Sheet 57 of 84
`
`US 8,493,344 B2
`
`Window
`
`Apple leads the way in customer
`satisfaction
`April 17, 2009
`WhenForrester Research released
`the results of its annual survey on
`customersatisfaction, says Jim
`Dalrymple (macwortd.com), "Apple
`
`Round Rect Button
`
`Textured Button
`
`on only...
`
`Figure 6G
`
`59
`
`59
`
`

`

`U.S. Patent
`
`Jul. 23, 2013
`
`Sheet 58 of 84
`
`US 8,493,344 B2
`
`Adve
`
`yokes
`
`®©©®©©©®®®
`
`L-9€9
`
`saun!
`
`GO/ELIP
`
`GO/6LIP
`
`60/8L/P
`
`6O/LLIP
`
`60/9b/P
`
`60/SL/P
`
`6O/rLit
`
`6O/ELIP
`
`6O/ELIP
`
`60/6L/P
`00°6S
`
`opeds&s|seopod&smousAL[ES]
`
`seuobuly
`
`SSIAOWHdIsnlytf
`
`algseunyJAYOLS
`
`Speo|uMogGl
`
`GayvHs<4
`
`
`
`SOapiAJISNYGl
`
`payeydoyAy&]
`
`rqsouny
`
`snug
`
`dISAW3,06Gl
`
`SLSTAVId&
`
`60
`
`“1*slaijaqplayAjLuuy4a]JOOB19]CO}padi0yajdoedJosaualS
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
` GEROET)enuesvouysuia29:0|[atigxS]93:0|[omexs]gs:0|Pomaxs]9s:0|(oTEKS]95:0|Forge]“unssysoysuysman[A]~aisuerxgsingdwe[A~“eudemjoesdeiiod(iSex3]“esieyo]weigajnow[A]93:0““SpUl}UBLUOMABOY[A]“860,sbegJejst1yo
`
`
`
`40:1|Fotexa]yeany-z[A“snrsepuag
`
`
`
`
`parediso)gzdo,[G]
`
`
`
`L-8€9
`
`c-8€9
`
`H9eunbi4
`
`v9
`
`HOO9IN
`
`
`
`pokelgAqusoeyFl
`
`
`
`
`
`peppyAnusoey[B]
`
`60
`
`
`

`

`U.S. Patent
`
`Jul. 23, 2013
`
`Sheet 59 of 84
`
`US 8,493,344 B2
`
`y