PETITIONER'S DEMONSTRATIVES
`
`APPLE 1115
`
`Apple v. Smith Interface
`F I S H Y IPR2024-01089
`
`
`
`
`
`
`
`
`Case Overview
`
`Second Duration Scenario: Hotelling Shows Pairing Tactile Feedback
`With a Long Duration Touch Was Known and Obvious (Elements [1.c],
`
`[16.h], [84.d])
`
`12
`
`Second Duration Scenario: The Combined Teachings of Hotelling and
`Martyn Show Calling a Contextual Menu With a Long Duration Touch
`Was Obvious (Elements [1.d], [16.i], [84.€e])
`
`17
`
`Second Menu Interaction: The Hotelling-Martyn-Cho Combination
`Renders Obvious the Claimed Features of Using Slide/Swipe Gestures to
`Change the Menu in Five Different Ways (Elements [1.f], [16.k], [84.g])
`
`23
`
`FISH.
`
`
`
`
`
`
`
`
`
`Overview: The ’'212 Patent —
`
`a2 United States Patent (10) Patent No.: US 10,671,212 B1
`Smith (45) Date of Patent: *Jun. 2, 2020 L
`
`uso
`
`S010671212B1
`
`(56) References Cited
`
`U.S. PATENT DOCUMENTS
`
`i - i 3358493 A 121967 Ribback
`S1, LLC. Wilmington, DE 3,524,169 A £1970 McAuliffe et al
`
`(71) Applicant: P47
`(US (Continued)
`
`(72) lnventor: Michael S. Smith, Palo Alto, CA (US) FOREIGN PATENT DOCUMENTS [ J
`ssignce: PATENTS - Wi o » 2008033739 A 22008
`
`(73) Assignee: PATENTSI, LLC, Wilmington, DI -3 ollans A aon
`
`(US) .
`
`(Continued)
`
`(*) Notice: Subject t any disclaimer, the term of this THER DI ~ AT Q
`
`patent is extended or adjusted under 35 OTHER PUBLICATIONS
`
`US.C. 154(b) by 0 days. US 7.965,578 B2, 06/2011, Bhakia et al. (withdrawn)
`
`This patent is subject to a terminal dis- (Continued)
`claimer. )
`Primary Examiner — Antonio Xavier [ ]
`o1 A ) (74) Attorney, Agens, or Firm — Patrick E. Caldwell,
`(@1 Appl. No.: 16/558.028 Esq: The Caldwell Firm, LLC
`
`(22) Filed: Aug. 30,2019 7 ABSTRACT
`
`In various embodiments, a system, method, and computer
`
`Related U, Application Data program product are provided, where one embodiment
`
`involves a non-transitory computer-readable media storing
`
`(63) Continuation of application No. 16/377,142. filed on computer instructions that; when executed by at least one
`
`Apr. 5, 2019, which is a continuation of appli processor of a mohile device including a touch screen, and
`
`(Continued) a memory, and an actuator coupled to the at least one
`processor; cause the mobile device to: display indi
`
`ciated with an application, utilizing the touch screen; when
`
`(51) 7
`= a first duration of a touch being applied 1o the touch screen
`z% %j;” :;’3‘:? 8:: is detected as ceasing in connection with the indicia, perform = o) o g
`N (Continued) different than the first duration of the touch, being applied to
`(52) the touch screen is detected in connection with the indicia
`
`CPC ...ccccc.... GO6F 30416 (2013.01). GO6F 3016 yfier the first duration is detected without the ceasing, output
`(2013.01); GO6F 3/044 (2013.01); GOGF foedback that is perceptible by touch. utilizing the actuator; .
`3045 (2013.01); when the second duration of the touch being applied to the 1 O 3—H0te 1 1 ln S 2 006/0 1 9 7 7 5 3
`(Continued) touch screen is detected in connection with the indicia after 9
`
`CPC o.o........... GOGF 3/04847; GOGF 3/04817: GOGF display at least one menu including a plurality of particular
`o OO SORRS S when it b i 2 o artyn an 0
`See application file for complete search history (Continued) 64 84 ?
`2
`
`~ (US 2011/0252362)
`
`hr N 5 56.57 §103-Hotelling, Martyn, Cho, and
`Chaudhri (US 2011/0252380)
`
`
`
`
`
`
`
`
`
`Overview: The ’'212 Patent —
`
`[ B
`C What is claimed is:
`(T 1. A non-transitory computer-readable media storing com-
`US0I0671212B1 puter instructions that; when executed by at least one
`a» United States Patent (10) Patent No.: US 10,671,212 B1 processor of a mobile device il]CIUding a touch screen, a
`
`Smith (#5) Date of Patent: “Jun. 2, 2020 memory, and an actuator coupled to the at least one proces-
`sor; cause the mobile device to:
`
`(54) GESTURE-EQUIPPED TOUCH SCREEN (56) References Cited . L. . ; . A A
`SYSTEM, METHOD, AND COMPUTER US. PATENT DOCUMENTS display indicia associated with an application, utilizing
`(71) Applicant: PATENTSL, LLC., Wilmingion, DE S A Rttt = : . . .
`“ontinu 7 Q 10 C ouc O ouc
`s (Contizued) when a first duration of a touch being applied to the touch
`(72) Inventor: Michael S. Smith, Palo Alto, CA (US) FOREIGN PATENT DOCUMENTS screen iS detected as CeaSlllg in COIllleCtiOll Witll tl]e
`O A gy o LLC: Wilington DE -y M s ¥ A indicia, perform an operation;
`(*) Notie: Subject 0 any dislaimer, the erm oftis TR PUBLICATIONS when a second duration of the touch, that is different than
`tent is extended or adjusted under 35 ) ~ . ~ . .
`USC. skt by O doys, U5 7965578 82, 0672011, Bhakn o . Gvthdrw) the first duration of the touch, being applied to the
`his patent is subject 1o a terminal dis- ‘ontinued . . - . . . .
`R touch screen is detected in connection with the indicia
`@1) Appl. Nos: 165558028 (74 Atorny Agen or Firm ek . Cadel, after the first duration is detected without the ceasing,
`@) Tl Aug 30,2019 “ ABSTRACT output feedback that is perceptible by touch, utilizing
`::. \1Ln:‘;||<p cm‘hn:limcm;, a \,iwlcm 1.|ncm.u‘_ ;nm:n ;:»nlly:l:r [he actuator;
`Related U.S. Application Data ’:‘f ;’w v "f’n “\'I"O f“;:‘ “-g“rn‘_';"'““f "\ ”“\‘fl e N . R . .
`T A S touch screen is detected in connection with the indicia
`PR e whh an applicaton. iling h tench seven: when after the first duration of the touch is detected without
`GOGF 3041 (2006.01) a fist duration o a touch being applied (0 the touch screen fl ) d . 1 : at l ot includi
`ok e (00600 i detected i ceming i comncton il the i, perorm he ceasing, display at least one menu including a
`. an operation; when a second duration of the touch, that is . ~ . .
`) Us.cl (Continued) different than the first duration of the touch, being applied to
`T CRC .. GoeF 30416 (2013.01): GOGF 3016 l:;;:;:i’:.;::::.?-.:;y:l.k.ic:l‘::x:':‘l‘:':::;\..:‘:‘:;::‘II.‘\:::;I:::.iul;:x:
`(2013.01); GO6F 3/044 'x:;::: ‘.'3;.??'.7'{ feedback that is perceptible by touch, utilizing the actuator; . ;
`Comime) v eren s dtoio n conneeion it he e sfr is detected in connection with at least one of the
`O o sty GG 404817 oGt diply t st one e nchding » phrslty of ol particular actions of the at least one menu after the
`3/0482; GOGF 3/04883 actions: when a selection touch being applied to the touch . . . .
`See application file for complte scarch history (Continued) second duration of the touch being applied to the touch
`- screen is detected in connection with the indicia after
`Ca—
`@ the first duration of the touch is detected without the
`T ceasing, perform the at least one particular action; and
`N when a slide or swipe gesture being applied to the touch
`
`== screen is detected after the second duration of the touch
`
`being applied to the touch screen is detected in con-
`
`N nection with the indicia after the first duration of the
`
`| touch 1s detected without the ceasing, change at least
`one aspect of the display of the at least one menu.
`
`FISH.
`
`
`
`
`
`
`
`
`Overview: The ’'212 Patent —
`
`when a first duration of a touch being applied to the touch
`screen 1s detected as ceasing in connection with the
`indicia, perform an operation;
`
`FISH.
`
`
`
`
`
`
`
`
`
`FISH.
`
`Overview: The ’'212 Patent —
`
`when a second duration of the touch, that 1s different than
`the first duration of the touch, being applied to the
`
`touch screen 1s detected in connection with the indicia
`after the first duration i1s detected without the ceasing,
`
`output feedback that 1s perceptible by touch, utilizing
`the actuator;
`
`when the second duration of the touch being applied to the
`touch screen 1s detected 1n connection with the indicia
`after the first duration of the touch is detected without
`the ceasing, display at least one menu including a
`plurality of particular actions;
`
`
`
`
`
`
`
`
`
`Overview: The ’'212 Patent —
`
`when a selection touch being applied to the touch screen
`
`1s detected 1n connection with at least one of the
`particular actions of the at least one menu after the
`second duration of the touch being applied to the touch
`screen 1s detected 1in connection with the indicia after
`the first duration of the touch 1s detected without the
`ceasing, perform the at least one particular action; and
`
`FISH.
`
`
`
`
`
`
`
`
`
`Overview: The ’'212 Patent —
`
`when a slide or swipe gesture being applied to the touch
`screen 1s detected after the second duration of the touch
`being applied to the touch screen 1s detected 1n con-
`nection with the indicia after the first duration of the
`touch 1s detected without the ceasing, change at least
`one aspect of the display of the at least one menu.
`
`FISH.
`
`
`
`
`
`
`
`
`
`et
`
`\/
`Tl 1426 @B 5‘0
`
`2L2a[| PPA cELL 1+-7228 T} auncher 7 7 A
`
`@oassPM
`| °4 [Tlessages
`S upe A TiMusIC| | VIDEO T LeLE % g
`0 new messages
`
`leame | | aps [r-202f || Create message
`
`21D 7| .
`Enter chat room &'fio“? it o)
`
`REMOTE camern b UL H ’ Nobody (Englsh Ver) N & J/
`
`e | Phone _——
`pe HANDTOP OTHER (©) \ 620
`
`211 :
`Figure 2
`FIG. 18
`
`FISH. _
`
`
`
`
`
`
`
`
`
`PETITION FOR INTER PARTES REVIEW OF UNITED STATES PATENT
`
`NO. 10,671,212 PURSUANT TO 35 U.S.C. §§ 311-319, 37 C.F.R. § 42
`
`Accordingly, the Hotelling-Martyn-Cho
`Combination represents the predictable and expected result of applying in the
`context of Hotelling’s touch screen mobile device: [a] Martyn’s teachings about
`providing a contextual shortcut menu of common application functions in response
`to a press-and-hold user input associated with an application icon, which builds on
`Hotelling’s similar disclosure of a pop-up contextual menu invoked by a long-touch
`gesture; and [b] Cho’s teachings about using slide/swipe touch gestures to
`manipulate information windows, which builds on Hotelling’s similar disclosure of
`
`using those same gestures for manipulating GUI elements. EX1003, 473.
`
`204 2,2
`i/ Tl 14:26 [
`2e2a-T] PoA r,?Jo?—!? Launcher
`-, [Messages
`e &S
`ZW—“—’W 0 new messages| ¢
`m m ~262fF || Create message
`adid /_- - Enter chat room
`,M_(,/“-w -m - et Phone
`21T Figure 2
`FIG. 18
`= ®
`MESSAGES CELL MESSAGES CELL MESSAGES CELL 1
`(0 new messages
`MUSIC VIDEO MuSIC VIDEO + Create message VIDEO
`Enter chat room
`GAME GPS | e— GAME GPS | e— GAME GPS
`Touch Hold
`
`REMOTE CAMERA REMOTE CAMERA REMOTE CAMERA
`HANDTOP OTHER HANDTOP OTHER HANDTOP OTHER
`
`FISH.
`
`
`
`
`
`
`
`
`
`m EgE
`- b
`verview: Petitioner’s Expert—
`=
`C : =
`o
`IN THE UNITED STATES PATENT AND TRADEMARK OFFICE IN THE UNITED STATES PATENT AND TRADEMARK OFFICE
`N . Y Qent In re Patent of: Michael S. Smith
`
`In re Patent of: Michael S. Smith . e
`
`U.S. PatentNo: 10671212 Attorney Docket No. 50095-01691P1 US. Patent No.: 10,671,212 Attorney Docket No. 50095-01691P1
`
`Issue Date: Tune 2. 2020 - Issue Date: June 2, 2020
`
`Appl. Serial No.: 16/558,028 Appl. Serial No.: 16/558.028
`
`Filine Date: August 30. 2019 Filing Date: August 30, 2019
`
`Title: GESTURE-EQUIPPED TOUCH SCREEN SYSTEM, Title: ‘?IFESTTHLORDEEg;)lfg%\?P{(%g}({}]{J};(()fia\\ fg}fg&[fi
`
`METHOD, AND COMPUTER PROGRAM PRODUCT ! Al : TRAL
`DECLARATION OF DR. ANDREW COCKBURN SUPPLEMENTAL DECLARATION OF DR. ANDREW COCKBURN
`o
`< of a POSITA’s baCkground
`\.
`\\\ 3
`. ’,{df
`[ J
`Cockburn Dec. (EX1003) Cockburn Supp. Dec. (EX1062) I3
`
`FISH.
`
`
`
`
`
`
`
`
`
`FISH.
`
`
`
`
`
`
`
`
`Hotelling Teaches “Long” Touch Inputs and
`
`PETITION FOR INTER PARTES REVIEW OF UNITED STATES PATENT
`NO. 10,671,212 PURSUANT TO 35 U.S.C. §§ 311-319, 37 C.F.R. § 42
`
`Hotelling further provides to the Combination, a teaching of outputting [0163] Audio and/or tactile feedback may be used to alert
`a user that a user input has been made. For example, in
`response to touching a virtual button on the GUI, the haptics
`may provide force feedback in the form of vibration and the
`speaker may provide audio feedback in the form of a click.
`01611, [0173]; supra, Element [ 1.pre] (mapping Hotelling’s “haptics mechanism™ to The tactile audio feedback can be used in conjunction with
`an input event including touch events, motion events,
`squeeze events. The feedback may provide information so
`that the user knows that they actually implemented an input
`(simulates the audio and tactile feel of a button or switch).
`
`“tactile feedback™ (feedback that is perceptible by touch) with a haptics mechanism
`
`to “alert a user that a user input has been made.” EX1004, [0163]; see also [0160-
`
`the claimed actuator). Specifically, Hotelling explains that “tactile...feedback can
`
`be used in conjunction with an input event including touch events”™—e.g., short and
`
`long (second) duration touches on “a virtual button.” /d. Thus, a POSITA would
`
`[0149] Duration is defined as the amount of time the finger
`stays at any one point. Duration may be variable or it may
`of the touch being applied to the touch screen is detected in connection with the include states such as short and long. The touch location may
`be a random point or a specific location such as an onscreen
`button.
`
`have understood Hotelling to teach the claimed feature of when a second duration
`
`indicia, output feedback that is perceptible by touch, utilizing the actuator.
`
`EX1003, 9125.
`
`FISH.
`
`
`
`
`
`
`
`
`
`Rebuttal
`
`to:
`
`he one section that discusses ‘short’and ‘long’ duration touches contains no reference to ‘input events’or
`‘touch events’that would explain how this disclosure connects to the haptic feedback section of Hotelling in
`the way Petitioner claims.” Sur
`
`Patent Owner’s Narrow View of Hotelling Disregards the POSITA’s Extensive
`
`Cockburn Supp. Dec. (EX1062)
`
`19. A POSITA would not have the narrow view of Hotelling’s disclosure
`that Dr. Madisett1 presents, especially not at the time of the *212 patent’s earliest
`claimed priority date in 2011. By that time, as explained in my Prior Declaration,
`the concept of using tactile feedback in response to user input commands on a touch
`screen had been known for decades and was already a deeply explored subject in the
`
`specific context of touch screen mobile devices. See EX1003, 9946—48.
`
`Therefore, we can classify all
`tactile feedback during touch
`screen interaction into five
`generic types: tactile feedback
`provided when the user starts, a
`gesture by touching a GUI
`element (T1), when the user
`then either drags (12) or holds
`(T3) her pen/finger, and,
`finally, when the user /ifs 1t off
`either inside (T4) or outside
`(T5) the GUI widget.
`
`EX1060, p.3.
`
`FISH.
`
`
`
`
`
`
`
`
`
`Rebuttal “The Petitions Proposed Combination Does Not Disclose or Suggest Providing Haptic Feedback Upon
`to: Detection of a Long Touch.” Sur
`
`Feedback at Release, Which Applies to a “Long” Touch
`PETITIONER’S REPLY TO PATENT OWNER’S RESPONSE
`In any event, setting aside that it stems from a fundamental misreading of PATENT OWNER RESPONSE
`
`Hotelling, Patent Owner’s view that tactile feedback would occur “upon release™ of \ Thus, a POSITA would have understood Hotelling to disclose “outputting
`
`a virtual button is a sufficient concession to settle the dispute. POR, 28-29. After tactile feedback™ with a haptics mechanism in order to mimic physical buttons by
`
`providing haptic feedback upon the initial press and/or upon release—
`
`all, the claims do not recite that the feedback occurs “during an ongoing hold,” (POR,
`
`29 (emphasis altered)), merely when a second duration of the touch is detected. PATENT OWNER SUR-REPLY
`
`. PO does not propose any construction beyond confirming all
`There is no reason why the end of a long touch—"upon release”™—cannot be the
`
`/ conditional limitations must be met. POR, 13-17.
`
`claimed second duration. Patent Owner’s contrary position seemingly stems from
`its now-debunked construction that the claims require a “multi-part gesture.” POR,
`
`26; see supra §1.
`
`FISH.
`
`
`
`
`
`
`
`
`Tactile Feedback Responsive to a “Long” Touch Event
`
`PETITION FOR INTER PARTES REVIEW OF UNITED STATES PATENT
`NO. 10,671,212 PURSUANT TO 35 U.S.C. §§ 311-319, 37 C.F.R. § 42
`
`IN THE UNITED STATES PATENT AND TRADEMARK OFFICE
`
`Second,
`
`o : : If the device takes an
`even setting aside Hotelling’s express suggestion, the POSITA would have known
`
`that tactile feedback was a well-known solution to the longstanding problem in action or switches modes absent feedback, the user may lose track of what 1s gomg
`
`mobile device computing of signaling significant events and/or prompting user on, which contradicts well-known principles of user interface design. See EX1054,
`
`interaction. EX1003, 127; see also 945-48 (discussing, with corroborating w .o . .
`p-1 (“The system should always keep users informed about what 1s going on, through
`
`B Y R N TN N N N NP T e
`
`solution. /d., §9128-130. As discussed, one of the significant events in Hotelling appropriate feedback within reasonable time.”). As discussed, “[wlithout tactile
`
`occurs when the user’s “long” touch triggers a context-specific menu. EX1004, feedback, the user can only rely on audio and visual senses, which have many
`
`[0151], Figures 27A-27B. The POSITA, appreciating that the user should be
`
`disadvantages.” EX1060, p.1.
`
`notified of this event and given a cue to interact with the menu, would have been
`
`)
`
`motivated to use the widespread solution of tactile feedback. EX1003, 4130.
`
`Cockburn Supp. Dec. (EX1062)
`
`FISH.
`
`
`
`
`
`
`
`
`FISH.
`
`
`
`
`
`
`
`
`— “Press and Hold”
`
`[0015] The present invention offers, in one implementa-
`tion, a snap-shot view which brings together, in one sum-
`mary window, a limited list of common functions and
`commonly accessed stored data which itself can be reached
`directly from the main menu listing some or all applications.
`This yields many advantages in ease and speed of naviga-
`tion, particularly on small screen devices. For example, a
`user can get to the summary window in just two steps—first,
`launch a main view which shows various applications; then,
`launch the appropriate summary window for the application
`of interest. This 1s far faster and easier than conventional
`navigation approaches. Once the summary window is
`launched, core data/functionality is displayed and can be
`accessed in more detail can typically be reached simply by
`selecting that data/functionality. Hence, only three steps may
`be needed from start up to reaching the required data/
`functionality; navigating from between each step is clear and
`straightforward.
`
`For example, the App Snapshot
`may be ‘called’ or ‘fired’ by using a right scroll function (if
`the mobile telephone has 4-way scrolling capacity); a press
`and hold of the select function when an application name is
`highlighted; or using a right cursor key on a highlighted
`application name.
`
`T il 14:26 [
`Launcher
`
`Messages
`
`0 new messages] 3
`
`Create message
`Enter chat room |
`
`Phone
`
`Launcher
`
`\éé mess'au;s;
`
`<& Contacts
`() Catendar
`Phone
`
`FISH.
`
`Figure 2
`
`
`
`
`
`
`
`
`
`Hotelling’s Main Menu GUI With Martyn’s App Snapshot,
`“ ” Touch Hotelling Uses for a Similar Purpose
`
`PETITION FOR INTER PARTES REVIEW OF UNITED STATES PATENT
`NO. 10,671,212 PURSUANT TO 35 U.S.C. §§ 311-319, 37 C.E.R. § 42
`
`-~
`
`e 2,0
`
`Tal 14:26 @B
`
`=
`624 PDA r‘u’u Launcher
`
`2026 ||©c Messages
`
`N
`A POSITA would have been motivated to supplement Hotelling’s teaching of e TPe] 0 now messages) 3
`. 4 "-—7—"7-F (E:reate ;nessage
`. . . . . . . nter chat room
`a touch screen mobile device that provides a main page GUI of application icons o] [t T e
`with Martyn’s complementary teaching to provide a contextual shortcut menu in FIG. 18 Higure 2
`response to a press-and-hold input on an application icon. EX1003, §75. The = 5 +
`ESSAGES CELL ] MESSAGES CELL ] l;ssms CELL ]
`
`predictable result of this Hotelling-Martyn Combination is a mobile device that
`
`Touch Hold
`
`displays a contextual menu, such as Martyn discloses, in response to a “long” touch
`
`JHIEIHIE
`4]
`l
`]
`]
`(1] 13) 8]
`
`gesture applied by the user to one of the icons on Hotelling’s main page GUI. Id.
`
`FISH.
`
`
`
`
`
`
`
`
`
`Rebuttal “Absent an articulated reason to combine these disparate disclosures in the face of contextual differences
`to: between each disclosure, the Petition s motivation can only be impermissible hindsight.” Sur
`
`Hotelling’s
`
`PETITION FOR INTER PARTES REVIEW OF UNITED STATES PATENT m A
`NO. 10,671,212 PURSUANT TO 35 U.S.C. §§ 311-319, 37 C.F.R. § 42 T o
`
`Third, a POSITA would have been prompted to pursue a combination with may be ‘called’ or ‘fired’ by using a right scroll function (if
`
`the mobile telephone has 4-way scrolling capacity); a press
`and hold of the select function when an application name is
`highlighted; or using a right cursor key on a highlighted
`application name.
`
`I
`| 0O O R
` United States
` Patent Application Publication . 197753 AL
`ot - Sep. 7.2
`
`[0093]. This disclosure by Hotelling would have prompted a POSITA to seek out SR e T
`
`Touchdown
`
`Martyn based on Hotelling’s suggestions to do so. EX1003, 9983-84. For example,
`
`Hotelling teaches the notion of displaying a “context sensitive menu pop-up” in
`
`response to a “long” touch by the user. EX1004, [0151], Figures 27A-27B; see also
`
`and apply the teachings from references in the same field, like Martyn, that provide Duration (vlm::; Proposed Behavior Feedback
`button?)
`implementation details on contextual menus launched in response to a “long” user
`On touch-down, do nothing. On c )
`n o.M ’s press-and-hold). EX1003, 983. virtual click-down highlight item under ontext sensitive
`put (e il artyn's press-a d-ho d) ’ fl e button touch (if any). After delay, bring- menu pop-up
`up context sensitive menu. ‘
`
`FISH I
`w
`
`
`
`
`
`
`
`
`Rebuttal “Absent an articulated reason to combine these disparate disclosures in the face of contextual differences
`to: between each disclosure, the Petition s motivation can only be impermissible hindsight.” Sur
`
`Hotelling’s
`
`IN THE UNITED STATES PATENT AND TRADEMARK OFFICE
`
`Given the prevalent use of long-duration touch gestures to invoke
`
`In re Patent of: Michael S. Smith
`
`U.S. Patent No.: 10,671,212 Attorney Docket No. 50095-0169IP1
`Issue Date: June 2, 2020
`
`Appl. Serial No.: 16/558.028
`
`FlingDue: A 0l CH SCREEN SYSTEM menus and control elements (see above) and Hotelling’s consistent use of this
`METHOD, AND COMPUTER PROGRAM PRODUCT
`
`SUPPLEMENTAL DECLARATION OF DR ANDREW COCKBURN approach, it would not have made sense for the POSITA to deviate from such a
`convention. Accordingly, the fact that Hotelling discloses a “long” touch gesture on
`a virtual button m the Figure 27 music player GUI for calling a contextual menu
`would have prompted the POSITA to use the same gesture for the similar function
`
`of mvoking Martyn’s App Snapshot in the Figure 18 main page GUI, which likewise
`
`includes virtual buttons.
`
`FISH
`g
`
`
`
`
`
`
`
`
`
`Rebuttal to: “The Reply’s Argument that Martyn is Directed to Touch Screen Devices is Both New and Incorrect.” Sur
`
`Touch Screen or Not, a POSITA Would Have Recognized Martyn’s
`Hotelling’s
`
`44. Additionally, even if Martyn were limited to a press-and-hold input
`
`IN THE UNITED STATES PATENT AND TRADEMARK OFFICE applied to a physical button or a virtual representation of a physical button (which I
`In re Patent of: Michael S. Smith
`U.S. Patent No.: 10.671.212 Attorney Docket No. 50095-01691P1
`e s 1o merely assume as a hypothetical), that would not change the fact that a “long” touch
`METHOD, AND COMPUTER PROGRAM PRODECT gesture on a virtual button/application icon is an obvious analog that a POSITA
`
`SUPPLEMENTAL DECLARATION OF DR. ANDREW COCKBURN
`
`would have been motivated to employ when integrating the App Snapshot into
`
`Hotelling’s main page GUIL As one example, I note that the online Android
`Developer’s guide in 2011 described an event listener called “onl.ongClick()” that
`
`et
`/\/‘\‘\\\\l applied equally to long duration inputs based on touch and physical key presses.
`“\\\\‘/
`T
`
`onLongClick ()
`From view.OnLongClickListener. This is called when the user either touches and holds the
`
`item (when in touch mode), or focuses upon the item with the navigation-keys or trackball and
`presses and holds the suitable "enter" key or presses and holds down on the trackball (for one
`
`second).
`
`Cockburn Supp. Dec. (EX1062) Iy EX1061, p.1.
`FISH.
`
`
`
`
`
`
`
`
`
`FISH.
`
`
`
`
`
`
`
`
`— Cho’s Teachings Render Obvious
`
`A. First Way: Highlighting Menu Items, Per Hotelling-Martyn................. 13
`B. Second Way: Dismissing the Menu, Per Hotelling-Martyn ................... 15
`C. Third Way: Scrolling the Menu, Per Hotelling-Martyn-Cho ................. 16
`D. Fourth Way: Resizing the Menu, Per Hotelling-Martyn-Cho................. 18
`E. Fifth Way: Dismissing the Menu, Per Hotelling-Martyn-Cho............... 20
`
`FISH.
`
`
`
`
`
`
`
`
`
`— Cho’s Teachings Render Obvious
`
`PETITION FOR INTER PARTES REVIEW OF UNITED STATES PATENT
`NO. 10,671,212 PURSUANT TO 35 U.S.C. §§ 311-319, 37 C.F.R. § 42
`
`[0127] Thecontents of the information window 620 may be
`scrolled independently in response to a flick on the informa-
`tion window 620. If a hyperlink object, if any, in the informa-
`tion window is selected, information relevant to the hyperlink
`object may be displayed.
`
`Cho further explains that the detection of a slide/swipe gesture can trigger at
`least three different changes to at least one aspect of the display of the information
`window (menu)—namely, via scrolling, re-sizing, and dismissing actions. EX1003,
`14147-150. First, Cho teaches that “[t]he contents of the information window 620
`may be scrolled...in response to a flick [(swipe)] on the information window,” thus
`changing the appearance (aspect) of the menu display. EX1006, [0127]; see also
`
`[0079] (describing a touch-and-flick (swipe) or touch-and-drag (slide) to invoke
`
`“scroll command”), [0091], [0108], [0132]; EX1003, §148.
`
`670
`9 - 610
`Y B o @0356PM J
`News (Cancel [[ - A
`Nbn news [ (Tw@o ilo \
`South Korea says no r \ 1‘
`
`Lawyers sef to use
`iPad in court
`
`Both prosecutors and
`lawyers tend to picker in
`
`l—)
`the coLrtroom each armed “ = = ! |
`with a mountain of = &/le 1 2Diflerent Tears [
`documents, which consur B« Vel - Tell me (English Ver ) N
`imost of the space on thelr (o o 3 S0 Hol (English Ver) P ‘ J
`
`ables. Althouch they are \o L_lo ¢ Notody (Engiish Ver) N I/
`equired to batk up(he K \ S, j/'
`cuts, warns Cameron
`
`(0) \ 620
`
`[0132] Thereafter, if a short or long touch on the predefined
`news article 673 is detected, an information window 680
`relevant to the predefined news article 673 may be displayed,
`as shown in FIG. 19(c). The size of the information window
`680 may be configured to be adjusted in response to, for
`example, a touch-and-drag on one of the boundaries of the
`information window 650. The contents of the information
`window 680 may be configured to be scrolled independently
`in response to a scroll command 683 detected inside the
`information window 680.
`
`FISH.
`
`
`
`
`
`
`
`
`
`— Cho’s Teachings Render Obvious
`
`Rebuttal “Instead, the Reply accuses the POR of being ‘shortsighted’
`
`to: 17 (citing EX1022) with Pet., 38.” Sur
`
`Explains That Cho’s
`
`PETITION FOR INTER PARTES REVIEW OF UNITED STATES PATENT
`
`NO. 10,671,212 PURSUANT TO 35 U.S.C. §§ 311-319, 37 C.F.R. § 42
`
`Second, Cho’s use of slide/swipe gestures to interact with a contextual menu
`in an information window was a known technique, and applying it in the similar
`setting of the Hotelling-Martyn contextual menu window to obtain a substantially
`similar result would have been obvious as a matter of law under KSR. EX1003,
`9989-90; see also 91, 94-97 (explaining, with corroborating evidence, that
`movement gestures like slides/swipes were well known and employed in commercial
`touch screen products for interfacing with windowed contextual menus), 992
`
`(explaining that a POSITA would have expected Cho’s teachings to apply similarly
`
`in Hotelling given the similarities between their touch screen mobile devices).
`
`Fourth, Cho’s use of slide/swipe gestures as commands for basic functions
`like scrolling would have been seen as particularly advantageous from the POSITA’s
`perspective. EX1003, 994. For example, these types of motion-based gestures were
`
`already used in this manner across various commercially available touch screen
`
`mobile devices. Id., 9991, 94-97. Provisioning a known gesture to trigger a known
`function would have made the Hotelling-Martyn-Cho device easy and intuitive to
`
`use for consumers with prior experience operating older mobile devices. 7d., 498.
`
`FiSH-.
`
`
`
`
`
`
`
`
`
`— Cho’s Teachings Render Obvious
`
`“Instead, the Reply accuses the POR of being ‘shortsighted’
`
`Rebuttal
`to: 17 (citing EX1022) with Pet., 38.” Sur
`
`Explains That Cho’s
`
`C Figure 4-9 List pickers that are too long to display all at once have scroll arrows
`Cockburn Dec. (EX1003)
`AT Y Y Y S /AT . v - Unfiled Notes - Folder 06
`IN THE UNITED STATES PATENT AND TRADEMARK OFFICE Fa— Folder 07
`Inre Patentof: Michael S. Smith Folder 01 Folder 08
`USS. PatentNo.: 10671212 Attorney Docket No. 50095-01691P1
`Jomue Date: June 2, 2020 o e 12 SO | rolder 02 e S0 ig:gz: lllg
`Appl. Serial No.: 16/558,028 Folder 03 Folder 11
`Filing Date: August 30, 2019
`Tiler G‘EiuTURE-LQUIPPED TOUCH SCREEN SYSTEM, Bel‘l a go:ger g; ee“ a Folder 12
`METHOD, AND COMPUTER PROGRAM PRODUCT Folde" 06 Folder 13 — Scroll arrows for
`oug L ey 07 049 U roider 13 picker
`DECLARATION OF DR. ANDREW COCKBURN Folder 08 Folder 15
`........... Folderog FDlderlfi
`Folder 10 Folder 17
`EX1020,p.6 | e Folder11 |} ... ::::g::glneous
`- Folder 12 vQuotes
`91. Cho’s teachings in this regard were widely known by 2011 and had | | oo i e
`= = 1. Before user taps down arrow 2. After user taps down arrow
`
`several times
`even matured to the point of commercial adoption. For example, the graphical user
`
`interface for Apple’s Newton, the touch screen PDA I discussed above in Section As usual, the color of the scroll arrow indicates whether tapping it will bring
`more items into view. An arrow is black if tapping it will bring more items
`
`VLB, featured a windowed menu called a “list picker” containing selectable into view. An arrow is white if tapping it will not bring more items into view.
`
`Users can also scroll list pickers by dragging from the middle of the picker
`
`“commands” that could be manipulated by the user (e.g., to highlight, scroll, and past the top or bottom of the picker. Users cannot scroll a list picker with the
`universal scroll arrows. Tnpping a universal scroll arrow or anywhere else
`dismiss the list picker) using sliding/dragging gestures. EX1022, p. 127 outside a list picker makes the picker go away.
`
`FISH.
`
`
`
`
`
`
`
`
`
`