APPLE INC. v. SMITH INTERFACE TECHNOLOGIES, LLC
`
`IPR2024-01085
`U.S. Patent No. 10,649,580
`
`Patent Owner’s Hearing Demonsiratives
`
`November 14, 2025
`
`Apple Inc. v. Smith Interface Techs., LLC
`IPR2024-01085 | Smith EX2013
`Page 1 of 54
`
`
`
`
`
`
`
`
`'580 Patent Overview
`
`Prior Art Overview
`
`Ground 1A (Ramos-Ramos-Ording)
`
`Ground 1B (Ramos-Ramos-Ording-Hayward)
`
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`
`
`
`
`
`
`
`'580 Patent Overview
`
`Prior Art Overview
`
`Ground 1A (Ramos-Ramos-Ording)
`
`Ground 1B (Ramos-Ramos-Ording-Hayward)
`
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`
`
`
`
`
`
`
`U.S. Part
`
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`
`
`
`
`
`
`
`U.S. Patent No. 10,649,580
`
`a2 United States Patent
`
`(10) Patent No.: ~ US 10,649,580 B1
`
`US010649580B1
`
`Smith (45) Date of Patent: May 12,2020
`(54) DEVICES .\u oDS, \\I) (-R\Plll( (56) References Cited
`1
`U.S. PATENT DOCUMENTS
`
`\\l)/()R HAP
`
`(71) Applicant: P4T
`(Us)
`
`C, Wilmington, DE
`
`(72) Tnventor: Michael § Smith, Palo Alto, CA (US)
`
`(73) Assignee: PATENTSI, LLC, Wilmington, DE
`(Us)
`
`(*) Notice: Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`US.C. 154(b) by 0 days.
`
`(21) Appl. No.: 16/664,777
`
`(22) Filed: Oct. 25, 2019
`
`Related U.S. Application Dat
`
`(63) Continuation of application No. 16/377.142, filed on
`Apr. 5, 2019, which is a continuation of application
`(Continued)
`
`3358493 A 121967 Erich
`3,524,169 A 81970 Ketal
`(Continued)
`
`FOREIGN PATENT DOCUMENTS
`
`BR 0213687 A 102004
`(&N 2626172 Al 1112009
`(Continued)
`
`OTHER PUBLICATIONS
`
`965,578 B2, 062011, Bhakia et al. (withdrawn)
`7965579 B1, 0612011, Solomon et al. vithdrawn)
`
`% 64 and IA-32 Architectures Software Developer's
`\hmml wl L: Busie \nhll\\lun Order No. 253665-037US, Jan
`Pp. 1540 (2 of §
`
`i 64 and TA-32 Architectures Software Developer's
`Manual,” vol. 1: Basic Architecture, Order No. 253665-037US, Jan.
`2011, pp. 1-540 (3 of §).
`
`(Continued)
`
`Primary Examiner — Antonio Xavier
`(74) tgent, or Firm — Patrick E. Caldwell,
`Esq.: dwell Firm,
`
`57 ABSTRACT
`In one embodiment, an electronic device is provided, com-
`at least one non-transitory memory:
`nera; and one or more process
`
`Wih the at least one non- transitory memory, the touch
`screen, and the camera, wherein the one or more processors
`execute instructions stored in the non-transitory memory to
`cause |hc.lpp. atus to: di i first
`virtual displ ay. via the
`touch screen
`the touch screen, at least a portion of fiouch on the |
`user interface element; when an aspect of the mmh is
`
`ne
`
`(Continued)
`
`TRATIVE EXHIBIT — NOT EVIDENCE
`
`(12)
`
`(54)
`
`(45)
`
`(22)
`
`United States Patent
`mith
`
`EVICES, METHODS, AND
`RAPHICAL USE INTERFACES
`OR MANIPULATING USER
`NTERFACE OBJECTS WITH
`ISUAL AND/OR HAPTIC
`EEDBACK
`
`ate of Patent: May 12, 2020
`
`Filed: Oct. 25, 2019
`
`
`
`
`
`
`
`
`
`Independent Claim 1 (and Claims 22 and 58)
`
`1. An apparatus comprising:
`
`[1.a-1.d] at least one non-transitory memory; ... instructions ... to cause the apparatus to:
`[1.e] display, via the touch screen, a first virtual display layer including contents;
`[1.f] display, via the touch screen, at least one user interface element;
`[1.g] detect, via the touch screen, at least a portion of touch on the least one user interface element;
`
`[1.h] when an aspect of the touch is detected to surpass a threshold, display, via the touch screen, a plurality of markings in a second
`virtual display layer that appears to have a lesser depth than the first virtual display layer,
`
`[1.i] where at least a portion of the second virtual display layer is at least partially translucent so that at least a portion of the contents
`of the first virtual display layer is visible through the at least portion of the second virtual display layer;
`
`[1.j] detect, via the touch screen, a touch movement on at least one of the plurality of marking; and
`
`[1.k] during detection of the touch movement on the at least one of the plurality of marking . . ., display, via the touch screen, a
`movement of the markings in the second virtual display layer as a function of the touch movement; and
`
`[1.1] perform a zoom operation on the at least portion of the contents . . . without performing the zoom operation on the . . . markings
`..., Where the zoom operation correlates with the movement of the markings in the second virtual display layer, and the at least
`portion of the second virtual display layer is at least partially translucent so that a result of the zoom operation on the at least
`portion of the contents of the first virtual display layer is visible through the at least portion of the second virtual display layer.
`
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE 6
`
`
`
`
`
`
`
`
`
`'580 Patent Overview
`
`Prior Art Overview
`
`Ground 1A (Ramos-Ramos-Ording)
`
`Ground 1B (Ramos-Ramos-Ording-Hayward)
`
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`
`
`
`
`
`
`
`Ramos Paper
`
`@ Ramos Paper @ Zlider widget and pressure cursor
`
`High precision parameter manipulation tasks typically
`require adjustment of the scale of manipulation in
`- | addition to the parameter itself. This paper introduces
`S~ \ | the notion of Zoom Sliding, or Zliding, for fluid
`@ | et rfi\ Il »)
`
`integrated manipulation of scale (zooming) via pressure
`need ¥ f achieved via x-y cursor movement (sliding). We also
`
`|
`
`input while parameter manipulation within that scale is
`
`le
`| ;’n""_,' present the Zlider (Figure 1), a widget that instantiates
`b) ) \ P the Zliding concept.
`‘ \ \ m EX1004, 1 (cited in Pet., 6-7; POR, 5-6)
`| | .l iy
`% ] 'I' i - # / t Though not integral to the Zlider design, we use a
`e b pressure cursor (Figure 2) across our implementations,
`N instead of the default cursor found in most GUIs. Our
`
`pressure cursor provides users with a real-time
`indicator of the pressure they are applying with the
`
`input transducer.
`EX1004, FIG. 1 (cited in POR, 5) EX1004, 3 (cited in Sur-Reply, 11)
`
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE 8
`
`
`
`
`
`
`
`
`
`Ramos Video
`
`Contents
`
`Video shows Zlider’s working area and
`UL VVernier markings adjacent to the
`content
`
`) 0037421 e =T L
`
`EX1005, 0:03 (annotated) (cited by Pet., 14); POR, 22—-24; Sur-Reply, 13
`
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE 9
`
`
`
`
`
`
`
`
`Ramos Video
`
`@ Ramos Video
`
`hovering
`
`Video shows stylus can manipulate the
`pressure cursor and Zlider widget
`without contacting the screen (i.e.,
`hovering)
`
`EX1005, 2:08 (annotated) (cited by POR, 7-8)
`
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`
`
`
`
`
`
`
`
`@ Ording’s hardware and apps
`
`Portable Multifunction Device
`
`~
`
`~
`
`206 |
`~
`Optical Proximity ]
`(Speaker m) [ _J (
`O 400A Sensor 164 Sensor 166
`$
`-
`
`2
`
`| 402
`
`. N\
`Current Time 404 & 406
`
`| 208 |
`
`LM eeel|
`Text Photos Camera Videos
`141 144 143 145
`B @)
`Weather Stocks Blog Calendar
`1491 1492 142 148
`/ N e Ty 7 User-
`+-x+ ‘ ABC ‘ ‘ Created
`. N A . Widget .
`Calculator Alarm Dictionary Widget
`149-3 149-4 149-5 1496
`. » g410 ) )
`() = &3 A
`Phone Mail Browser Music
`138 140 147 146
`408
`Touch Screen 112
`. J
`Microphone "’ Home * ("Acceleromeler(s)
`113 168
`
`J
`
`Figure 4A
`
`EX1006, FIG. 4A (cited in POR, 15)
`
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`@ Ording’s virtual click wheel
`
`“In some embodiments, the device 100 may include a physical or virtual
`click wheel as an input control device 116. A user may navigate among
`and interact with one or more graphical objects (henceforth referred to as
`icons) displayed in the touch screen 112 by rotating the click wheel or by
`moving a point of contact with the click wheel (e.g., where the amount of
`movement of the point of contact is measured by its angular
`displacement with respect to a center point of the click wheel). The click
`wheel may also be used to select one or more of the displayed icons.
`
`For a virtual click wheel, the click wheel may be either an opaque or
`semitransparent object that appears and disappears on the touch screen
`display in response to user interaction with the device. In some
`embodiments, a virtual click wheel is displayed on the touch screen of a
`portable multifunction device and operated by user contact with the
`touch screen.
`
`EX1006, [0045] (cited in POR, 14-15; Sur-Reply, 3—4)
`
`11
`
`
`
`
`
`
`
`
`
`'580 Patent Overview
`
`Prior Art Overview
`
`Ground 1A (Ramos-Ramos-Ording)
`
`* [1.h], [1.i], [22.d], and [58.c]: no semi-transparent second virtual display layer
`* [1.h], [22.d], and [58.c]: “no touch” is not an “aspect of touch”
`
`« [1.1], [22.g]., and [58f]: zoom operation is performed on the markings
` Claims 29 and 65: no “single continuous gesture with constant contact”
`
`« Claims 9, 32, and 69: no credible theory to add duration magnitude threshold
`
`Ground 1B (Ramos-Ramos-Ording-Hayward)
`
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`12
`
`
`
`
`
`
`
`
`Ground 1A: Limitations [1.h], [1.i], [22.d], and [58.c]
`
`RROD lacks “plurality of markings in a [partially translucent] second virtual display layer”
`
`@ Claim 1 (and Claims 22 and 58)
`
`[1.h] when an aspect of the touch is detected to surpass a threshold, display, via the touch screen, a plurality
`of markings in a second virtual display layer that appears to have a lesser depth than the first virtual
`display layer,
`
`[1.i] where at least a portion of the second virtual display layer is at least partially translucent so that at least
`a portion of the contents of the first virtual display layer is visible through the at least portion of the
`second virtual display layer
`
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`
`
`
`
`
`
`
`
`Ground 1A: Limitations [1.h], [1.i], [22.d], and [58.c]
`
`Two theories.
`
`0 Petitioner’s Theory 1 (Zlider Adjacent)
`
`Contents
`
`Working area
`
`P P ) 322/421 » @ 8 = o5]
`
`O Petitioner’s Theory 2 (Zlider Overlay)
`
`Portable Multifunction Device 100
`SIM Card Slot 210 212
`
`— \_head- )
`(o) (o) (Gor) e
`
`208
`—
`
`;
`
`Pet., 39 (citing EX1005, 3:22 (annotated))
`
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`Pet., 59 (citing EX1006, FIG. 2 (modified) with Ramos Zlider)
`
`
`
`
`
`
`
`
`
`Ground 1A: Limitations [1.h], [1.i], [22.d], and [58.c]
`
`First Theory: Ramos discloses Zlider widget adjacent to content
`
`O Ramos Video — Zlider adjacent to content
`
`Contents
`
`Working area
`
`P Pl ) 322/42 i @ O (& O 5] [0
`
`Pet., 39 (citing EX1005, 3:22 (annotated)); POR, 22—-24; Sur-Reply, 13
`
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`
`
`
`
`
`
`
`Ground 1A: Limitations [1.h], [1.i], [22.d], and [58.c]
`
`First Theory relies on pressure cursor to satisfy “partially translucent” requirement
`
`@ Ramos Paper @ Ramos Video
`
`Pressure cursor of Zlider overlaying the
`underlying image
`
`Underlying image
`
`Contents
`
`target
`lines
`selection 2
`line -F
`
`b
`
`Working area
`
`/ [_k
`(]
`a
`
`8 I 00 K]
`
`viewport -~ _
`wmking,/l ] e
`— area |
`— Figure 6: Elements in the experimeh;al setup. e we @ & & O K] O
`Zlider
`Pet., 56 (citing EX1004, Figure 6 (annotated)) Reply, 16—17 (citing EX1005, 0:03 (annotated))
`
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`
`
`
`
`
`
`
`Ground 1A: Limitations [1.h], [1.i], [22.d], and [58.c]
`
`“virtual display layer” includes elements assigned the same apparent depth
`
`“In various embodiments, a 3D layered user interface
`may be implemented in a device having single layer
`displays through the use of virtual display layers. In the
`context of the present description, a virtual display
`layer refers to a collection of display elements which
`have all been assigned the same apparent depth within
`the 3D layered user interface. In various embodiments,
`a 3D layered user interface may make use of multiple
`virtual display layers. For example, in one embodiment,
`a 3D layered user interface may have a virtual
`foreground display layer and a virtual background
`display layer.”
`
`EX1001, 68:7-11 (cited at POR, 2, 31; Sur-Reply, 11)
`
`@ ’580 Patent — “virtual display layer”
`
`@ ’580 Patent — “3D depth cues”
`
`In various embodiments, virtual display layers may be
`given the appearance of depth through the use of 3D
`depth cues. In the context of the present description, a
`3D depth cue refers to an effect, manipulation,
`transformation, animation, or operation which gives a
`visual indication of simulated depth. For example, in
`one embodiment, a 3D depth cue may be a blur
`operation, such that layers located at successively
`greater depths may appear blurrier than those closer to
`the user.
`
`EX1001, 68:25-33 (cited at POR, 2, 22; Sur-Reply, 11)
`
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`
`
`
`
`
`
`
`
`Ground 1A: Limitations [1.h], [1.i], [22.d], and [58.c]
`
`Pressure cursor is not in the alleged second virtual display layer with the markings
`(instead appears above the Zlider working area and Vernier)
`
`@ Ramos Paper @ Ramos Video — Pressure cursor with shadow
`
`“Though not integral to the Zlider design, we use a
`
`pressure cursor (Figure 2) across our implementations,
`
`instead of the default cursor found in most GUIs.”
`EX1004, 3 (cited in Sur-Reply, 11)
`
`P Pl ) o0n4/22
`
`EX1005, 0:14 (cited in POR, 29); POR, 28-32; Sur-Reply, 11-12
`
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE 18
`
`
`
`
`
`
`
`
`Ground 1A: Limitations [1.h], [1.i], [22.d], and [58.c]
`
`Petitioner sub-component theory contradicts the 580 Patent and Ramos.
`
`O Petitioner’s Theory 1 m Dr. Cockburn’s Supplemental Declaration
`
`“[A] POSITA would have understood that the Zlider— “A POSITA would have understood that when user
`including its components—are in the same virtual interface widgets, such as the Zlider, are placed within a
`display layer.” graphical user interface, they exist within a single layer
`Reply, 14 of that interface. This is the case even when a widget
`“To separate these components of the Zlider or consists of multiple sub-components that are drawn
`consider them otherwise located on different “planes” using a particular placement order such that some
`would be counter to the overall idea that these are subcomponents appear on top of others.”
`simply parts of a whole widget.” o
`Reply, 15-16 EX1032, 929 (cited in Reply, 15)
`
`@ Ramos Paper
`
`“Though not integral to the Zlider design, we use a
`
`pressure cursor (Figure 2) across our implementations,
`
`instead of the default cursor found in most GUIs.”
`EX1004, 3 (cited in Sur-Reply, 11)
`
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`
`
`
`
`
`
`
`
`Ground 1A: Limitations [1.h], [1.i], [22.d], and [58.c]
`
`Dependent claim 21’s open language does not require or permit multiple layers;
`cannot expand meaning beyond the 580 Patent’s clear single layer requirement
`
`O Petitioner’s Theory 1 m Phillips v. AWH
`
`“Notably, the ‘580 Patent’s claim 21 (dependent on “Claim language must always be read in view of the
`claim 1) explicitly describes scenarios where the second specification, of which they are a part.”
`virtual display layer comprises more than one layer. For Phillips v. AWH, 415 F.3d 1303, 1315 (Fed. Cir. 2005) (en banc)
`
`instance, claim 21 recites “the second virtual display (cited in Sur-Reply, 11 n.1)
`
`layer |nc.ludes at least F)ne of: an op’e’:ratlon layer, an m Retractable Techs. v. Becton Dickinson
`emphasis layer, or a clipboard layer,” thus
`
`contemplating the second virtual display layer including “In addition, the asserted claims lack language that limits the
`multiple layers. needle holder or the retainer member to elements that are
`
`"made of" one piece. Ultimately, the inventor's statement, on
`Reply, 18 its own, lacks the clarity required to exclude from the scope of
`the claims a needle holder and a retainer member that form
`distinct portions of a single structure..”
`
`Retractable Techs. v. Becton Dickinson,
`653 F.3d 1296, 1304—05 (Fed. Cir. 2011)
`(cited in Sur-Reply, 11 n.1)
`
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE 20
`
`
`
`
`
`
`
`
`
`Ground 1A: Limitations [1.h], [1.i], [22.d], and [58.c]
`
`First Theory fails because the cursor is not in the same display layer as the markings.
`
`Claim 1 (and Claims 22 and 58) iti ol Adjacent)
`
`1. An electronic device comprising:
`
`[1.h] when an aspect of the touch is detected to surpass a
`threshold, display, via the touch screen, a plurality of
`markings in a second virtual display layer that appears to
`have a lesser depth than the first virtual display layer,
`
`[1.i] where at least a portion of the second virtual display
`layer is at least partially translucent so that at least a portion
`of the contents of the first virtual display layer is visible
`through the at least portion of the second virtual display
`
`layer
`
`Zlider
`
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`
`
`
`
`
`
`
`Ground 1A: Limitations [1.h], [1.i], [22.d], and [58.c]
`
`Second Theory fails to identify any credible motivation to combine in the alleged way.
`
`O Petitioner’s Theory 2 (Zlider Overlay)
`
`Portable Multifunction Device 100 \
`— £
`
`SIM Card Slot 210
`
`1. Petition identifies no reason to
`o 2 () &) move Zlider working area from
`being adjacent to content to being
`on top of content
`
`[208] [208 ]
`
`Petition invokes semitransparency
`on Zlider to reduce “intrusive”
`“obstruction” caused by moving
`Zlider on top of content
`
`Pet., 59 (citing EX1006, FIG. 2 (modified) with Ramos Zlider) POR, 17-21; Sur-Reply, 5-10
`
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`
`
`
`
`
`
`
`Ground 1A: Limitations [1.h], [1.i], [22.d], and [58.c]
`
`Petition does not provide any credible motivation to combine Ording to
`(1) move the Zlider on top of content and (2) then make the Zlider semitransparent
`
`144
`
`“First, Ording offers foundational hardware (including processor and memory) for implementing a widget, . . ..
`Pet., 31-32 (citing EX1006, [0030], [0047]); POR, 12; Sur-Reply, 3-4
`
`“Second, Ording teaches the use of various applications configured to use a widget in the RROD, including “an e-mail
`application,” “a photo management application, a digital camera application, a digital video camera application, a web
`browsing application,” and “a digital video player application.”
`
`Pet., 32 (citing EX1006, [0027]); POR, 12; Sur-Reply, 3—4
`
`“Third, Ording further explicitly teaches providing “user interfaces that are intuitive and transparent,” such as a “virtual
`click wheel” that can be a “semitransparent object.
`
`nn
`
`Pet., 32 (citing EX1006, [0028], [0045]); POR 12-14; Sur-Reply, 4-5
`
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`
`
`
`
`
`
`
`
`Ground 1A: Limitations [1.h], [1.i], [22.d], and [58.c]
`
`Ording does not generally teach user interfaces that are optically transparent
`
`“Third, Ording further explicitly teaches providing “user interfaces that are intuitive and transparent,” such as a “virtual
`click wheel” that can be a “semitransparent object.””
`
`Pet., 32 (citing EX1006, [0028], [0045]); POR 12—14; Sur-Reply, 4-5
`
`“The various applications that may be executed on the device may use at least one common physical user-interface
`device, such as the touch screen. One or more functions of the touch screen as well as corresponding information
`displayed on the device may be adjusted and/or varied from one application to the next and/or within a respective
`application. In this way, a common physical architecture (such as the touch screen) of the device may support the variety
`of applications with user interfaces that are intuitive and transparent.”
`
`EX1006, [0028] (cited in POR, 12—14; Sur-Reply, 4-5)
`
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`
`
`
`
`
`
`
`
`Ground 1A: Limitations [1.h], [1.i], [22.d], and [58.c]
`
`Ording’s single mention of a transient virtual click wheel that may be semitransparent
`is not a credible motivation to move the Zlider over the content and make it translucent
`
`“In some embodiments, the device 100 may include a physical or virtual click wheel as an input control device 116. A
`user may navigate among and interact with one or more graphical objects (henceforth referred to as icons) displayed in
`the touch screen 112 by rotating the click wheel or by moving a point of contact with the click wheel (e.g., where the
`amount of movement of the point of contact is measured by its angular displacement with respect to a center point of
`the click wheel). The click wheel may also be used to select one or more of the displayed icons.
`
`For a virtual click wheel, the click wheel may be either an opaque or semitransparent object that appears and disappears
`on the touch screen display in response to user interaction with the device. In some embodiments, a virtual click wheel is
`displayed on the touch screen of a portable multifunction device and operated by user contact with the touch screen.”
`
`EX1006, [0045] (cited in POR, 14-15)
`
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`
`
`
`
`
`
`
`Ground 1A: Limitations [1.h], [1.i], [22.d], and [58.c]
`
`Ramos Paper and Ramos Video do not allow the
`Zlider working area with Vernier to obstruct the content
`
`O Ramos Video — Zlider adjacent to content
`
`Contents
`
`Working area
`
`P Pl O 322/421 e B @ (& o 5] o2
`
`POR, 22-23 (citing EX1005, 3:22 (annotated)); Sur-Reply, 5, 13
`
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`
`
`
`
`
`
`
`Ground 1A: Limitations [1.h], [1.i], [22.d], and [58.c]
`
`Despite implicitly conceding that it would be intrusive and obstruct the content,
`Petitioner identifies no reason or benefit for moving the Zlider on top of the content
`
`0 Petitioner’s Theory 2 O Petitioner’s Theory 2
`
`“Moreover, a POSITA would have found it obvious to modify j_m o) () & & L ) () (&)
`the Zlider (if/as needed) to make it semitransparent and
`
`thus a “less intrusive,” because increased transparency
`would minimize obstruction on image content. /d. For
`instance, a POSITA would have understood that, if only part
`of the Zlider is translucent (e.g., pressure cursor), the
`working area of the Zlider would obstruct the underlying
`content that is otherwise visible through the modified Zlider
`(Pet., 59), as illustrated below.”
`
`Underlying content Underlying content
`obstructed unobstructed
`Reply, 11-12 Reply, 11-12; Sur-Reply, 5-7
`
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`
`
`
`
`
`
`
`
`Ground 1A: Limitations [1.h], [1.i], [22.d], and [58.c]
`
`Petitioner’s unmapped “alternative Zlider” eyedropper tool
`is not a disclosure of the mapped Zlider widget being displayed on top of content
`
`O Unmapped “Alternative Zlider”
`
`No identified working area
`
`No Vernier markings
`
`P Pl ) 400/421
`
`Reply, 7-10 (citing EX1005, 4:00 (annotated)); Sur-Reply, 8
`
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`
`
`
`
`
`
`
`Ground 1A: Limitations [1.h], [1.i], [22.d], and [58.c]
`
`A translucent widget degrades core function of Zlider
`
`O Opaque, clearly visible, contrasting O On content, translucent, variable visibility
`
`> Pl ) 3167421 ne @B 8 (= O 3] 2
`
`EX1005, 3:16 (cited in POR, 16, 18; Sur-Reply, 7-8) EX1005, 3:16 (modified) (cited in POR, 18-20; Sur-Reply 7-8)
`
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`
`
`
`
`
`
`
`Ground 1A: Limitations [1.h], [1.i], [22.d], and [58.c]
`
`Second Theory fails to identify any credible motivation to combine in the alleged way.
`
`Claim 1 (and Claims 22 and 58)
`
`[1.h] when an aspect of the touch is detected to
`surpass a threshold, display, via the touch screen, a
`plurality of markings in a second virtual display
`layer that appears to have a lesser depth than the
`first virtual display layer,
`
`[1.i] where at least a portion of the second virtual
`display layer is at least partially translucent so that
`at least a portion of the contents of the first virtual
`display layer is visible through the at least portion of
`the second virtual display layer
`
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`
`
`
`
`
`
`
`'580 Patent Overview
`
`Prior Art Overview
`
`Ground 1A (Ramos-Ramos-Ording)
`
`* [1.h], [1.i], [22.d], and [58.c]: no semi-transparent second virtual display layer
`* [1.h], [22.d], and [58.c]: “no touch” is not an “aspect of touch”
`
`« [1.1], [22.g]., and [58f]: zoom operation is performed on the markings
` Claims 29 and 65: no “single continuous gesture with constant contact”
`
`« Claims 9, 32, and 69: no credible theory to add duration magnitude threshold
`
`Ground 1B (Ramos-Ramos-Ording-Hayward)
`
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`31
`
`
`
`
`
`
`
`
`Ground 1A: Limitations [1.h], [22.d], and [58.c]
`
`RROD’s stylus hover is not an “aspect of a touch” “detected to surpass a threshold”
`
`@ Claim 1 (and Claim 22 and 58)
`
`[1.g] detect, via the touch screen, at least a portion of touch on the least one user interface
`element;
`
`[1.h] when an aspect of the touch is detected to surpass a threshold, display, via the touch
`screen, a plurality of markings in a second virtual display layer that appears to have a
`lesser depth than the first virtual display layer,
`
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`
`
`
`
`
`
`
`
`Ground 1A: Limitations [1.h], [22.d], and [58.c]
`
`Petition relies on Ramos’ display of the Vernier when the stylus hovers—not on contact.
`
`@ Ramos Video — Vernier appears on hover O Petitioner’s Theory
`
`“As discussed above for [1.g], Ramos Paper and Ramos Video
`similarly teach or suggest the use of a threshold associated
`with a detected aspect of a touch: when an aspect of the
`touch is detected to surpass a threshold (i.e., the signals
`associated with the hover threshold described further
`below) when using the Zlider, the Vernier is displayed on the
`touchscreen.”
`
`Pet., 50 (citing EX1003, 1185)
`
`“A POSITA would have understood or found obvious that the
`tracking state is activated when signals from the stylus
`exceed a threshold (i.e., the stylus is close enough to the
`touch surface to be registered as movement near the
`surface).”
`
`P Pl o) 0147421 3 EX1003, 985
`
`EX1005, 0:14 ( cited in Pet., 52)
`
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`
`
`
`
`
`
`
`Ground 1A: Limitations [1.h], [22.d], and [58.c]
`
`A “touch event” requires “contact with an interaction surface” —hovering is not contact
`
`@ ’580 Patent — “touch event”
`
`“In the context of the present description, a touch event refers to an event in which
`an implement of interaction comes into contact with an interaction surface. For
`
`example, in one embodiment, pressing on a pressure-sensitive backtouch interface
`with a finger may be a touch event. Another example may be making contact with a
`
`touch-sensitive display with a stylus.”
`EX1001, 52:48-50 (cited at POR, 41; Sur-Reply, 17)
`
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`
`
`
`
`
`
`
`Ground 1A: Limitations [1.h], [22.d], and [58.c]
`
`“No touch” is the absence of a detected touch.
`
`@ ’580 Patent
`, “For example, in one embodiment, the lower
`@ 580 Patent boundary 1306 of the lowest touch state (e.g., the
`
`touch state associated with the lowest contact
`
`pressures) may be displayed.”
`EX1001, 47:36-38 (cited in Sur-Reply, 17)
`
`“For example, in one embodiment, contact pressure
`levels may be represented as one of four predefined
`touch states: no touch, light touch, medium touch,
`and heavy touch. As an option, the lowest touch state
`may require some form of proximity.”
`
`EX1001, 17:25-29 (cited in Reply, 25); POR, 40-41
`
`“In the context of the present description, contact
`pressure is the pressure associated with one or more
`contact points or contact areas, the pressure being
`exerted on an interaction surface.”
`
`EX1001, 16:14-17 (cited in Sur-Reply, 17)
`
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`
`
`
`
`
`
`
`
`'580 Patent Overview
`
`Prior Art Overview
`
`Ground 1A (Ramos-Ramos-Ording)
`
`* [1.h], [1.i], [22.d], and [58.c]: no semi-transparent second virtual display layer
`* [1.h], [22.d], and [58.c]: “no touch” is not an “aspect of touch”
`
`« [1.1], [22.g]. and [58f]: zoom operation is performed on the markings
` Claims 29 and 65: no “single continuous gesture with constant contact”
`
`« Claims 9, 32, and 69: no credible theory to add duration magnitude threshold
`
`Ground 1B (Ramos-Ramos-Ording-Hayward)
`
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`36
`
`
`
`
`
`
`
`
`Ground 1A: Limitations [1.1], [22.g]. and [58f]
`
`RROD’s “zoom operation” is performed on the Zlider markings.
`
`@ Claim 1 (and Claims 22 and 58)
`
`[1.1] perform a zoom operation on the at least portion of the contents of the first virtual display layer
`without performing the zoom operation on the plurality of markings in the second virtual display
`layer, where the zoom operation correlates with the movement of the markings in the second
`virtual display layer, and the at least portion of the second virtual display layer is at least partially
`translucent so that a result of the zoom operation on the at least portion of the contents of the
`first virtual display layer is visible through the at least portion of the second virtual display layer.
`
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`
`
`
`
`
`
`
`Ground 1A: Limitations [1.1], [22.g]. and [58f]
`
`Petitioner’s implied construction of “zoom operation” is overly narrow
`
`@ Petitioner’s Theory
`
`“The two-dimensional image magnification function applied to the image
`in the Ramos Video is not applied to the Vernier.”
`
`Pet., 69
`
`@ Petitioner’s Theory
`
`“Thus, any change in scale of the Vernier is not the “zooming” function
`
`applied to the content.”
`Pet., 70
`
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`
`
`
`
`
`
`
`
`Ground 1A: Limitations [1.1], [22.g]. and [58f]
`
`Vernier scales or zooms in lockstep with the content—it is the same “zoom operation”
`
`@ Vernier scaling during the zoom operation m Dr. Mahon Declaration
`
`“As shown above, as the user performs a zoom
`operation on the working area the zoom operation
`also occurs on the plurality of markings as seen in the
`video by the plurality of markings expanding—i.e.,
`appearing to get closer to the user—as can be seen
`when viewing the video.”
`
`EX2012, 91112 (cited in POR, 37; Sur-Reply, 15)
`
`The markings progressive expand as the user zooms in.
`
`EX1005, 0:20-0:23 (Ramos Video) (annotated)
`
`POR, 37 (citing EX1005, 0:20-0:23 (annotated)); Sur-Reply, 14-15
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`
`
`
`
`
`
`
`
`Ground 1A: Limitations [1.1], [22.g]. and [58f]
`
`Zooming is a change in the apparent distance from the observer
`
`@ Webster’s Dictionary (“zoom”)
`
`“3a: to focus a camera or microscope on an object
`using a zoom lens so that the object’s apparent
`distance from the observer changes—often used with
`in or out
`b: FOCUS, ZERO—used with in <trying to zoom in on
`the cause of these problems>"
`
`EX1009 (cited in POR, 38; Sur-Reply, 15)
`
`m Dr. Mahon Declaration
`
`“As shown above, as the user performs a zoom
`operation on the working area the zoom operation
`also occurs on the plurality of markings as seen in the
`video by the plurality of markings expanding—i.e.,
`appearing to get closer to the user—as can be seen
`when viewing the video.”
`
`EX2012, 91112 (cited in POR, 37; Sur-Reply, 15)
`
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`
`
`
`
`
`
`
`
`Ground 1A: Limitations [1.1], [22.g]. and [58f]
`
`Ramos uses “zoom” to refer to the manipulation of scale or
`adjusting the granularity of the parameter space
`
`@ Ramos Paper @ Ramos Paper
`
`“This paper introduces the notion of Zoom Sliding, or “In this paper, we propose and study a mechanism for
`Zliding, for fluid integrated manipulation of scale use with pressure sensitive input devices, called Zoom
`(zooming) via pressure input while parameter Sliding, or Zliding for short, in which users use the
`manipulation within that scale is achieved via x-y cursor pressure modality to fluidly and explicitly zoom or adjust
`movement (sliding).” the granularity of the parameter space, while sliding or
`EX1004, 1 (cited in POR, 5; Sur-Reply, 15) dragging the input device to perform high precision
`parameter manipulation within that zoomed parameter
`space.”
`EX1004, 1 (cited in POR, 54-55, Sur-Reply, 15)
`
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`
`
`
`
`
`
`
`
`'580 Patent Overview
`
`Prior Art Overview
`
`Ground 1A (Ramos-Ramos-Ording)
`
`* [1.h], [1.i], [22.d], and [58.c]: no semi-transparent second virtual display layer
`* [1.h], [22.d], and [58.c]: “no touch” is not an “aspect of touch”
`
`« [1.1], [22.g]., and [58f]: zoom operation is performed on the markings
`
`« Claims 29 and 65: no “single continuous gesture with constant contact”
`
`« Claims 9, 32, and 69: no credible theory to add duration magnitude threshold
`
`Ground 1B (Ramos-Ramos-Ording-Hayward)
`
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`42
`
`
`
`
`
`
`
`
`Ground 1A: Claims 29 and 65
`
`RROD does not disclose a “single continuous gesture with constant contact”
`
`@ Claim 29 (and Claim 65)
`
`29. The method of claim 23, wherein the touch and the another touch are detected to be part of a single
`continuous gesture with constant contact with the electronic device.
`
`65. The non-transitory computer readable storage medium of claim 58, wherein the one or more programs
`include instructions which, when executed by the electronic device, cause the electronic device to operate
`such that: the touch and the another touch are detected to be part of a single continuous gesture with
`constant contact with the electronic device.
`
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`
`
`
`
`
`
`
`
`Ground 1A: Claims 29 and 65
`
`The Petition identified two separate inputs:
`(1) the stylus hovering and (2) the stylus moving on touch screen
`
`“The RROD renders obvious [7]. APPLE-1003, 91112. The
`Ramos Video shows the user making a single
`continuous touch movement that includes the user
`moving a stylus within proximity of the touch screen
`(causing display of the Vernier on the working area),
`and continue moving on the touchscreen along the
`Zlider and during the continuous touch movement,
`both the markings are displayed and the zoom
`operation is performed, as discussed with respect to
`[1.h] and [1.j]- [1.k]. APPLE-1003, 91112; APPLE-1005,
`3:07-3:22”
`
`Pet., 80 (cited by reference at Pet., 90)
`
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`
`
`
`
`
`
`
`
`Ground 1A: Claims 29 and 65
`
`If hov