Apple Inc. (Petitioner)
`
`V.
`Smith Interface Technologies, LLC (Patent
`
`Owner)
`Petitioner Demonstratives
`
`Case No. IPR2024-01256
`U.S. Patent No. 11,740,727
`
`Before Hon. Jeffrey S. Smith, Miriam L. Quinn, And Sharon Fenick
`Administrative Patent Judges
`
`APPLE 1029
`
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE )
`F I S H Apple v. Smith Interface
`. 1 IPR2024-01256
`
`
`
`
`
`
`
`
`Table of Contents
`
`Overview of the Instituted Grounds
`
`3
`
`Background Info: The 727 Patent and Prior Art 5
`Issue 1: Ahn-Chaudhri Renders Progressively Blurring Obvious 9
`1A: Ahn and Chaudhri-842 Both Teach the Menu Following the User’s Finger 10
`1B: Motivation to Combine with Chaudhri-842 22
`Issue 2: Dependent Claims 30
`
`FISH.
`
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`
`
`
`
`
`
`
`
`Overview of the Instituted Grounds
`
`FISH.
`
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`
`
`
`
`
`
`
`Overview of the Instituted Grounds
`
`Ground Challenged Claims Basis for Rejection
`
`§103—Ahn (US 2008/0207188), Chaudhri
`
`1 1-6,10-14, 16, and 171 ¢ 15 175 2007/0150842)
`
`FISH‘ Pt |
`
`DEMONSTRATIVE EXHIBIT - NOT EVIDENCE
`
`
`
`
`
`
`
`
`Background Info: The 727 Patent and Prior Art
`
`FISH.
`
`EEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEE
`
`
`
`
`
`
`
`
`Exemplary Claim 1 of the 727 Patent
`
`1. A device, comprising:
`at least one non-transitory memory;
`a touch screen; and
`one or more processors in communication with the at least
`one non-transitory memory, and the touch screen,
`wherein the one or more processors execute instruc-
`tions in the at least one non-transitory memory, to cause
`the device to:
`display an object and at least one other object;
`detect at least part of a gesture on the touch screen; and
`during detection of at least a portion of the gesture
`before a completion thereof is detected, blur, based
`on a change in a distance magnitude of the gesture
`being detected on the touch screen to thereby move
`the object on the touch screen, at least a portion of
`the at least one other object that is not overlapped,
`such that a magnitude of the blur itself, and not a
`magnitude of an area of the touch screen that is
`blurred, 1s increased as a function of an increase in
`the distance magnitude.
`
`FISH.
`
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`
`
`
`
`
`
`
`Ahn
`
`FI1G. 11
`
`FIG. 4B A
`Yl R O 8 1219% Yl X O & 1219 l;/Start
`
`420a -_— Y
`
`40 : - s
`. S1100 ~— Display standby screen I
`420c Y
`
`300- > 420
`4206 S1110 —— Detect touch and drag operation
`
`of specific direction
`
`420e
`Y
`
`420f o
`Display item list matched
`
`S1120 to specific direction
`
`P i3
`End
`
`Clock widget is blurred when user
`menu 420 is opened
`
`EX1004, FIG. 11
`
`EX1004, FIGS. 3A and 4B
`
`FISH.
`
`APPLE-1004, FIGS. 3A, 4B, 11
`Pet., 8, 11
`
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`
`
`
`
`
`
`
`
`Chaudhri-842
`
`TN
`
`602
`
`While an electronic device is in a first user-
`interface state, detect progress towards
`satisfaction of a user input condition needed to
`transition to a second user-interface state
`
`While the device is in the first user-interface
`state, indicate progress towards satisfaction of
`the condition by transitioning an optical intensity
`of one or more user interface objects associated
`with the second user-interface state
`
`Transition the device to the second user-
`interface state if the condition is satisfied
`
`Figure 6
`
`FISH.
`
`[0006] Accordingly, there is a need for more efficient,
`user-friendly procedures for unlocking such devices, touch
`screens, and/or applications. More generally, there is a need
`for more efficient, user-friendly procedures for transitioning
`such devices, touch screens, and/or applications between
`user interface states (e.g., from a user interface state for a
`
`first application to a user interface state for a second appli-
`cation, between user interface states in the same application,
`or between locked and unlocked states). In addition, there is
`a need for sensory feedback to the user regarding progress
`towards satisfaction of a user input condition that is required
`for the transition to occur.
`
`[0048] The device 100 may have a plurality of user
`interface states. A user interface state is a state in which the
`device 100 responds in a predefined manner to user input. In
`some embodiments, the plurality of user interface states
`includes a user-interface lock state and a user-interface
`unlock state. In some embodiments, the plurality of user
`interface states includes states for a plurality of applications.
`
`In some embodiments, as the
`user transitions from a first application and to a second
`application, a user interface for the second application may
`fade in (i.e., increase in intensity) and a user interface for the
`first application may fade out (i.e., decrease in intensity).
`The fade in and fade out may occur smoothly over a
`pre-determined time interval, such as 0.2 s, 1s or 2s.
`
`APPLE-1005, FIG. 6, [0006], [0048], [0079]
`
`Pet., 14, 16
`
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`
`
`
`
`
`
`
`
`Ahn-Chaudhri Renders
`Progressively Blurring Obvious
`
`FISH.
`
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`
`
`
`
`
`
`
`Ahn and Chaudhri-842 Combination Teaches
`the Menu Following the User’s Finger
`
`FISH.
`
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`
`
`
`
`
`
`
`Ahn Teaches Opening Menu With Dragging Gesture
`
`FIG. 18
`
`730 740 750 760
`
`300
`
`APPLE-1004, FIGS. 18(a) and 7B
`
`F I S H Pet. 10, 28-30
`©
`
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`
`
`
`
`
`
`
`FISH.
`
`[0117] In the above-described implementations, if the
`touch and drag operation of a specific direction is performed
`in the touch input device 10, an identifier for notifying the
`user that an item list matched to the specific direction is
`displayed in the display 40 can be displayed in the display 40.
`The identifier may be graphical in form, such as an icon. For
`example, the identifier may be a specific icon for notifying a
`start position of the touch and drag operation.
`
`[0118] FIG. 18 illustrates an example of a screen for dis-
`playing a graphical display for notifying a start position of a
`touch and drag operation to a user. For example, referring to
`FIG. 18(a), if the user drags in a right direction after touching
`the screen, a graphical shadow 1800 is displayed in the dis-
`play 40, and the menu list 1500 shown in FIG. 15(5) can be
`displayed in the display 40. In this case, as the user drags the
`graphic shadow 1800, the menu list 1500 can be displayed
`along the dragged graphic 1800.
`
`[0119] Further, for example, referring to FIG. 18(b), if the
`user touch drags to the left, the menu list 1500 shown in FIG.
`15(b) can be displayed 1in the display 40. In this case, as the
`user may press the graphic 1800 with a finger, the graphic
`1800 gradually disappears in the display 40, and the menu list
`1500 can be displayed as emerging from the left side of the
`display 40 while the graphic 1800 disappears in the display
`40.
`
`Ahn’s Description of Menu Opening with Graphic Shadow
`
`APPLE-1004, [0117]-[0119]
`Pet., 28-29, 44
`
`12
`
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`
`
`
`
`
`
`
`
`Combination with Chaudhri-842 Includes Moving Graphics
`
`interface states. See EX1005, [0006], [0082], [0088]-[0089], [0108]. To help direct
`a user with respect to successfully performing the predefined gesture, Chaudhri 842
`teaches that the user interface may additionally mmclude “cues” to help the user know
`what gesture to utilize and whether they are correctly performing the gesture (e.g.,
`
`oraphics such as movable images that, when combined with the transition in optical
`
`intensity, help the user visualize the correct gesture and how to complete it
`
`successfully). See EX1005, Abstract, [0054], [0062], [0073].
`
`APPLE-1003, 52
`
`F I S H Pet., 20-21; Reply, 4-5, 19 13
`°
`
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`
`
`
`
`
`
`
`FISH.
`
`Chaudhri-842 Teaches Graphic Follows the User’s Finger
`
`Graphic following user’s finger
`during transition gesture
`
`Touch Screen 408
`
`Vi /
`/ vV
`Touch Screen 408
`402
`
`404
`
`3./
`
`Movement
`
`502 3 504
`Menu
`Button 410
`
`APPLE-1005, FIGS. 5A-5B
`Pet., 39-41; Reply, 5-7 14
`
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`
`
`
`
`
`
`
`
`Chaudhri-842 Teaches Graphic Follows the User’s Finger
`
`[0075] In FIG. 5B, the user is in the process of performing
`the gesture by moving her finger, which 1s in continuous
`contact with the touch screen 408, in the direction of
`movement 504. The unlock 1image 402 is dragged along the
`channel 404 as a result of the gesture. The channel 404
`reminds the user that the unlock gesture i1s a horizontal
`
`APPLE-1005, [0075]
`
`F I S H Pet., 19-22; Reply, 5-6 15
`e
`
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`
`
`
`
`
`
`
`Purpose of Ahn and Chaudhri-842 Graphics the Same
`
`Chaudhri-842
`
`[0054] In some embodiments, the locked device displays
`on the touch screen one or more visual cues of an unlock
`action that the user may perform to unlock the device (204).
`The visual cue(s) provide hints or reminders of the unlock
`action to the user. The visual cues may be textual. graphical
`or any combination thereof. In some embodiments, the
`
`Ahn
`
`[0117] In the above-described implementations, if the
`touch and drag operation of a specific direction 1s performed
`in the touch input device 10, an identifier for notifying the
`user that an item list matched to the specific direction is
`displayed in the display 40 can be displayed in the display 40.
`The 1dentifier may be graphical in form, such as an icon. For
`example, the identifier may be a specific icon for notifying a
`start position of the touch and drag operation.
`
`APPLE-1005, FIGS. 5A-5B
`
`F I S H Pet., 29, 40-41; Reply, 6 16
`°
`
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`
`
`
`
`
`
`
`Ahn-Chaudhri Menu Follows User’s Finger
`
`User touches near the right
`
`bt R St e R Menu matched to left touch
`
`on menu identifer and drags As the user drags the menu and drag direction is
`Ll taibid saites ol tiioeh identifier icon, the menu is displayed on right side of
`dragged onto the screen touch screen 10
`
`screen 10.
`
`Y R O 8 1219, 40 Y R O
`
`Menu identifier icon
`
`Amalgamation of APPLE-1004, FIGS. 3a and 13b
`
`FISH‘ Pet. 40
`
`17
`
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`
`
`
`
`
`
`
`
`Ahn and Chaudhri-842 Teachings
`Consistent with Broader Prior Art
`
`FISH.
`
`APPLE-1011, FIG. 21
`Reply, 8
`
`18
`
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`
`
`
`
`
`
`
`
`FISH.
`
`Ahn and Chaudhri-842 Teachings
`Consistent with Broader Prior Art
`
`Nan
`
`[0052] Note that method 200 and other methods described
`hereafter can be performed 1n real-time, such as while a
`gesture 1s being made and received. This permits, among
`other things, a user interface presented in response to a ges-
`ture to be presented prior to completion of the gesture. Fur-
`ther, the user interface can be presented progressively as the
`
`ging out the user interface from the edge as the gesture 1s
`
`gesture is received. This permits a user experience of drag_—
`
`performed with the user interface am)earing to “stick” to the]
`
`gesture (e.g., to a mouse point or person’s finger making the
`
`gesture).
`
`APPLE-1010, [0052]
`Reply, 8-9
`
`19
`
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`
`
`
`
`
`
`
`
`Petition Relied Upon Obviousness
`Based on Ahn and Chaudhri-842
`
`Rebuttal The Petition did not articulate a modification of Ahn with Chaudhri to move “graphic 1800
`
`with the gesture..” Sur-reply, 5
`
`to:
`
`drag operation 1s an ebject, as recited in [1.b]. EXI1003, [78]. With respect to
`graphical shadow 1800, Ahn describes that “as the user drags the graphic shadow
`
`1800, the menu list 1500 can be displayed along the dragged graphic 1800.”
`
`EX1004, [0118]. In other words, Ahn describes that the graphic 1800 moves (i.e.,
`
`1s “dragged”) in response to and along with a user’s touch and drag gesture on the
`
`touch screen. See EX1004, [0118]; EX1003, [78].
`
`Pet. 40
`FISH 20
`|
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`
`
`
`
`
`
`
`
`Petition Relied Upon Obviousness
`Based on Ahn and Chaudhri-842
`
`Rebuttal The Petition did not articulate a modification of Ahn with Chaudhri to move “graphic 1800
`
`with the gesture..” Sur-reply, 5
`to:
`
`Indeed, as described by Chaudhri 842, it was known to utilize “cues™ to help
`the user know what gesture to utilize and whether they are correctly performing the
`
`gesture (e.g., graphics such as movable images that help the user visualize the correct
`
`gesture and how to complete it successtully). See EX1005, Abstract, [0054], [0062],
`
`FISH‘ Pet. 40 21
`
`DEMONSTRATIVE EXHIBIT - NOT EVIDENCE
`
`
`
`
`
`
`
`
`
`Motivation to Combine with Chaudhri-842
`
`FISH.
`
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`
`
`
`
`
`
`
`Ahn’s Menu Blurs Underlying Ul Object
`
`FIG. JA FIG. 48
`
`Y R O B 1219 Yl X O A 1219
`
`420a
`
`420b
`
`420c
`300 -
`
`420d
`
`420e
`
`420f)
`
`Clock widget is blurred when user
`menu 420 is opened
`
`EX1004, FIGS. 3A and 4B
`
`APPLE-1004, FIGS. 3A, 4B
`Pet., 12, 36
`
`23
`
`FISH.
`DEMONSTRATIVE EXHIBIT - NOT EVIDENCE
`
`
`
`
`
`
`
`
`
`Chaudhri-842 Teaches Transition in Optical Intensity During
`Ul Transitions
`
`600 [0006‘]' Accordingly, theye is a nf:ed for more efficient,
`
`\‘ user-friendly procedures for unlocking such devices, touch
`screens, and/or applications. More generally. there is a need
`for more efficient, user-friendly procedures for transitioning
`such devices, touch screens, and/or applications between
`
`602 While an electronic device is in a first user-
`
`interface state, detect progress towards user interface states (e.g., from a user interface state for a
`satisfaction of a user input condition needed to first application to a user interface state for a second appli-
`transition to a second user-interface state cation, between user interface states in the same application,
`
`or between locked and unlocked states). In addition, there is
`a need for sensory feedback to the user regarding progress
`towards satisfaction of a user input condition that is required
`for the transition to occur.
`
`604 While the device is in the first user-interface
`
`\‘ state, indicate progress towards satisfaction of - -
`the condition by transitioning an optical intensity [0048] The device 100 may have' a plur a!lty O_f user
`of one or more user interface objects associated interface states. A user interface state is a state in which the
`
`with the second user-interface state device 100 responds in a predefined manner to user input. In
`some embodiments, the plurality of user interface states
`includes a user-interface lock state and a user-interface
`unlock state. In some embodiments, the plurality of user
`interface states includes states for a plurality of applications.
`
`. Transition the device to the second user-
`
`interface state if the condition is satisfied In some embodiments, as the
`
`user transitions from a first application and to a second
`application, a user interface for the second application may
`fade in (i.e., increase in intensity) and a user interface for the
`first application may fade out (i.e., decrease in intensity).
`Figure 6 The fade in and fade out may occur smoothly over a
`pre-determined time interval, such as 0.2 s, 1s or 2s.
`
`APPLE-1005, FIG. 6, [0006], [0048], [0079]
`
`FISH‘ Pet., 14, 16
`
`24
`
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`
`
`
`
`
`
`
`
`Chaudhri-842 Teaches Transition in Optical Intensity
`As Function of User Gesture
`
`75. Chaudhr1 842 describes that the optical intensity, and thus “the
`indication of progress towards completion of the user input condition[,] 1s a function
`of the user’s satisfaction of the condition.” EXI1005, [0087]. “If the user mnput
`condition includes a predefined gesture then the indication of progress of the gesture
`may be defined in terms of how much of the gesture 1s completed and how much of
`
`the gesture 1s remaining.” EX1005, [0088]. As the user input condition (e.g.. a
`
`gesture) progresses, the optical intensity can correspondingly vary in magnitude
`
`based on the magnitude of the progress toward gesture completion according to both
`
`linear and non-linear functions, such as those illustrated in FIGS. 8A-8C. See
`
`EX1005, [0087]-[0088].
`
`APPLE-1003, {[75
`
`FISH‘ Pet., 38
`
`25
`
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`
`
`
`
`
`
`
`
`Chaudhri-842 Teaches Transition in Optical Intensity
`As Function of User Gesture
`
`Figure 8A
`
`Optical Intensity
`
`o
`X
`
`Completion 100%
`
`APPLE-1005, FIG. 8A
`
`FISH . Pet., 38 26
`
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`
`
`
`
`
`
`
`Petition’s Unrebutted Motivation
`
`55. Fourth, a POSITA would have known that varying optical intensity for
`a user interface object (e.g., the blurring of Ahn’s clock widget) as a function of the
`magnitude of the user’s touch and drag gesture would have been one of a limited
`number of ways of implementing the blurring effect that’s explicitly shown in FIGS.
`4B, 10, and 13B of Ahn. Either a POSITA could have implemented the blurring
`
`effect in a binary fashion (e.g., toggling instantaneously between unblurred and
`
`blurred) or progressively varied the blur as a function of the user’s gesture, as taught
`
`by Chaudhr1 *842.
`
`EX1003, 55
`Pet., 23
`
`FISH.
`
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`
`
`
`
`
`
`
`FISH.
`
`Motivations Taught by Chaudhri-842
`
`transitioning between user interface states.” EXI1005, [0082]. Based on these
`teachings a POSITA would have understood and found obvious that, as a user is
`performing a predefined gesture, for example, the changes in optical intensity
`provide the user a visual indication that they are performing a particular operation
`(e.g., the optical intensity changes help the user see more clearly that the user
`interface state 1s changing), what the operation is doing (e.g., the optical intensity
`changes help the user visualize what 1s changing as a result of the gesture), and how
`far the user 1s in completing the gesture that will effectuate the change in user
`interface states. See EX1005, [0006], [0082], [0088]-[0089], [0108]. To help direct
`a user with respect to successfully performing the predefined gesture, Chaudhri 842
`teaches that the user interface may additionally include “cues” to help the user know
`what gesture to utilize and whether they are correctly performing the gesture (e.g.,
`graphics such as movable images that, when combined with the transition in optical
`intensity, help the user visualize the correct gesture and how to complete it
`
`successfully). See EX1005, Abstract, [0054], [0062], [0073].
`
`EX1003, 952
`Pet., 20-21
`
`28
`
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`
`
`
`
`
`
`
`
`Motivations Taught by Chaudhri-842
`
`[0054] In some embodiments, the locked device displays
`on the touch screen one or more visual cues of an unlock
`action that the user may perform to unlock the device (204).
`The visual cue(s) provide hints or reminders of the unlock
`action to the user. The visual cues may be textual. graphical
`or any combination thereof. In some embodiments, the
`visual cues are displayed upon particular events occurring
`while the device is locked. The particular events that trigger
`display of the visual cues may include an incoming call,
`
`[0062] While the device is locked, the device may display
`one or more visual cues of the unlock action, as described
`above. In some embodiments, the device may also display,
`along with the visual cues, an unlock image. The unlock
`image is a graphical, interactive user-interface object with
`which the user interacts in order to unlock the device. In
`other words, the unlock action is performed with respect to
`the unlock image. In some embodiments. performing the
`unlock action with respect to the image includes dragging
`the unlock image in a predefined manner, which moves the
`unlock image across the touch screen. In some embodi-
`ments, if the unlock action 1s not completed, the GUI display
`can show reverse progress towards the locked state by
`gradually returning the unlock image to its position in the
`locked state
`
`[0085] The optical intensity of a user-interface object, as
`used herein, is the object’s degree of visual materialization.
`The optical intensity may be measured along a scale
`between a predefined minimum and a predefined maximum.
`In some embodiments, the optical intensity may be mea-
`sured along a logarithmic scale. In some embodiments, the
`optical intensity may be perceived by users as a transparency
`effect (or lack thereof) applied to the user-interface object. In
`some embodiments, the minimum optical intensity means
`that the object is not displayed at all (i.e., the object is not
`perceptible to the user), and the maximum optical intensity
`means that the object is displayed without any transparency
`effect (i.e., the object has completely materialized visually
`and is perceptible to the user). In some other embodiments,
`the optical intensity may be the visual differentiation
`between the user-interface object and the background, based
`on color, hue, color saturation, brightness, contrast, trans-
`parency. and any combination thereof.
`
`FISH.
`
`APPLE-1005, [0054], [0062], [0085]
`Pet., 21
`
`29
`
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`
`
`
`
`
`
`
`
`Dependent Claims
`
`FISH.
`
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`
`
`
`
`
`
`
`Dependent Claim 2
`
`2. The device of claim 1, wherein the device 1s configured
`such that:
`
`the object includes a graphic object;
`
`the object is displayed closer to a top edge of the touch
`screen than a bottom edge of the touch screen;
`
`an aspect of the object other than a location thereof 1s
`modified, based on the change in the distance magni-
`tude of the gesture being detected on the touch screen;
`and
`
`the at least one other object includes a background image.
`
`31
`
`FISH.
`
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`
`
`
`
`
`
`
`
`Menu Position is Variable
`
`played.
`
`[0104] A position in which the 1tem list 1s displayed 1n the
`display 40 can be preset. For example, referring to FIGS. 13
`to 15, a menu list matched to a left direction 1s set to be
`displayed at the right side of the display 40 (FIG. 13(4)), a
`menu list matched to a upward direction 1s set to be displayed
`at a lower end of the display 40 (FIG. 14(4)), a menu list
`matched to a right direction 1s set to be displayed at the left
`side of the display 40 (FIG. 15(5)). The memory 20 can store
`information about a position in which the item list 1s dis-
`
`APPLE-1004, [0104]
`Pet., 47
`
`32
`
`FISH.
`
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`
`
`
`
`
`
`
`
`FISH.
`
`Hisatomi Confirms Menus at All Sides
`
`[FIG. 6]
`
`B classification |
`
`]
`\ | menu |
`D'l
`lassifi o classifica
`classifica - .
`‘ . ag tion
`tion
`menu
`menu
`
`= N
`
`s NC—— ¥ v b B
`
`—
`
`| C classification
`
`menu
`
`APPLE-1011, FIG. 6
`Pet., 47-48
`
`33
`
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`
`
`
`
`
`
`
`
`Ahn’s Background Image
`
`FIG. 3A
`
`Y XX OH A
`
`40
`
`Background image of
`trees against a sky
`
`APPLE-1004, FIG. 3A
`
`FISH‘ Pet., 52 34
`
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`
`
`
`
`
`
`
`FISH.
`
`Blurring Background to Differentiate
`
`[0085] The optical intensity of a user-interface object, as
`used herein, 1s the object’s degree of visual materialization.
`The optical intensity may be measured along a scale
`between a predefined minimum and a predefined maximum.
`In some embodiments, the optical intensity may be mea-
`
`sured along a logarithmic scale. In some embodiments, the
`
`optical intensity may be perceived by users as a transparency
`effect (or lack thereof) applied to the user-interface object. In
`some embodiments, the minimum optical intensity means
`that the object 1s not displayed at all (i.e., the object 1s not
`perceptible to the user), and the maximum optical intensity
`means that the object 1s displayed without any transparency
`effect (i.e., the object has completely materialized visually
`and 1s perceptible to the user). In some other embodiments,
`the optical intensity may be the visual differentiation
`between the user-interface object and the background, based
`on color, hue, color saturation, brightness, contrast, trans-
`parency. and any combination thereof.
`
`APPLE-1005, [0085]
`
`35
`
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`
`
`
`
`
`
`
`
`FISH.
`
`Blurring Background to Differentiate
`
`99. While Ahn only specifically illustrates that the clock widget being
`blurred, a POSITA would have at least found it obvious to also blur the rest of
`standby screen 300. Indeed, Chaudhri *842 teaches that “the optical intensity may
`
`be the visual differentiation between the user-interface object and the backeround .
`
`7 EX1005, [0085]. Among the various types of user-interface objects
`contemplated by Chaudhri *842 are, “without limitation, text, images, icons, soft
`
`keys (or ‘virtual button’), pull-down menus, radio buttons, check boxes, selectable
`
`lists, and so forth.” EX1005, [0060]. Accordingly, Chaudhri ’842 explicitly
`describes differentiating between a menu and “the background”—not simply a
`portion thereof—displayed on the touch screen during its user interface state
`
`transition process.
`
`APPLE-1003, 199
`Pet., 53
`
`36
`
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`
`
`
`
`
`
`
`
`Dependent Claim 3
`
`3. The device of claim 2, wherein the device 1s configured
`such that the object 1s
`displayed closer to a right edge of the touch screen than
`a left edge of the touch screen.
`
`37
`
`FISH.
`
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`
`
`
`
`
`
`
`
`Menu and Identifier Position is Variable
`
`[0104] A position in which the 1tem list 1s displayed 1n the
`display 40 can be preset. For example, referring to FIGS. 13
`to 15, a menu list matched to a left direction 1s set to be
`displayed at the right side of the display 40 (FIG. 13(4)), a
`menu list matched to a upward direction 1s set to be displayed
`at a lower end of the display 40 (FIG. 14(4)), a menu list
`matched to a right direction 1s set to be displayed at the left
`side of the display 40 (FIG. 15(5)). The memory 20 can store
`information about a position in which the item list 1s dis-
`played.
`
`APPLE-1004, [0104]
`
`F I S H Pet., 47 38
`L5
`
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`
`
`
`
`
`
`
`FISH.
`
`Menu and Identifier Position is Variable
`
`106. Furthermore, as discussed with respect to [2.b], supra, it was well
`known before the Critical Date for edge menus to be associated with any and all
`edges of the display. See, e.g., EX1006, [0113] (describing “support[ing] the same
`number of bezel menu slots on all four edges of the screen™), [0114] (teaching that
`“the left and bottom edges, for a right-handed individual, may be more likely to be
`swiped by accident, and may be left without bezel menus if desired,” suggesting that
`the right and top edges may still be associated with bezel menus); EX1011, FIG. 6,
`[0019]-[0023] (1llustrating and describing pull-out menus associated with all found
`edges of the display). Accordingly, a POSITA would have recognized that the object
`[being] displayed closer to a right edge of the touch screen than a left edge of the
`
`touch screen was simple rearrangement of elements (e.g., the position of the menus
`
`disclosed in Ahn and the identifier icons associated with them) and an obvious
`
`design choice.
`
`APPLE-1003, {106
`Pet., 57-58
`
`39
`
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`
`
`
`
`
`
`
`
`FISH.
`
`Dependent Claims 5 and 17
`
`5. The device of claim 2, wherein the device is configured
`such that, during at least a part of at least an initial portion
`of the change in the distance magnitude, an entirety of the
`object 1s blurred.
`
`17. The device of claim 2, wherein the device is config-
`ured such that:
`
`the at least part of the gesture is detected on the touch
`screen on the object;
`
`during at least a part of an initial change in the distance
`magnitude, an entirety of the object is blurred; and
`
`the aspect of the object other than the location thereof 1s
`modified while the object continues to be displayed,
`based on the change in the distance magnitude of the
`gesture being detected on the touch screen on the
`object.
`
`40
`
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`
`
`
`
`
`
`
`
`Ahn Teaches Graphic Shadow Disappearing
`
`[0119] Further, for example, referring to FIG. 18(4), if the
`user touch drags to the left, the menu list 1500 shown in FIG.
`15(b) can be displayed in the display 40. In this case, as the
`user may press the graphic 1800 with a finger, the graphic
`
`1800 gradually disappears in the display 40, and the menu list
`1500 can be displayed as emergmg from the left side of the
`
`display 40 while the graphic 1800 disappears in the display
`40,
`
`FISH.
`
`APPLE-1004, [0119]
`
`Pet., 17-18
`41
`
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`
`
`
`
`
`
`
`
`Chaudhri-842 Teaches Fading Out First Ul Elements and
`Fading In Second Ul Elements
`
`In some embodiments, as the
`user transitions from a first application and to a second
`application, a user interface for the second application may
`fade 1in (1.e., increase 1n intensity) and a vuser interface for the
`first application may fade out (i.e., decrease in mtensity).
`The fade in and fade out may occur smoothly over a
`pre-determined time interval, such as 0.2 s, 1Is or 2s. The
`
`APPLE-1005, [0079]
`
`FISH‘ Pet., 60
`
`42
`
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`
`
`
`
`
`
`
`
`Petition Turned to Both Ahn and Chaudhri-842
`
`from the left side of the display 40 while the graphic 1800 disappears 1n the display
`40 and the disappearance includes the “fading out” taught in Chaudhri *842, a
`POSITA would have understood and found obvious that the mobile device of the
`Ahn-Chaudhri would have been configured such that, during at least a part of at
`least an initial portion of the change in the distance magnitude, an entirety of the
`object is blurred (e.g., the entire menu 1dentifier icon fades out as the user drags the
`
`icon to open the menu), as recited 1in [5]. EX1003, [113].
`
`Thus, in those implementations where the menu 1s “displayed as emerging
`
`FISH.
`
`Pet., 61
`
`43
`
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`
`
`
`
`
`
`
`
`Dependent Claim 6
`
`6. The device of claim 2, wherein the device 1s configured
`such that, during at least a part at least an initial portion of
`the change 1n the distance magnitude, an entirety of a virtual
`display layer in which the object 1s displayed, 1s blurred.
`
`FISH.
`
`44
`
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`
`
`
`
`
`
`
`
`Ahn Teaches Graphic Shadow Disappearing
`
`[0119] Further, for example, referring to FIG. 18(4), if the
`user touch drags to the left, the menu list 1500 shown in FIG.
`15(b) can be displayed in the display 40. In this case, as the
`user may press the graphic 1800 with a finger, the graphic
`
`1800 gradually disappears in the display 40, and the menu list
`1500 can be displayed as emergmg from the left side of the
`
`display 40 while the graphic 1800 disappears in the display
`40,
`
`FISH.
`
`APPLE-1004, [0119]
`
`Pet., 17-18
`45
`
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`
`
`
`
`
`
`
`
`FISH.
`
`Blurring Virtual Layer Was Well-Known
`
`f -
`|
`L
`
`relevant
`
`2D SDOF scene 3D intermediate rendered image
`
`Figure 4. The photo-realistic camera and 2D SDOF.
`
`urelevant focus plane
`
`Il
`/
`]
`
`Oe
`
`SCCIC
`
`APPLE-1008, FIG. 4
`Pet., 54
`
`46
`
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`
`
`
`
`
`
`
`
`FISH.
`
`Blurring Virtual Layer Was Well-Known
`
`3.2. Relevance and Blurring
`
`Independently of the spatial arrangement, the blur level of
`each object 1s determined. This 1s done 1n two steps: First,
`each object 1s assigned a relevance value r by the relevance
`function rel. The value of r is in the interval [0; 1], where 1
`means the object 1s maximally relevant, and 0 means the ob-
`ject 1s completely wrrelevant. This relevance value 1s trans-
`lated into a blur value b through the blur function blur.
`
`The relevance function 1s application-specific and thus
`can be very different between applications (see Sect. 5.2 for
`examples). The blur function can theoretically also take on
`any shape. but we have found the function depicted in Fig. 3
`to be sufficient for our current uses. The user can specify
`the threshold value ¢, the step height /2, and the maximum
`blur diameter b,,,.,. The gradient ¢ 1s then calculated by the
`
`application.
`
`APPLE-1008, 3
`Pet., 65
`
`47
`
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`
`
`
`
`
`
`
`
`FISH.
`
`Blurring Virtual Layer Was Well-Known
`
`120. As discussed with respect to Element [2.d], supra, a POSITA would
`have understood and found obvious that the relevance of any object or set of objects
`(and thus the extent to which each object or set of objects 1s blurred or displayed
`sharply in focus) was a known design choice, and was known to be settable by the
`system in accordance with a designers programming or even by the user. See
`EX1008, 3-4. This 1s illustrated in the following figure, in which the red objects are
`assigned to the layer on which a user is currently focused, and the light blue objects
`are assigned to “lower” or “higher” layers of semantic depth from the focus layer,
`and are therefore assigned a lower relevance value based on their semantic distance
`
`from the focus layer and blurred based on this lower relevance value. See EX1008,
`
`-4.
`
`I
`
`APPLE-1003, 9120
`Pet., 65
`
`48
`
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`
`
`
`
`
`
`
`
`Dependent Claims 11 and 12
`
`11. The device of claim 2, wherein the device 1s config-
`
`ured such that the aspect of the object includes a size of the
`object.
`
`12. The device of claim 2, wherein the device 1s config-
`ured such that the aspect of the object includes a size of the
`object, and another size of the at least one other object 1s not
`
`modified in addition to the blur, when the size of the object
`is modified.
`
`FISH.
`
`49
`
`DEMONSTRATIVE EXHIBIT — NOT EVIDENCE
`
`
`
`
`
`
`
`
`
`Ahn Teaches Graphic Shadow Disappearing
`
`[0119] Further, for example, referring to FIG. 18(4), if the
`user touch drags to the left, the menu list 1500 shown in FIG.
`15(b)