IN THE UNITED STATES PATENT AND TRADEMARK OFFICE
`
`Attorney Docket No. 50095-0175IP1
`
`In re Patent of: Michael S. Smith
`11,740,727
`U.S. Patent No.:
`Issue Date:
`Aug. 29, 2023
`Appl. Serial No.: 17/206,107
`Mar. 18, 2021
`Filing Date:
`Title:
`DEVICES, METHODS, AND GRAPHICAL USER
`INTERFACES FOR MANIPULATING USER INTERFACE
`OBJECTS WITH VISUAL AND/OR HAPTIC FEEDBACK
`
`DECLARATION OF DR. LOREN TERVEEN
`
`1
`
`APPLE 1003
`
`

`

`Attorney Docket No. 50095-0175IP1
`IPR of U.S. Patent No. 11,740,727
`TABLE OF CONTENTS
`
`
`
`INTRODUCTION ........................................................................................... 3
`I.
`QUALIFICATIONS ........................................................................................ 3
`II.
`III. LEGAL PRINCIPLES ..................................................................................... 6
`A.
`Person of Ordinary Skill in the Art ....................................................... 6
`B.
`Obviousness ........................................................................................... 6
`C.
`Claim Interpretation .............................................................................. 8
`IV. MATERIALS CONSIDERED ........................................................................ 9
`V.
`THE ’727 PATENT ....................................................................................... 11
`A.
`Pertinent Disclosure ............................................................................ 11
`B. Prosecution History ................................................................................... 14
`VI. PERSON OF ORDINARY SKILL IN THE ART ........................................ 16
`A. A POSITA’s Ordinary Level of Skill .................................................. 16
`VII. OVERVIEW OF CONCLUSIONS FORMED ............................................. 16
`VIII. OVERVIEW OF REFERENCES AND COMBINATIONS ........................ 17
`A. Ahn (EX1004) ..................................................................................... 17
`B.
`Chaudhri ’842 (EX1005) ..................................................................... 23
`C.
`The Combinations, Reasons to Combine, & Expectation of
`Success ................................................................................................ 29
`IX. GROUND 1: AHN AND CHAUDHRI ’842, RENDER CLAIMS 1-6,
`10-14, 16, and 17 OBVIOUS ........................................................................ 34
`A.
`Claim Element Analysis ...................................................................... 34
`CONCLUSION .............................................................................................. 86
`
`X.
`
`2
`
`
`

`

`Attorney Docket No. 50095-0175IP1
`IPR of U.S. Patent No. 11,740,727
`
`
`I, Loren Terveen, PhD, declare as follows:
`
`I.
`
`INTRODUCTION
`
`1.
`
`I have been retained as a technical expert by counsel on behalf of Apple
`
`Inc. (“Apple”/“Petitioner”). I understand that Apple is requesting that the Patent
`
`Trial and Appeal Board institute an inter partes review (“IPR”) proceeding with
`
`respect to U.S. Patent No. 11,740,727 (“the ’727 patent”) (EX1001).
`
`2.
`
`I have been asked to provide my independent analysis of the ’727 patent
`
`in light of certain prior art publications, and I have done so based on my personal
`
`knowledge and experience.
`
`3.
`
`I am not, and never have been, an employee of Apple. I received no
`
`compensation for this Declaration beyond my normal hourly compensation based on
`
`my time actually spent analyzing the ’727 patent, the prior art publications cited
`
`below, and various issues related thereto. I will not receive any added compensation
`
`based on the outcome of any IPR or other proceeding involving the ’727 patent.
`
`II. QUALIFICATIONS
`
`4.
`
`In formulating my opinions, I have relied on my knowledge, training,
`
`and experience in the relevant field. My education and experience are described
`
`more fully in the attached curriculum vitae (Appendix A). For ease of reference, I
`
`have highlighted certain information below.
`
`3
`
`
`

`

`Attorney Docket No. 50095-0175IP1
`IPR of U.S. Patent No. 11,740,727
`I received a B.A. in Computer Science, Mathematics, and History from
`
`5.
`
`the University of South Dakota in 1984. I received an M.S. in Computer Sciences
`
`from the University of Texas in 1988. I further received a Ph.D. in Computer
`
`Sciences from the University of Texas in 1991.
`
`6.
`
`I am a Distinguished McKnight University Professor at the University
`
`of Minnesota in the Department of Computer Science and Engineering, and I have
`
`substantial experience with human computer interaction and user interfaces,
`
`including well before August 5, 2011, the date I have been informed is the alleged
`
`invention date of the ’727 patent at issue in this matter. I was a Member of Technical
`
`Staff at AT&T Bell Laboratories from 1991 to 1996, and then a Principal Member
`
`of Staff at AT&T Labs – Research from 1996 to 2002. I was honored by the
`
`Association for Computing Machinery (ACM) as a Distinguished Scientist in 2009
`
`and served as the president of ACM’s Special Interest Group on Computer-Human
`
`Interaction (ACM SIGCHI) from 2015-2018, was elected to the ACM SIGCHI
`
`Academy in 2019, and received the SIGCHI Lifetime Service Award in 2023.
`
`7.
`
`I have taught numerous courses at the University of Minnesota related
`
`to human-computer interaction and user interface design. Additionally, I have
`
`published over 150 peer-reviewed articles, a book, contributed chapters in six
`
`additional books, and have given over 50 presentations all across the United States
`
`4
`
`
`

`

`Attorney Docket No. 50095-0175IP1
`IPR of U.S. Patent No. 11,740,727
`and internationally. My published works revolve around the fields of human-
`
`computer interaction, user interface design, and social computing.
`
`8.
`
`I am also a named inventor on ten U.S. patents related to user interfaces,
`
`interactive data analysis, and methods for selecting and displaying information to
`
`users. My publications and patents have addressed many aspects of searching for
`
`information on the World Wide Web, including algorithms to gather information
`
`from the World Wide Web, additional algorithms to parse and analyze the HTML
`
`content of web pages, data structures to represent this information, and user
`
`interfaces that allowed people to access the information my systems collected. Much
`
`of my research also involved graphical user interface techniques, for example:
`
`enabling users to navigate large information structures (including the World Wide
`
`Web), specify multimodal interfaces including for interactive voice systems, interact
`
`with online maps, and tailor the performance of recommender systems.
`
`9.
`
`I have served as Chair of the leading conferences in Human-Computer
`
`Interaction, served on four different editorial boards, and have sat on well over 20
`
`program committees, including well over a dozen times on the program committee
`
`for the ACM Conference on Human Factors in Computing Systems (the CHI
`
`Conference), the leading international conference in Human-Computer Interaction.
`
`5
`
`
`

`

`III. LEGAL PRINCIPLES
`
`Attorney Docket No. 50095-0175IP1
`IPR of U.S. Patent No. 11,740,727
`
`10.
`
`In forming my analysis and conclusions expressed in this Declaration,
`
`I have applied the legal principles described in the following paragraphs, which were
`
`provided to me by counsel for the Petitioner.
`
`A.
`11.
`
`Person of Ordinary Skill in the Art
`I understand that the factors considered in determining the ordinary
`
`level of skill in a field of art include: the level of education and experience of persons
`
`working in the field; the types of problems encountered in the field; the teachings of
`
`the prior art regarding solutions to such problems; and the sophistication of the
`
`technology at the time of the alleged invention. I understand that a person of
`
`ordinary skill in the art (“POSITA”) is not a specific real individual, but rather is a
`
`hypothetical individual having the qualities reflected by the factors above and
`
`knowledge of all relevant prior art references, including the references I cite below.
`
`B.
`12.
`
`Obviousness
`I have been informed that a patent claim is invalid as “obvious” in light
`
`of one or more prior art references if it would have been obvious to a POSITA at the
`
`time of the alleged invention, taking into account (1) the scope and content of the
`
`prior art, (2) the differences between the prior art and the claims, (3) the level of
`
`ordinary skill in the art, and (4) any so called “secondary considerations” of non-
`
`obviousness, which include: (i) “long-felt but unresolved need” for the claimed
`
`6
`
`
`

`

`Attorney Docket No. 50095-0175IP1
`IPR of U.S. Patent No. 11,740,727
`invention, (ii) commercial success attributable to the claimed invention, (iii)
`
`unexpected results of the claimed invention, and (iv) “copying” of the claimed
`
`invention by others.
`
`13. While I do not know the exact date that the alleged invention of the
`
`’727 patent was made, I do know that the ’727 patent’s earliest claimed priority date
`
`is August 5, 2011. For purposes of my obviousness analysis, I have applied a date
`
`of August 5, 2011 as the date of the alleged invention (“Critical Date”), although in
`
`many cases the same analysis would hold true even if the date of the alleged
`
`invention were earlier or later.1
`
`14.
`
`I have been informed that a claim can be obvious in light of a single
`
`prior art reference or multiple prior art references. To be obvious in light of a single
`
`prior art reference or multiple prior art references, there must be a reason that would
`
`have prompted a POSITA to modify or supplement the single prior art reference, or
`
`to combine two or more references, in a manner that provides the elements of the
`
`claimed invention. This reason may come from a teaching, suggestion, or
`
`motivation to combine, or may come from the reference(s) themselves, the
`
`knowledge or “common sense” of a POSITA, or from the nature of the problem to
`
`
`1 I am not a lawyer so I did not offer an opinion on whether the ’727 patent is entitled
`
`to its earliest claimed priority date.
`
`7
`
`
`

`

`Attorney Docket No. 50095-0175IP1
`IPR of U.S. Patent No. 11,740,727
`be solved, and this reason may be explicit or implicit from the prior art as a whole.
`
`I have been informed that, under the law, the combination of familiar elements
`
`according to known methods is likely to be obvious when it does no more than yield
`
`predictable results. I also understand it is improper to rely on hindsight in making
`
`the obviousness determination.
`
`15.
`
`I understand that an obviousness determination also requires that a
`
`POSITA would have had a reasonable expectation of success. This concept has been
`
`explained to me as relating to the relative likelihood or predictability—from the
`
`perspective of a POSITA—of successfully modifying the prior art in a manner that
`
`would meet the claimed limitations of the patent being challenged (here the ’727
`
`patent). I understand that the expectation of success need only be “reasonable” and,
`
`thus, does not require the absolute certainty gleaned from physically creating the
`
`proposed prior art modification.
`
`C.
`16.
`
`Claim Interpretation
`I understand that, for purposes of my analysis in this inter partes review
`
`proceeding, the terms appearing in the patent claims should be interpreted according
`
`to their “ordinary and customary meaning.” In determining the ordinary and
`
`customary meaning, I understand that the words of a claim are first given their plain
`
`meaning that those words would have had to a person of ordinary skill in the art at
`
`the time of the alleged invention. I also understand that the structure of the claims,
`
`8
`
`
`

`

`Attorney Docket No. 50095-0175IP1
`IPR of U.S. Patent No. 11,740,727
`the specification and file history may be used to better construe a claim. Moreover,
`
`I understand that even treatises and dictionaries may be used, albeit under limited
`
`circumstances, to determine the meaning attributed by a person of ordinary skill in
`
`the art to a claim term.
`
`
`
`IV. MATERIALS CONSIDERED
`
`17. My analysis in this Declaration is based on my knowledge and
`
`experience. Based on my above-described qualifications in Section II, the Patent
`
`Trial and Appeal Board should consider me to be an expert in the field. Also, based
`
`on my experiences, I understand and know of the capabilities of persons of ordinary
`
`skill in this field.
`
`18. As part of my independent analysis for this Declaration, I have
`
`considered the following: the ’727 patent (EX1001) and its prosecution history
`
`(EX1002); the background knowledge/technologies that were commonly known to
`
`persons of ordinary skill; my own knowledge and experience gained from my work
`
`in the field; my experience in teaching and advising others in this field; and my
`
`experience working with others involved in this field. In addition, I have analyzed
`
`the following publications and materials:
`
`EX1004
`
`U.S. Appl. Pub. No. 2008/0207188 to Hye Sang Ahn et al.
`(“Ahn”)
`
`9
`
`
`

`

`Attorney Docket No. 50095-0175IP1
`IPR of U.S. Patent No. 11,740,727
`U.S. Appl. Pub. No. 2007/0150842 to Imran Chaudhri et al.
`(“Chaudhri ’842”)
`
`U.S. Appl. Pub. No. 2011/0209093 to Kenneth P. Hinckley and
`Koji Yatani (“Hinkley”)
`
`U.S. Appl. Pub. No. 2006/0015818 to Imran A. Chaudhri et al.
`(“Chaudhri ’818”)
`
`Robert Kosara et al., “Semantic Depth of Field,” IEEE
`Symposium on Information Visualization 2001, pp. 97-104
`(2001)
`
`EX1005
`
`EX1006
`
`EX1007
`
`EX1008
`
`EX1009
`
`NEW OXFORD AMERICAN DICTIONARY (3rd Ed. 2010)
`
`EX1011
`
`Certified translation of JP Published Patent Application No. 2002-
`55750 (“Hisatomi”), published February 20, 2002
`
`EX1013
`
`U.S. Patent No. 10,642,413 to Michael S Smith
`
`U.S. Publication No. 2010/0066763 to Francis MacDougall and
`Evan Hildreth (“MacDougall”)
`
`EX1014
`
`
`
`19. Note that for my citations to patent literature, I’ve used the column/line
`
`numbers or paragraph numbers. Note also that, unless I’ve indicated otherwise, all
`
`emphasis (bold/italics/underline) in any quoted text are ones I have added.
`
`20. Although I cite and quote to selected portions of various references in
`
`this Declaration, the reader should understand that these citations are representative
`
`examples. A POSITA would have viewed each reference in its entirety and in
`
`combination with other references. Accordingly, I intend the references identified
`
`in this Declaration to be viewed as incorporated in their entireties.
`
`
`
`10
`
`
`

`

`V. THE ’727 PATENT
`
`Attorney Docket No. 50095-0175IP1
`IPR of U.S. Patent No. 11,740,727
`
`A.
`Pertinent Disclosure
`21. The ’727 patent “generally relate[s] to touch screens” and describes
`
`embodiments where “a user experience [with a touch screen device] may be altered”
`
`based on a “touch signal.” EX1001, 1:40-41, 14:11-15:44, Figure 3. The claims of
`
`the ’727 Patent generally relate to a hodge podge of touch screen gestures (e.g., a
`
`gesture that includes a “distance magnitude”) and display functions on the user
`
`interface associated with those gestures (e.g., blurring or moving displayed objects).
`
`See, e.g., EX1001, 102:31-50 (Claim 1). However, as discussed below, the support
`
`for these claimed gestures and claimed user interface functions is scattered
`
`throughout the specification of the ’727 Patent, often without explanation about any
`
`relationship between the disparate teachings or their utilization together in a single
`
`embodiment. Nonetheless, as I will describe below, the claimed gestures and user
`
`interface functions were well-known well before the ’727 Patent.
`
`22.
`
`In one part of the specification, the ’727 Patent describes a method that
`
`involves performing operations that correspond to identified gestures. See EX1001,
`
`55:65-56:51. “If it is determined that a touch event has occurred, then the initial
`
`touch event attributes are determined,” and “the gesture is identified from gesture
`
`input.” EX1001, 56:8-12. According to the ’727 Patent, “gesture input refers to the
`
`collection of inputs, flags, and signals by which a gesture is identified and
`
`11
`
`
`

`

`Attorney Docket No. 50095-0175IP1
`IPR of U.S. Patent No. 11,740,727
`parameterized,” and is thus broadly defined. EX1001, 56:13-15. “For example, in
`
`one embodiment, a two-finger pinch gesture may be identified by two contact points
`
`combined with their motion with respect to each other, and parameterized by the
`
`extent of the motion.” EX1001, 56:15-19. “Once the gesture has been identified,
`
`the corresponding operation is performed.” EX1001, 56:50-51
`
`
`
`
`
`EX1001, FIG. 3 (left), FIG. 18 (right)
`
`23. The ’727 Patent describes that “[p]attern-based gestures may be simple
`
`to perform and easy for a user to remember.” EX1001, 65:15-17. One such gesture
`
`is a “pinch slide gesture.” See EX1001, 65:62-66:5. The pinch slide “gesture may
`
`be performed by pinching the device, then sliding the pinch across the interaction
`
`surfaces while maintaining contact.” EX1001, 65:66-66:2. In one embodiment, “a
`12
`
`
`

`

`Attorney Docket No. 50095-0175IP1
`IPR of U.S. Patent No. 11,740,727
`pinch slide gesture may be used to display menus of common operations or
`
`applications.” EX1001, 66:42-44. “For example, in one embodiment, a pinch slide
`
`gesture which begins near an edge of a display and then moves inward may cause a
`
`menu or window to slide in from that edge of the display.” EX1001, 66:44-47.
`
`24. Separately, the ’727 Patent describes embodiments that utilize a “3D
`
`layered user interface” in which “displayed objects may exist in one of a plurality of
`
`layers.” EX1001, 67:44-55. “[A] 3D layered user interface may be implemented in
`
`a device having single layer displays through the use of virtual display layers.”
`
`EX1001, 68:15-17. “[A] virtual display layer refers to a collection of display
`
`elements which have all been assigned the same apparent depth within the 3D
`
`layered user interface.” EX1001, 68:18-21. In some embodiment, “virtual display
`
`layers may be given the appearance of depth through the use of 3D depth cues.”
`
`EX1001, 68:37-39. “For example, in one embodiment, a 3D depth cue may be a
`
`blur operation, such that layers located at successively greater depths may appear
`
`blurrier than those closer to the user.” EX1001, 68:42-45.
`
`25. Notably,
`
`the
`
`’727 Patent does not explicitly describe
`
`the
`
`aforementioned edge menus with respect to the virtual display layers, nor the manner
`
`in which such menus would operate with respect to the virtual display layers. Rather,
`
`the ’727 Patent describes its 3D layered user interface more generically with respect
`
`to “objects” or “display elements” and collections thereof. See generally EX1001,
`
`13
`
`
`

`

`Attorney Docket No. 50095-0175IP1
`IPR of U.S. Patent No. 11,740,727
`67:22-71:18. These “objects” are described as including, but not limited to, “digital
`
`objects (i.e. images, text boxes, UI control widgets, selections, etc.), subject matter
`
`objects (i.e. people within a photograph, letters and/or words within text, etc.), and/or
`
`any other type of identifiable visual object.” EX1001, 68:27-36. While the ’727
`
`Patent describes that these general “objects” or “elements” may be “located in a
`
`virtual display layer” (see EX1001, 68:34-36), there is sparse detail about how
`
`specific “objects” or “elements” are assigned to a particular virtual display layer, and
`
`no clear discussion of whether or how “objects” or “elements” may be reassigned or
`
`otherwise moved between virtual display layers. See generally EX1001, 67:22-
`
`71:18. Rather, the ’727 Patent focuses on the resulting “movement” of objects
`
`between layers giving a user visual cues and the appearance of such “movement”
`
`(“i.e. magnifying and fading display layers as they are passed by, sharpening
`
`previously blurry display layers as they grow closer, etc.”), which is a distinct
`
`concept. EX1001, 70:43-55.
`
`B. Prosecution History
`26. On March 18, 2021, the Applicant (P4TENTS1, LLC) filed the ’727
`
`patent as U.S. Appl. No. 17/206,107. EX1002, 1230-1232. In the first non-final
`
`Office Action dated October 5, 2021, the Examiner rejected the claims as obvious
`
`under 35 U.S.C. § 103 based on U.S. Publication Nos. 2012/0180001 (“Griffin”) and
`
`2010/0159995 (“Stallings”), and for non-statutory double patenting. EX1002, 564-
`
`14
`
`
`

`

`Attorney Docket No. 50095-0175IP1
`IPR of U.S. Patent No. 11,740,727
`580. In response, the Applicant made substantial amendments to the claims, but
`
`these amendments led to a rejection under 35 U.S.C. § 112, first paragraph for lack
`
`of written description support. EX1002, 540-558, 487-496. The Examiner
`
`maintained this § 112 rejection for another Office Action, until the Applicant
`
`canceled all pending claims and filed an entirely new set of claims. EX1002, 396-
`
`408, 381-391.
`
`27. After an interview on June 2, 2023 in which the Examiner “identified
`
`allowable subject matter and called to propose an amendment and expedite an
`
`allowance,” the Examiner issued a Notice of Allowance (NOA) with an Examiner’s
`
`Amendment that incorporated then-pending dependent claim 33 into then-pending
`
`independent claim 29, which issued as independent claim 1. EX1002, 34-56. In the
`
`NOA, the Examiner identified the following feature recited in issued independent
`
`claim 1 that was alleged to distinguish the prior art:
`
`during detection of at least a portion of the gesture before a completion
`
`thereof is detected, blur, based on a change in a distance magnitude of the
`
`gesture being detected on the touch screen to thereby move the object on
`
`the touch screen, at least a portion of the at least one other object that is
`
`not overlapped, such that a magnitude of the blur itself, and not a
`
`magnitude of an area of the touch screen that is blurred, is increased
`
`as a function of an increase in the distance magnitude
`
`15
`
`
`

`

`Attorney Docket No. 50095-0175IP1
`IPR of U.S. Patent No. 11,740,727
`
`EX1002, 52-53 (emphasis in original).
`
`
`
`VI. PERSON OF ORDINARY SKILL IN THE ART
`
`A.
`A POSITA’s Ordinary Level of Skill
`28. Based on my knowledge and experience in the field and my review of
`
`the ’727 patent and its file history, I believe that a POSITA would have included a
`
`Bachelor of Science degree in computer science, computer engineering, electrical
`
`engineering, human-computer interaction, or a comparable field and at least two
`
`years of professional experience in the design and development of graphical user
`
`interfaces (GUIs), human-computer interfaces, or equivalents thereof. Such
`
`experience could be obtained through research and study in a graduate program or
`
`through comparable exposure through industry employment, and additional years of
`
`experience could substitute for the advanced-level degree.
`
`29. My analysis and conclusions expressed in this Declaration are based on
`
`the perspective of a POSITA having this level of knowledge and skill. My
`
`education, technical expertise and personal knowledge discussed in Section II shows
`
`that I meet the qualifications of a POSITA.
`
`
`
`VII. OVERVIEW OF CONCLUSIONS FORMED
`30. This Declaration explains the conclusions that I have formed based on
`
`16
`
`
`

`

`Attorney Docket No. 50095-0175IP1
`IPR of U.S. Patent No. 11,740,727
`my independent analysis. Based upon my knowledge and experience, and my
`
`review of the materials listed above, I believe that Claims 1-6, 10-14, 16, and 17 of
`
`the ’727 Patent are obvious in view of Ahn (EX1004) and Chaudhri ’842 (EX1005).
`
`See Sections VIII-IX.
`
`
`
`VIII. OVERVIEW OF REFERENCES AND COMBINATIONS
`A. Ahn (EX1004)
`31. Ahn generally relates to a “method of displaying a menu in a mobile
`
`communication terminal.” EX1004, Abstract, [0004]. More specifically, Ahn
`
`describes a method of displaying different menus on a display of the mobile
`
`communication terminal based on a “selected direction” of user input. See EX1004,
`
`[0032]-[0033].
`
`32. With respect to FIGS. 3A-10, Ahn describes various methods for
`
`displaying menus on the mobile communication terminal, and provides for control
`
`of the menus by allowing “the user [to] select a direction using any input device that
`
`can select a direction, such as . . . a touch screen . . . .” EX1004, [0072]. Ahn
`
`describes that the mobile communication terminal 100 illustrated in FIG. 1 includes
`
`touch input device 10 (e.g., a “touch screen”) through which “a user inputs various
`
`information or instruction with a touch method.” EX1004, [0027]-[0028]. And Ahn
`
`17
`
`
`

`

`Attorney Docket No. 50095-0175IP1
`IPR of U.S. Patent No. 11,740,727
`provides several additional figures that focus on using a touch screen 10 as an input
`
`for the display of menus. See generally EX1004, [0095]-[0119].
`
`EX1004, FIG. 1 (highlighted)
`
`
`
`33. Specifically, with respect to FIGS. 11-18, Ahn describes “a method of
`
`displaying a menu in a mobile communication terminal having a touch input device,”
`
`in which a menu is displayed in response to a “touch and drag operation.” EX1004,
`
`[0095]-[0099]. Initially, a controller 50 of the mobile communication terminal
`
`“detects a touch and drag operation of a specific direction on the touch screen 10
`
`(S1110).” EX1004, [0098]. “As the touch and drag operation is detected, the
`
`controller 50 controls to display an item list matched to the specific direction in the
`
`display 40 (S1120).” EX1004, [0099]. The “item list” may be for example, a menu
`
`18
`
`
`

`

`Attorney Docket No. 50095-0175IP1
`IPR of U.S. Patent No. 11,740,727
`list or a file list. EX1004, [0099]. Each different direction of a touch and drag
`
`operation may be associated with a different menu or file list. EX1004, [0099].
`
`EX1004, FIG. 11 (annotated).
`
`
`
`34.
`
`In one example illustrated in FIGS. 13a and 13b, “if the user touches a
`
`specific position of the touch screen 10 and drags the touch in a left direction a menu
`
`list 1300 matched to a left direction is displayed in the display 40.” EX1004, [0102].
`
`As can be seen in FIG. 13a, a user initially touches near the right side or edge of the
`
`touch screen 10 and initiates the drag gesture to open the menu. See EX1004, FIG.
`
`13a, [0102]. As a result of controller 50 detecting the gesture, “a menu list matched
`
`to a left direction is set to be displayed at the right side of the display 40 (FIG.
`
`13(b)).” EX1004, [0104].
`
`19
`
`
`

`

`Attorney Docket No. 50095-0175IP1
`IPR of U.S. Patent No. 11,740,727
`
`
`
`EX1004, FIGS 13a and 13b (annotated).
`
`35. Ahn describes that the displayed menus may be “transparently
`
`displayed on the standby screen 300.” EX1004, [0070]. This can be seen, for
`
`example, by comparing FIG. 3A, which shows standby screen 300 without any
`
`menus, and FIG. 7B, which shows the basic setting menu list 720 “displayed in a
`
`transparent window form in a left area of the standby screen 300.” EX1004, [0064].
`
`Ahn describes that transparent menu windows ensure that “the preset standby screen
`
`20
`
`
`

`

`Attorney Docket No. 50095-0175IP1
`IPR of U.S. Patent No. 11,740,727
`300 is not hidden by the pop-up menu window 340.” EX1004, [0039]. However,
`
`Ahn is not limited to these transparent menu windows.
`
`Trees that are part of standby 
`screen 300 can be seen through the 
`transparent menu window 
`
`EX1004, FIGS. 3A and 7B (annotated)
`
`
`
`36. Ahn teaches at least two ways that its user interface may utilize blur.
`
`First, Ahn teaches that, instead of a transparent form, its menus “can be embodied
`
`in a translucent form or an opaque form, depending on implementation.” EX1004,
`
`[0070] (emphasis added). A POSITA would have known and found it obvious that,
`
`when differentiated from “transparent” and “opaque” in the manner they are used by
`
`Ahn, the word “translucent” means “allowing light, but not detailed images, to pass
`21
`
`
`

`

`Attorney Docket No. 50095-0175IP1
`IPR of U.S. Patent No. 11,740,727
`through; semitransparent.” EX1009, 6. A POSITA would have understood not
`
`transmitting detail as a type of blurring, because the word “blur” means to “make or
`
`become unclear or less distinct.” EX1009, 3. In other words, a POSITA would have
`
`known and found it obvious that Ahn contemplates that, when embodied in
`
`translucent form, its menus would have blurred at least a portion of the background
`
`standby screen (e.g., shown in FIG. 12).
`
`
`
`EX1004, FIGS. 3A and 4B (annotated).
`
`37. Second, Ahn explicitly shows that opening a menu can blur at least a
`
`portion of the standby screen 300. In FIG. 3A “a standby screen 300 set by a user is
`
`displayed in the display 40 of the mobile communication terminal 100.” EX1004,
`
`22
`
`
`

`

`Attorney Docket No. 50095-0175IP1
`IPR of U.S. Patent No. 11,740,727
`[0036]. Ahn illustrates that a clock widget is part of the background standby screen
`
`300. See EX1004, [0036], FIG. 3A. A similar clock widget is shown as part of the
`
`background standby screen 300 in FIGS. 12 and 13a. See EX1004, [0097], [0102],
`
`FIGS.12, 13a. As can be seen in FIG. 4B, when the user menu 420 is opened, Ahn
`
`illustrates that the clock widget is blurred, which is pointed out with red arrows in
`
`the above annotation of FIGS. 3A and 4B. See EX1004, [0102], FIG. 13a. A similar
`
`blurring effect of the clock widget can also be seen in FIGS. 10 and 13B. See
`
`EX1004, FIGS. 10, 13B.
`
`38. Ahn does not explicitly explain or otherwise limit the implementation
`
`details a POSITA might utilize in providing this effect. Thus, a POSITA would have
`
`naturally turned to teachings like those provided by Chaudhri ’842 when developing
`
`the blurring shown in FIGS. 4B, 10, and 13B of Ahn.
`
`B. Chaudhri ’842 (EX1005)
`39. Like the ’727 Patent and Ahn, Chaudhri ’842 “relate[s] generally to
`
`user interfaces that employ touch-sensitive displays.” EX1005, [0002]. Chaudhri
`
`’842 recognized that “there is a need for more efficient, user-friendly procedures for
`
`transitioning such devices, touch screens, and/or applications between user interface
`
`states (e.g., from a user interface state for a first application to a user interface state
`
`for a second application, between user interface states in the same application, or
`
`between locked and unlocked states).” EX1005, [0006].
`
`23
`
`
`

`

`Attorney Docket No. 50095-0175IP1
`IPR of U.S. Patent No. 11,740,727
`In one embodiment described in greater detail below, Chaudhri ’842
`
`40.
`
`describes supplying visual feedback to a user who is transitioning between user
`
`interface states on a touch screen by providing “visual differentiation between the
`
`user-interface object and the background, based on color, hue, color saturation,
`
`brightness, contrast, transparency, and any combination thereof.” See EX1005,
`
`[0085]. Through this embodiment, Chaudhri ’842 provides at least one possible set
`
`of implementation details particularly relevant to the blurring shown in Ahn.
`
`41. Like Ahn’s mobile communication terminal, Chaudhri ’842 describes
`
`a portable electronic device 100 that includes a touch screen 126. EX1005, [0026].
`
`“The touch screen 126 provides both an output interface and an input interface
`
`between the device and a user.” EX1005, [0033]. The portable electronic device
`
`also includes “one or more processors 106 [that] run various software programs
`
`and/or sets of instructions stored in the memory 102 to perform various functions for
`
`the device 100 and to process data.” EX1005, [0028]. The “software components
`
`include an operating system 132, a communication module (or set of instructions)
`
`134, a contact/motion module (or set of instructions) 138, a graphics module (or set
`
`of instructions) 140, a user interface state module (or set of instructions) 144, and
`
`one or more applications (or set of instructions) 146.” EX1005, [0037].
`
`42.
`
`“The contact/motion module 138
`
`includes various software
`
`components for performing various operations related to detection of contact with
`
`24
`
`
`

`

`Attorney Docket No. 50095-0175IP1
`IPR of U.S. Patent No. 11,740,727
`the touch screen 122, such as determining if contact has occurred, determining if
`
`there is movement of the contact and tracking the movement across the touch screen,
`
`and determining if the contact has been broken (i.e., if the contact has ceased).”
`
`EX1005, [0040]. “The graphics module 140 includes various known software
`
`components for rendering and displaying graphics on the touch screen 126.”
`
`EX1005, [0