Page 1 of 85
`UNITED STATES PATENT AND TRADEMARK OFFICE
`______________
`
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`______________
`
`
`APPLE INC.,
`
`Petitioner
`
`
`v.
`
`
`SMITH INTERFACE TECHNOLOGIES, LLC
`
`Patent Owner
`
`______________
`
`IPR2024-01256
`Patent 11,740,727
`______________
`
`
`
`DECLARATION OF CRAIG ROSENBERG, Ph.D.
`
`
`
`
`
`
`
`
`
`
`Apple Inc. v. Smith Interface Techs., LLC
`IPR2024-01256 | Smith EX2006
`
`
`
`
`
`
`
`Page 2 of 85
`TABLE OF CONTENTS
`Page
`I. INTRODUCTION .................................................................................................................. 4
`II. PROFESSIONAL BACKGROUND ...................................................................................... 4
`III. COMPENSATION ........................................................................................................... 11
`IV. LEGAL CONSIDERATION ............................................................................................ 11
`A. Anticipation....................................................................................................................... 11
`B. Obviousness ...................................................................................................................... 12
`C. Claim Interpretation .......................................................................................................... 13
`D. Overview of Task & Basis for Opinions........................................................................... 14
`E. Level of Skill in the Relevant Art ..................................................................................... 15
`V. OVERVIEW OF THE ’727 PATENT .................................................................................. 15
`A. Independent Claim 1 ......................................................................................................... 17
`VI. OVERVIEW OF THE PRIOR ART................................................................................. 18
`A. U.S. Patent App. No. 2008/0207188 (“Ahn”) .................................................................. 18
`B. U.S. Patent App. No. 2007/0150842 (“Chaudhri”) .......................................................... 27
`VII. CLAIM CONSTRUCTION .............................................................................................. 30
`A. “Blur” ................................................................................................................................ 30
`B. “Distance Magnitude” ....................................................................................................... 31
`C. “Top,” “Bottom,” “Right,” and “Left” .............................................................................. 31
`VIII. A POSITA WOULD NOT HAVE BEEN MOTIVATED TO COMBINE AHN AND
`CHAUDHRI ................................................................................................................................. 31
`A. Dr. Terveen Failed to Articulate Any Credible Motivation to Combine Ahn and Chaudhri
` 31
`B. Dr. Terveen’s Proposed Modification Would Not Achieve the Functionality or Benefits
`Allegedly Motivating the Ahn-Chaudhri Combination ............................................................ 35
`1. Dr. Terveen’s proposed modification does not provide visual feedback of the progress
`of a user’s gesture ................................................................................................................. 36
`2. Dr. Terveen’s proposed modification does not help differentiate between objects ...... 39
`3. Dr. Terveen’s implementation details motivation is contrary to Chaudhri .................. 41
`4. Dr. Terveen’s choice of progressive blurring is not supported ..................................... 41
`IX. DR. TERVEEN’S ANALYSIS FAILS TO SHOW OBVIOUSNESS OF CLAIM 1 BY
`THE AHN-CHAUDHRI COMBINATION ................................................................................. 43
`A. Dr. Terveen’s Mapping to Limitation [1.d.i] is Deficient ................................................. 43
`
`
`
`
`
`
`
`Page 3 of 85
`1. Dr. Terveen’s theory does not establish that a “gesture” “move[s] the object on the
`touch screen” ......................................................................................................................... 43
`a) Ahn’s Menus are Not Progressive ............................................................................. 44
`2. Dr. Terveen’s theory does not establish that “the at least one other object” is “blur[red],
`based on a change in distance magnitude of the gesture” ..................................................... 53
`a) Dr. Terveen’s modification does not show progress ................................................ 54
`b) Dr. Terveen’s modification operates contrary to Chaudhri ...................................... 59
`3. Ahn-Chaudhri would not blur during detection of the gesture and before a completion
`of the gesture is detected ....................................................................................................... 64
`B. Dr. Terveen’s Mapping to Limitation [1.d.ii] is Deficient ............................................... 68
`1. Ahn-Chaudhri would not increase the magnitude of the blur of Ahn’s clock widget as a
`function of an increase in the distance magnitude of the gesture ......................................... 69
`2. Chaudhri’s change in “optical intensity” does not apply to Dr. Terveen’s modification
` 71
`X. DR. TERVEEN DOES NOT ESTABLISH THAT AHN-CHAUDRHI TEACHES
`MULTIPLE DEPENDENT CLAIMS .......................................................................................... 74
`A. Claim 2 .............................................................................................................................. 74
`B. Claim 3 .............................................................................................................................. 77
`C. Claims 5 and 17 ................................................................................................................ 79
`D. Claim 6 .............................................................................................................................. 79
`E. Claims 11 and 12 .............................................................................................................. 81
`F. Claims 13 and 14 .............................................................................................................. 83
`XI. CONCLUSION ................................................................................................................. 85
`
`
`
`
`
`
`
`
`Page 4 of 85
`I. INTRODUCTION
`1. My name is Dr. Craig Rosenberg, Ph.D. I have been retained by Smith
`Interface Technologies, LLC as an independent expert consultant in this inter
`partes review (“IPR”) proceeding before the United States Patent and Trademark
`Office (“PTO”).
`II. PROFESSIONAL BACKGROUND
`2. All of my opinions stated in this Declaration are based on my own
`personal knowledge and professional judgment. In forming my opinions, I have
`relied on my knowledge and experience in designing, developing, researching, and
`teaching the technology referenced in this Declaration.
`3. I am over 18 years of age and, if I am called upon to do so, I would be
`competent to testify as to the matters set forth herein. I understand that a copy of
`my current curriculum vitae, which details my education and professional and
`academic experience, is being submitted as EX2007. The following provides a
`brief overview of some of my experiences that are relevant to the matters set forth
`in this Declaration.
`4. My qualifications to testify about the patents in this case and the
`relevant technology are set forth in my curriculum vitae (“CV”), which I have
`included as Exhibit A. In addition, a brief summary of my qualifications is
`included below.
`
`
`
`
`
`
`
`Page 5 of 85
`5. I hold a Bachelor of Science in Industrial Engineering, a Master of
`Science in Human Factors, and a Ph.D. in Human Factors from the University of
`Washington School of Engineering. For 30 years, I have worked in various areas,
`including human factors, user interface design, software development, software
`architecture, systems engineering, and modeling and simulation, across a wide
`range of application areas, such as aerospace, communications, entertainment, and
`healthcare.
`6. I graduated from the University of Washington in 1988 with a
`Bachelor of Science degree in Industrial Engineering. After graduation, I continued
`my studies at the University of Washington College of Engineering. In 1990, I
`obtained an M.S. in Human Factors. Human Factors is an engineering discipline
`that studies the design of products, processes, equipment, and systems to enhance
`their efficiency in interaction with humans. The goal of human factors is to reduce
`human error, increase productivity and efficiency, and enhance safety and comfort,
`with a specific focus on the interaction and interfaces between humans and the
`products, processes, or systems with which they interact. In 1994, I earned a Ph.D.
`in Human Factors, with a focus on advanced interface design. My B.S., M.S., and
`Ph.D. were all taught at the University of Washington College of Engineering.
`7. During my doctoral studies, I worked as an Associate Assistant
`Professor of Human Factors at the University of Washington's Industrial
`
`
`
`
`
`
`
`Page 6 of 85
`Engineering Department. My duties included teaching, writing research proposals,
`designing and conducting funded human factors experiments for the National
`Science Foundation, and hiring and supervising students. While studying at the
`University of Washington, I also worked as a human factors researcher, designing
`and conducting advanced human factors experiments related to virtual
`environments, interface design, stereoscopic displays, and advanced visualization
`research, which the National Science Foundation funded. My duties included user
`interface design, system design, software development, graphics programming,
`experimental design, and interfacing between hardware and software.
`8. I have published twenty-one research papers in professional journals
`and proceedings in the areas of user interface design, computer graphics, and the
`design of spatial, stereographic, and auditory displays. I also authored a book
`chapter on augmented reality displays in the book “Virtual Environments and
`Advanced Interface Design” (Oxford University Press, 1995). In addition, I created
`one of the first virtual spatial musical instruments called the MIDIBIRD that
`utilized the MIDI protocol, two six-dimensional spatial trackers, a music
`synthesizer, and a computer graphics workstation to create an advanced and novel
`musical instrument.
`9. For the past 21 years, I have served as a consultant for Global
`Technica, Sunny Day Software, Stanley Associates, Techrizon, CDI Corporation,
`
`
`
`
`
`
`
`Page 7 of 85
`and the Barr Group. In this capacity, I have provided advanced engineering
`services for many companies.
`10. I consulted for the Boeing Company for over 16 years as a senior
`human factors engineer, user interface designer, and software architect for a wide
`range of advanced commercial and military programs. Many of the projects that I
`was involved with at Boeing involved advanced software development, user
`interface design, agent-based software, and modeling and simulations in the areas
`of missile defense, homeland security, battle command management, computer-
`aided design, networking and communications, air traffic control, location-based
`services, and Unmanned Aerial Vehicle (“UAV”) command and control.
`Additionally, I served as the lead system architect, developing advanced air traffic
`controller workstations and air traffic control analysis applications, toolsets, and
`trade study simulations for Boeing Air Traffic Management.
`11. I was also the architect of the Boeing Human Agent Model. The
`Boeing Human Agent Model is an advanced model for simulating human sensory,
`cognitive, and motor performance as applied to the roles of air traffic controllers,
`pilots, and UAV operators. In another project, I served as the lead human factors
`engineer and user interface designer for Boeing’s primary vector and raster
`computer-aided drafting and editing system, which produces maintenance manuals,
`shop floor illustrations, and service bulletins for aircraft manufactured by the
`
`
`
`
`
`
`
`Page 8 of 85
`Boeing Commercial Aircraft Company. Additional responsibilities during my time
`as a consultant include system engineering, requirements analysis, functional
`specification, use case development, user stories, application prototyping,
`modeling and simulation, object-oriented software architecture, graphical user
`interface analysis, and design, as well as software development using UML, C++,
`C#, and Java.
`12. In 1995 and 1996, I was hired as the lead human factors engineer and
`user interface designer for the first two-way pager produced by AT&T. Prior to
`this technology, individuals could receive pages but had no way to respond using
`their pagers. This new technology enabled users to use a small, handheld device to
`receive and send canned or custom text messages, access and update their address
`book, and view and update their personal calendar. This high-profile project
`involved designing the entire feature set, user interface/user interaction design,
`specifications, as well as all graphical design standards.
`13. From 1999–2001, I was the lead human factors engineer and user
`interface designer for a company called Eyematic Interfaces, where I was
`responsible for all user interface design and development activities associated with
`real-time mobile handheld 3D facial tracking, animation, avatar creation, and
`editing software for a product for Mattel. My work involved user interface design,
`human factors analysis, requirements gathering and analysis, and functional
`
`
`
`
`
`
`
`Page 9 of 85
`specifications.
`14. I was the lead user interface designer for a company called
`ObjectSpeed, which developed a portable handheld telephone for use in homes and
`businesses that had many of the same capabilities that we take for granted in
`mobile cellular phones. This portable multifunction device supports voice, email,
`chat, video conferencing, internet radio, streaming media, Microsoft Outlook
`integration, photo taking and sharing, etc. The ObjectSpeed device was specifically
`designed and developed as a portable handheld device.
`15. I was the lead UI designer for Ahaza, a company that builds IPv6
`routers. I designed the user interfaces for configuring and controlling these
`advanced network hardware devices. My responsibilities included requirements
`analysis, functional specification, user interface design, user experience design,
`and human factors analysis.
`16. I am the founder, inventor, user interface designer, and software
`architect of WhereWuz. WhereWuz is a company that develops advanced mobile
`software for GPS-enabled smartphones and handheld devices. WhereWuz allows
`users to record exactly where they have been and query this data in unique ways
`for subsequent retrieval based on time or location. WhereWuz was specifically
`designed and developed to run on small handheld devices.
`17. I am the co-founder of a medical technology company called
`
`
`
`
`
`
`
`Page 10 of 85
`Healium. Healium developed advanced wearable and handheld user interface
`technology, enabling physicians to interact more effectively with electronic
`medical records.
`18. I am the co-founder of StratoScientific, a medical technology
`company. StratoScientific is developing an innovative case for a smartphone that
`transforms a standard handheld device into a full-featured digital stethoscope,
`incorporating visualization and machine learning for telemedicine and automated
`diagnosis.
`19. I designed and developed a large software project for Disney World,
`called xVR, which enabled operational employees to use a handheld device to view
`the current and historical status of all guests within multiple attractions and one of
`their restaurants. The application could run in a real-time/live mode, displaying
`data collected from sensors that show the location and status of all guests within
`the attraction. The application could also run in a fast time/simulated mode. The
`application was developed on a laptop computer and was specifically designed to
`run on various devices, including laptops, PCs, smartphones, and tablets.
`20. I have received several awards for my engineering work relating to
`interface design, computer graphics, and the design of spatial, stereographic, and
`auditory displays, including a $10,000 scholarship from the I/ITSEC for advancing
`the field of interactive computer graphics for flight simulation and a Link
`
`
`
`
`
`
`
`Page 11 of 85
`Foundation award for furthering the field of flight simulation and virtual interface
`design. I have also created computer graphic illustrations for several popular book
`covers and animations for a movie produced by MIRAMAR.
`III. COMPENSATION
`21. My work on this matter is being billed at my customary rate of $650
`an hour. Additionally, I am reimbursed for reasonable expenses incurred in relation
`to my services. I have no pecuniary interest in the outcome of this proceeding. I
`understand I will be compensated regardless of the outcome of any proceeding in
`which my work is used.
`IV. LEGAL CONSIDERATION
`22. I am not an attorney. My understanding of the law is based on
`information provided by counsel for Smith Interface Technologies, LLC.
`A. Anticipation
`23. I understand that a reference anticipates a claim if it discloses each
`and every element recited in the claim, arranged as in the claim, so as to enable a
`person of ordinary skill in the art (“POSITA”) to make and use the claimed
`invention without the need for undue experimentation in light of the general
`knowledge available in the art. I understand that a U.S. Patent document is
`presumed to have sufficient description to include sufficient detail for a person of
`ordinary skill in the art to make and use the subject matter that the document
`describes.
`
`
`
`
`
`
`
`Page 12 of 85
`24. I also understand that a claim limitation may be present in an alleged
`prior art reference either expressly or inherently, but inherency may be established
`only if such limitation is necessarily present in the reference. I understand that
`probabilities or possibilities cannot establish inherency, and the mere fact that a
`certain thing may result from a given set of circumstances is not sufficient to
`establish inherency.
`B. Obviousness
`25. I understand that a claimed invention is obvious and, therefore, not
`patentable if the subject matter claimed would have been obvious to a POSITA at
`the time that the invention of U.S. Patent No. 11,740,727 (“the ’727 Patent”),
`which I have been asked to treat as August 5, 2011.
`26. I understand that a claim can be obvious in view of a single prior art
`reference (e.g., via modification of that prior art reference) or multiple prior art
`references (e.g., via a combination of two or more prior art references), if such a
`modification or combination was within the skill of a POSITA. I understand that
`there must be some articulated reasoning with some rational underpinning to
`support a conclusion of obviousness. I also understand that to establish a finding of
`obviousness one must show that one of ordinary skill in the art would have had a
`motivation to combine the prior art references to produce the claimed invention
`and a reasonable expectation that the combination would be successful.
`
`
`
`
`
`
`
`Page 13 of 85
`27. I further understand that exemplary rationales that may support a
`conclusion of obviousness include: (1) simply arranging old elements in a way in
`which each element performs the same function it was known to perform, and the
`arrangement yields expected results, (2) merely substituting one element for
`another known element in the field, if the substitution yields no more than a
`predictable result, (3) combining elements in a way that was “obvious to try”
`because of a design need or market pressure, where there was a finite number of
`identified, predictable solutions, (4) that design incentives or other market forces in
`a field would have prompted variations in a work that were predictable to a person
`of ordinary skill in the art, and (5) that there was some teaching, suggestion, or
`motivation in the prior art that would have led a POSITA to modify or combine
`prior art references to arrive at the claimed invention.
`C. Claim Interpretation
`28. I understand that a claim term is interpreted according to its ordinary
`and customary meaning as a POSITA would have understood the term in light of
`the surrounding claim language, other claims, the specification, and the patent’s
`prosecution history, which are referred to as intrinsic evidence. I also understand
`that prior art references cited in the patent’s prosecution history are considered
`intrinsic evidence. I further understand that evidence outside the patent and its
`prosecution history (e.g., dictionaries and technical articles) may inform the
`
`
`
`
`
`
`
`Page 14 of 85
`context in which a POSITA would have understood the claims of a patent. I
`understand this ordinary and customary meaning applies absent unique
`circumstances, such as where a patent clearly expresses an intent to set forth a
`special meaning for a term.
`D. Overview of Task & Basis for Opinions
`29. I have been asked to review the ’727 Patent. I have been asked to
`provide opinions related to certain issues from the perspective of a person of
`ordinary skill, having knowledge of the relevant art, as of August 5, 2011, and—
`except where otherwise noted—the opinions stated in this declaration are from that
`perspective. The qualifications and abilities of such a person are described in
`Section IV.E below.
`30. My opinions are based on my education, training, and experience as
`well as items that I reviewed to prepare my opinions, including the Petition, the
`documents listed in the Exhibit List included with the Petition, and any other items
`that I reference in my analysis below.
`31. I understand that other issues may arise that require further
`explanation, and I will provide that explanation if appropriate. As a result, I
`respectfully reserve the right to update and supplement this declaration and the
`information and opinions provided herein.
`
`
`
`
`
`
`
`Page 15 of 85
`E. Level of Skill in the Relevant Art
`32. I understand that a patent must be analyzed from the perspective of a
`POSITA as of the time of invention of the patent, which is typically considered to
`be the patent’s earliest filing date. While I do not necessarily agree with every
`aspect of Dr. Terveen’s explanation of the level of ordinary skill in the art, I do not
`currently believe the aspects of that explanation I would change or clarify would
`impact the analysis set forth in this declaration. For purposes of this declaration,
`my opinions are therefore provided from the perspective of the POSITA described
`by Dr. Terveen’s declaration (EX1003) as of August 5, 2011, which is produced
`below:
`“a Bachelor of Science degree in computer science,
`computer engineering, electrical engineering, human-
`computer interaction, or a comparable field and at least
`two years of professional experience in the design and
`development of graphical user interfaces (GUIs), human-
`computer interfaces, or equivalents thereof. Such
`experience could be obtained through research and study
`in a graduate program or through comparable exposure
`through industry employment, and additional years of
`experience could substitute for the advanced-level
`degree.”
`EX1003, ¶28.
`V. OVERVIEW OF THE ’727 PATENT
`33. The ’727 Patent describes devices, methods, and graphical user
`interfaces for manipulating user interface objects with visual and/or haptic
`feedback. See EX1001, at Title (“Devices, Methods, and Graphical User Interfaces
`
`
`
`
`
`
`
`Page 16 of 85
`for Manipulating User Interface Objects with Visual and/or Haptic Feedback”).
`34. The ’727 Patent describes a mobile device having various
`components, including user interface devices such as a touch screen. EX1001,
`13:41-58. In operation, signals can be received in association with the touch
`interface (e.g., touch screen) and can correspond to a signal indicative of a touch
`interface being touched. Id., 14:11-33. The user experience can then be altered
`utilizing these touch signals. Id., 15:19-31, FIG. 3.
`35. An example of these operations can be seen in FIG. 18, where
`attributes of touch events are detected, and gesture inputs related to these touch
`event attributes (both initial and over time) can be identified. EX1001, 56:8-35.
`
`
`
`
`
`
`
`
`
`Page 17 of 85
`EX1001, FIG. 18.
`36. These attributes can include the duration and/or distance of a gesture
`because often different gestures will be indistinguishable based on initial contact.
`EX1001, 56:8-35, 60:43-47. Once the gesture is identified, a corresponding
`operation can be performed (step 1808). EX1001, 56:50-51. Such operations can
`include the modification of one or more display attributes such as blurring the
`display attributes. EX1001, 57:9-10, 68:37-52. In some operations, “a user may be
`given feedback as a gesture is performed.” Id., 59:23-24. For example, some
`“gestures may be associated with operations that are carried out as functions of the
`gesture input” EX1001, 58:58-59:3. In this way, the device can provide a user with
`feedback as a gesture is performed. See EX1001, 59:23-29, FIG. 18 (step 1802
`showing continuous detection of touch event as gesture is performed).
`A. Independent Claim 1
`37. Independent claim 1 is directed to a “device” having one or more
`processors that are configured to execute instructions that cause the device to
`perform functions. EX1001, 102:31-50. Specifically, claim 1 recites:
`[1.a] A device, comprising: at least one non-transitory memory; a
`touch screen; and one or more processors in communication with the
`at least one non-transitory memory, and the touch screen, wherein the
`
`
`
`
`
`
`
`Page 18 of 85
`one or more processors execute instructions in the at least one non-
`transitory memory, to cause the device to:
`[1.b] display an object and at least one other object;
`[1.c] detect at least part of a gesture on the touch screen; and
`[1.d.i] during detection of at least a portion of the gesture before a
`completion thereof is detected, blur, based on a change in a distance
`magnitude of the gesture being detected on the touch screen to thereby
`move the object on the touch screen, at least a portion of the at least
`one other object that is not overlapped
`[1.d.ii] such that a magnitude of the blur itself, and not a magnitude of
`an area of the touch screen that is blurred, is increased as a function of
`an increase in the distance magnitude.
`EX1001, 102:31-50.
`VI. OVERVIEW OF THE PRIOR ART
`38. Dr. Terveen primarily relies on two cited references for his invalidity
`analysis. See EX1003, ¶62 (the Ahn-Chaudhri combination).
`A. U.S. Patent App. No. 2008/0207188 (“Ahn”)
`39. U.S. Patent App. No. 2008/0207188 is titled “Method of Displaying
`Menu in a Mobile Communication Terminal” and lists as its first inventor Hye
`Sang Ahn. I will refer to this reference as “Ahn.”
`
`
`
`
`
`
`
`Page 19 of 85
`40. As depicted in FIG. 2 (inset below), Ahn discloses a mobile
`communication terminal that includes a display 40 along with a key input device
`30 and a touch input device (e.g., element 10 in FIG. 1). EX1004, Abstract,
`¶[0027]. Ahn’s proposed operation attempts to solve the problem where device
`characteristics are unavailable when a menu list is displayed. EX1004, ¶¶[0005]-
`[0006]. In conventional browsers, the standby screen disappears when a menu
`screen is displayed. Thus, Ahn proposes to provide menus that only take up a
`portion of the display so that the standby screen can still be displayed concurrently
`with the menu.
`
`41. Against this backdrop, Ahn discloses a method of displaying menus in
`
`
`
`
`
`
`
`
`Page 20 of 85
`a mobile communication terminal, where each menu can correspond with a
`selection of one of a plurality of specific directions associated with the menus.
`EX1004, Abstract (“menu corresponding to the selected specific direction on the
`standby screen”). Ahn discloses three separate methods for displaying its menu via
`the use of (1) directional keys, (2) a simple touch and drag operation, and (3) via a
`start position operation. In each method, the menu is displayed after a
`corresponding input (e.g., button click or drag) is completed. That is, the display of
`the menu is a binary operation—appearing only when the input is satisfied. None
`of Ahn’s menus are described as dynamically following a user’s finger.
`42. In a first embodiment, Ahn describes a method of displaying a menu
`via a selection of a specific direction on a direction key. See EX1004, ¶¶[0030],
`[0034]-[0035]. ]. As shown in FIGS. 7A-7C below, a menu (e.g., setting menu
`720) can be accessed “when a leftward direction key is selected.” Id., ¶[0062]. Ahn
`describes “if the user selects the leftward direction key 38 . . . [t]hereafter, a basic
`setting menu list 720 . . . is displayed”. EX1004,¶[0063]-[0064]1; FIG. 2 (showing
`directional keys).
`
`
`
`
`
`
`
`Page 21 of 85
`
`43. Ahn refers to an input of the its direction keys (e.g., 34, 38) as a
`“click.” EX1004, ¶¶[0045]-[0047]. The direction keys are single input buttons that
`can only receive one type of input and a POSITA would have recognized a
`selection (i.e., click) of a direction key as an input used to initiate a binary
`operation. Accordingly, if the direction key is pressed, then the menu is displayed
`“thereafter,” and if the key is not pressed, the menu is not displayed. EX1004,
`¶¶[0051], [0060], [0064]. Accordingly, the opening of Ahn’s menu is performed in
`a binary manner rather than progressively.
`44. In another embodiment, Ahn describes a method of displaying a menu
`via “a touch and drag operation of a specific direction.” Id., ¶[0101], see also
`¶¶[0102]-[0104]. The menus of Ahn’s “touch and drag” embodiments are identical
`to the menus of Ahn’s “directional key” embodiments, only differing with regard
`to the method of initiating their display. Compare EX1004, FIGS. 4B, 6B, 7B, with
`
`
`
`
`
`
`
`
`Page 22 of 85
`FIGS. 13B, 14B, 15B. The opening of the menus according to Ahn’s touch and
`drag operation is also performed in a binary manner, as explained below.
`45. As shown in FIGS. 15a-b of Ahn (inset below), “[a] controller 50
`detects a touch and drag operation of a specific direction in the touch screen 10.”
`Id., ¶[0107].
`
`46. Then “controller 50 determines whether a drag distance of the
`detected touch and drag operation is a first distance or more.” Id., ¶[0109]. After
`the user performs the touch and drag gesture and the controller compares the drag
`distance to a threshold, the menu can then be displayed. Id.; see also FIG. 16
`(detecting touch at step 1610 and displaying menu at step 1630)
`
`
`
`
`
`
`
`
`Page 23 of 85
`
`EX1004, FIG. 16.
`47. Ahn’s descriptions of its “touch and drag” embodiments do not
`include any disclosures that would lead a POSITA to understand the “touch and
`drag” menus to display any more progressively than the directional key menus. See
`EX1004, ¶¶[0102]-[0103] (for example, “if the user . . . drags the touch in a left
`direction a menu list 1300 matched to a left direction is displayed in the display
`40”). Therefore, a POSITA would recognize that, despite being initiated via a
`different type of user input, Ahn’s “touch and drag” menus are displayed via the
`same binary process by which its directional key menus are displayed.
`
`
`
`
`
`
`
`
`Page 24 of 85
`48. This operation is also depicted in Ahn’s FIG. 11, which shows that the
`display of the item list is after the operation is detected (i.e., completed).
`
`49. To the extent it is argued that Ahn’s FIG. 11 operates differently than
`FIG. 16 by displaying a menu before the “touch and drag” operation is completed,
`this is inconsistent with Ahn. While paragraph [0099] states “[a]s the touch and
`drag operation is detected, the controller 50 controls to display an item list matched
`to the specific direction in the display 40 (S1120),” a POSITA would not
`understand the phrase “[a]s the touch and drag operation is detected” to mean the
`menu is displayed before the operation is compete. Ahn uses this phrase several
`other times to mean that the menu is