UNITED STATES PATENT AND TRADEMARK OFFICE
`______________
`
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`
`______________
`
`APPLE INC.,
`
`PETITIONER
`
`V.
`
`SMITH INTERFACE TECHNOLOGIES, LLC,
`
`PATENT OWNER
`
`______________
`
`INTER PARTES REVIEW NO. IPR2024-01090
`PATENT 10,725,581 B1
`______________
`
`DECLARATION OF DR. MARK P. MAHON
`
`Apple Inc. v. Smith Interface Techs., LLC
`IPR2024-01090 | Smith EX2008 | Page 1 of 99
`
`

`

`Table of Contents
`
`Page
`
`INTRODUCTION ............................................................................................... 1
`I.
`BACKGROUND AND QUALIFICATIONS ................................................. 1
`II.
`III. COMPENSATION .......................................................................................... 9
`IV.
`LEGAL CONSIDERATIONS ......................................................................10
`A.
`Anticipation ................................................................................................10
`B.
`Obviousness ...............................................................................................11
`C.
`Claim Interpretation ...................................................................................12
`V.
`TASK SUMMARY .......................................................................................12
`VI. OVERVIEW OF THE ’581 PATENT ..........................................................15
`A. Multi-Part-Pattern Based Gestures .............................................................15
`1. Figure 16 ....................................................................................................16
`2. Figure 6 ......................................................................................................21
`3. Figure 18 ....................................................................................................23
`B.
`Independent Claims 1 and 42 .....................................................................23
`1. “Conditional” Limitations of Claim 1 .......................................................26
`2. “Conditional” Limitations of Claim 42 .....................................................33
`3. The First Contact .......................................................................................33
`VII. LEVEL OF ORDINARY SKILL IN THE ART ...........................................34
`VIII. OVERVIEW OF THE PRIOR ART ..........................................................35
`A.
`Blumenberg ................................................................................................35
`1. Hyperlink Preview Functionality ...............................................................35
`2. Hyperlink Activation Functionality ............................................................37
`3. Dr. Cockburn’s Mapping of Blumenberg ..................................................38
`4. Blumenberg’s Fourth Operational Scenario .............................................41
`B.
`Bhumkar .....................................................................................................42
`1. Preview Window With Toolbar Functionality ...........................................42
`C.
`Griffin .........................................................................................................44
`
`ii
`
`Page 2 of 99
`
`

`

`IX. GROUND 1: THE PETITION AS SUPPORTED BY DR. COCKBURN’S
`DECLARATION FAILS TO SHOW OBVIOUSNESS OF CLAIMS 1-3, 21, 29-31, 42-44
`OVER BLUMENBERG, BHUMKAR, AND GRIFFIN ........................................46
`A.
`Dr. Cockburn’s Combination Fails to Meet All the Claimed Elements of Claims 1 and
`42
`46
`1. Limitations [1.m.], [42.h] ..........................................................................47
`2. Limitations [1.n], [42i] ..............................................................................54
`3. Limitations [1.o], [42j] ..............................................................................59
`B.
`A POSITA Would Not Have Been Motivated to Modify Blumenberg’s Link
`Information to Have a Menu ................................................................................69
`1. Dr. Cockburn’s Modifications To Blumenberg Are Incompatible With Its Teachings71
`2. Dr. Cockburn’s Motivations Do Not Resolve the Incompatibilities Of The
`Combinations ....................................................................................................79
`DEPENDENT CLAIMS 2, 21, 29-30, 43, 54, 62-63 ....................................90
`X.
`Claims 2 and 43 ..........................................................................................90
`A.
`Claims 21 and 54 ........................................................................................92
`B.
`Claims 29-30 and 62-63 .............................................................................94
`C.
`XI. CONCLUSION ..............................................................................................96
`
`iii
`
`Page 3 of 99
`
`

`

`I.
`
`INTRODUCTION
`
`
`
`1. My name is Mark Mahon. I am over the age of twenty-one (21) years,
`
`of sound mind, and capable of making the statements set forth in this Declaration. I
`
`am competent to testify about the matters set forth herein. All the facts and
`
`statements contained herein are with my personal knowledge, and they are true and
`
`correct to the best of my knowledge.
`
`2.
`
`I have been asked by Smith Interface Technologies, LLC (“Smith” or
`
`“Patent Owner”) to submit this declaration in response to the challenge of the
`
`validity of claims 1-3, 21, 29-31, 42-44 (“Challenged Claims”) of U.S. Patent No.
`
`10,725,581 (the “’581 Patent”) by Petitioner Apple Inc. (“Apple” or “Petitioner”).
`
`II. BACKGROUND AND QUALIFICATIONS
`
`3.
`
`Exhibit 2006 is a true and correct copy of my curriculum vitae (“CV”),
`
`containing my complete qualifications and professional experience. The following
`
`is a brief summary of my relevant qualifications and professional experience.
`
`4.
`
`I am a Teaching Professor in the School of Electrical Engineering and
`
`Computer Science at Pennsylvania State University, University Park, PA (“Penn
`
`State” or “PSU”). I have worked on telecommunications and wireless networks,
`
`including Z-Wave, Bluetooth, Zigbee, Wi-Fi, NFC, AMPS, IS-95, CDMA2000,
`
`GSM, EDGE, UMTS/WCDMA, LTE, and 5G cellular systems since 1988.
`
`5.
`
`I received my B.S. in Electronics Engineering from the University of
`
`1
`
`Page 4 of 99
`
`

`

`Scranton in 1987. I received my M.S. in Electrical Engineering and Ph.D. in
`
`Acoustics from Penn State in 1991 and 2001, respectively.
`
`6.
`
`In 1988, after I received my bachelor’s degree, I joined the Central
`
`Intelligence Agency (CIA) while pursuing my M.S. degree at Penn State part-time.
`
`My first job at the CIA involved designing and testing systems to automatically
`
`capture and characterize telecommunication signals and emissions from various
`
`wireless and computer networking devices.
`
`7.
`
`I returned to Penn State in early 1990 to pursue graduate research full-
`
`time and complete my M.S. degree. My graduate research work focused on
`
`wideband beamforming and adaptive signal processing. After completing my M.S.
`
`degree in EE in 1991, I accepted a full-time faculty research position at the Applied
`
`Research Lab at PSU, primarily working on classified programs, and began working
`
`on diverse radio frequency and acoustic sensor systems including wireless
`
`communications and small wireless networks for acoustic tracking, source
`
`localization, and feature extraction.
`
`8.
`
`I began pursuing my Ph.D. part-time in 1993 while continuing my
`
`faculty research position. In 1997, as part of my faculty research position, I began
`
`working on classified programs focused on mathematical analytical modeling of
`
`cellular communication networks and the development of hardware and software
`
`systems to test against cellular networks. My role was to develop the algorithms and
`
`2
`
`Page 5 of 99
`
`

`

`write the code running on a specially developed embedded system. For this work, I
`
`received a letter of recognition as the “genius behind the VELA software algorithms”
`
`from the Director of National Reconnaissance Office (NRO) Systems Engineering
`
`and Technology Office. As part of this same work, I was extensively involved in
`
`protocol and signaling analysis as well as researching model-specific performance
`
`and unique functional characteristics associated with individual mobile devices. The
`
`work involved testing dozens of handsets from many manufacturers in controlled
`
`and real-world environments against network simulators and live operational
`
`networks for each research project.
`
`9.
`
`In 2000, my research extended into utilizing non-orthogonal wavelets
`
`for improving detection and localization of cellular handsets from high altitude
`
`sensor systems. In 2001, I completed my Ph.D. and my research focused on the
`
`utilization of advanced communication signals for wideband characterization and
`
`remote sensing of propagation channels.
`
`10. Beginning in 1997, my cellular communications research work focused
`
`on CDMA, GSM, EDGE, UMTS, LTE, and 5G cellular systems primarily under
`
`grants sponsored principally by the Department of Defense. This classified research
`
`work required 3GPP protocol analysis and development of real-time embedded
`
`hardware and software systems capable of interacting with cellular networks and
`
`cellular handsets. A large portion of my work was directed at architectures,
`
`3
`
`Page 6 of 99
`
`

`

`protocols, software, and signaling.
`
`11.
`
`I have been working on classified projects since 1988. Before 1998,
`
`because the work was not deemed highly classified, I was able to publish eight
`
`journal and conference papers prior to 2000. Between 1999 and 2015, however, I
`
`was allowed to publish only one article in an unclassified symposium and published
`
`and presented about a dozen articles in classified settings. This is because during this
`
`period, the vast majority of my research was highly classified. As a result, nearly all
`
`of my research results were summarized in classified reports and not available to the
`
`general public. Further, because the U.S. government owns any intellectual property
`
`resulting from the sponsored research work, I did not pursue or file patent
`
`applications.
`
`12. Beginning in 2003, I was co-principal investigator and technical lead
`
`on a 3 year multi-million-dollar research effort for developing the Global
`
`Information Grid (GIG). This project was sponsored by the Secretary of Defense’s
`
`Office with a goal of developing a real-time, multi-intelligence (multi-Int) network
`
`for collecting, processing, storing, disseminating, and managing information on
`
`demand for decision makers including the warfighter, combatant command centers,
`
`policy makers, and support personnel and was the largest network-centric warfare
`
`project in development at the time. My research team (Ubiquitous Automated
`
`Information Manager) focused on building and deploying a scalable application to
`
`4
`
`Page 7 of 99
`
`

`

`perform real-time, multi-int data fusion to support every user in the system. This
`
`software application was deployed in Combat Operation Centers, Joint Interagency
`
`Task Force locations, and on various platforms (mobile and small computing
`
`environments) used by various warfighters. The fused data sources included various
`
`content management systems, supply chain logistic reports, GPS-based reports, new
`
`feeds, backend databases, sensor system reports, and various other broad data
`
`sources.
`
`13. Beginning in 1994, I worked on various projects that involved the
`
`implementation and design of user interfaces to support the use of the research,
`
`design, analysis tools and signal processing systems for the end user. The user
`
`platforms included a broad array of footprints including, small handheld devices
`
`with screens or visual displays and those limited to audio-only inputs, medium sized
`
`laptops and small desktop systems, and also large, heavy devices, such as servers
`
`and clusters in operational environments (including combat operations centers and
`
`3D immersive environments).
`
`14. Specifically, for the GIG research effort, I led a team of software
`
`engineers to deploy our inferencing tool for a disparate range of end users, from
`
`dismounted combat soldiers with only a microphone as an interface to soldiers in
`
`mules with smart devices (tablets and laptops with multiple visual display interaction
`
`capabilities) to field command centers (medium sized clients with significant video
`
`5
`
`Page 8 of 99
`
`

`

`displays and processing power with touch and stylus interfaces to the ) to combat
`
`operation centers with high-powered processing and complex user-interface
`
`capabilities/modalities.
`
`15.
`
`In 2015, I transferred to the School of Electrical Engineering and
`
`Computer Science at Penn State as a teaching faculty member. In that role, I have
`
`continued teaching graduate and undergraduate courses, guiding Ph.D. and M.S.
`
`students in communication and mobile networking (including LTE and 5G cellular
`
`networks), and pursuing research in this and related areas. Since 2015, I have been
`
`an author on nine refereed papers as listed in my curriculum vitae (CV).
`
`16. Because of my decades of research and my continuing work at Penn
`
`State, I have intimate knowledge of telecommunication networks, including the
`
`technology involved in the patents in this case. I have been highly recognized as an
`
`expert in such systems within the research community. I was recognized twice by
`
`the National Reconnaissance Office with commendation letters for work dealing
`
`with detecting cellular signals in low signal to noise ratio environments. The U.S.
`
`government awarded me over $12M in grants between 2003 and 2015 for projects
`
`focused on mobile communication devices and networks, in which I served as a
`
`Principal Investigator (PI), Co- PI, and/or technical lead.
`
`17. Additionally, during my research career, I interacted extensively with
`
`computer scientists and engineers responsible for the design, development, and
`
`6
`
`Page 9 of 99
`
`

`

`testing of telephony and data networking systems and testbeds. As a research faculty
`
`member, I oversaw engineers and computer scientists that executed many joint
`
`projects with development organizations. These interactions exposed me to a wide
`
`range of computer scientists and engineers working on telecommunication network
`
`technologies. Since 2011, I have been teaching undergraduate and graduate classes
`
`in communication and mobile networking and am familiar with the curricula being
`
`taught to electrical engineers and computer scientists. The interactions with a wide
`
`range of computer scientists and engineers working on telecommunication network
`
`technologies and the familiarity with the classes taught to electrical engineers and
`
`computer scientists have allowed me to have a good understanding of the level of
`
`skills possessed by a person of ordinary skill in the field of cellular technology.
`
`18.
`
`I have extensive experience with mobile networks in general and LTE
`
`and 5G specifically. While most of my research efforts between 1998 and 2015 were
`
`highly classified, I can state that they included detailed investigation of network
`
`architectures, signaling, and functional behavior. A typical research effort would
`
`involve studying 3GPP, 3GPP2, IEEE, and other protocol standards to fully
`
`comprehend all aspects of L1, L2, and L3 requirements including timing, bit-level
`
`construction of the control and user plane messages, and timing characteristics for a
`
`given standard as well as functional behavior of network components and user
`
`equipment.
`
`7
`
`Page 10 of 99
`
`

`

`19. From 2006 through 2015, my research focused specifically on LTE. My
`
`research continues to this day, although I am no longer operating in a classified
`
`environment. During this time, I investigated the performance and functional
`
`differences of many varied network and handset devices to see how differing
`
`signaling and hardware configurations (including MIMO) and environmental factors
`
`influenced the behavior of user equipment in a given network environment. This
`
`included how diversity techniques (transmit and receive), synchronization, timing,
`
`and signal to interference plus noise ratio (SINR) for a given device would affect
`
`specific functional aspects including elements of the receiver structure, decoding and
`
`demodulation performance, calculation of parameters used by the device for making
`
`decisions and deriving parameters reported to the network.
`
`20. As part of my research work, I built several custom CDMA, GSM,
`
`UMTS, and LTE platforms that implemented specific network-side and user
`
`equipment-side functionality including custom signal generation and processing
`
`structures, particularly the signal processing chains on both the transmit and receive
`
`sides. This equipment was developed using network simulation hardware in a
`
`laboratory environment and was later tested with corresponding networks in both
`
`controlled and fully operational environments. Implementing the transmit and
`
`receive chains for custom built protocol-enabled equipment required me to gain an
`
`intimate understanding of the relevant 3GPP protocol specifications and the
`
`8
`
`Page 11 of 99
`
`

`

`underlying structures. Since 2015, I have been primarily focused on guiding
`
`graduate students pursuing research including using code domain non-orthogonal
`
`multiple access (NOMA) combined with MIMO sparse coding multiple access to
`
`minimize latency and maximize user density in grant free Internet of Things (IoT)
`
`environments. Additionally, I am guiding my graduate students in pursuing research
`
`in optimized distributed processing algorithms, implementation of block chain
`
`coding techniques to improve handover security, and edge computing resource
`
`allocation in 4G (LTE)/5G (NR) networks.
`
`21. Much of the classified research work I performed also led to similar
`
`approaches for other wireless protocols including IEEE 802.11 and 802.15 (e.g.,
`
`Zigbee, Bluetooth, and UWB), HART, and other short-range standards as well as
`
`HF radio and Wi-MAX.
`
`22. Throughout my career, I have accumulated significant expertise in the
`
`area of user interface design and the development of software and software tools for
`
`implementing and facilitating efficient user interaction, dissemination of complex
`
`information, and simple, intuitive user interfaces for screen displays.
`
`III. COMPENSATION
`
`23. My work on this matter is being billed at my customary rate of $550 an
`
`hour. Also, I am being reimbursed for reasonable expenses I incur in relation to my
`
`services. I have no pecuniary interest in the outcome of this proceeding. I understand
`
`9
`
`Page 12 of 99
`
`

`

`I will be paid regardless of the outcome of any proceeding in which my work is used.
`
`IV. LEGAL CONSIDERATIONS
`
`24.
`
`I am not an attorney and have not been asked to offer my opinion on
`
`the law. However, as an expert offering an opinion on whether the claims of the ’581
`
`Patent are patentable, I have been told that I am obliged to follow existing law. My
`
`understanding of the law is based on information provided by counsel.
`
`25.
`
`I understand that in an inter partes review proceeding, a patent claim is
`
`unpatentable if it is shown by a preponderance of the evidence that the claim is
`
`anticipated or obvious. I also understand that the anticipation or obviousness
`
`determination is viewed from the perspective of a person of ordinary skill in the art
`
`(“POSITA”).
`
`A. Anticipation
`
`26.
`
`I understand that a reference anticipates a claim if it discloses each and
`
`every element recited in the claim, arranged as in the claim, so as to enable a
`
`POSITA to make and use the claimed invention without the need for undue
`
`experimentation in light of the general knowledge available in the art. I understand
`
`that a U.S. Patent document is presumed to have sufficient description to include
`
`sufficient detail for a POSITA to make and use the subject matter that document
`
`describes.
`
`27.
`
`I also understand that a claim limitation may be present in an alleged
`
`10
`
`Page 13 of 99
`
`

`

`prior art reference either expressly or inherently, but inherency may be established
`
`only if such limitation is necessarily present in the reference. I understand inherency
`
`cannot be established by probabilities or possibilities, and the mere fact that a certain
`
`thing may result from a given set of circumstances is not sufficient to establish
`
`inherency.
`
`B. Obviousness
`
`28. For a claim to be obvious, I understand that every limitation of the claim
`
`must be disclosed in a combination of prior art references, as viewed from the
`
`perspective of a POSITA. I understand that a POSITA is presumed to know the
`
`relevant prior art.
`
`29.
`
`I understand that while there is no rigid rule that a reference or
`
`combination of references must contain a “teaching, suggestion, or motivation” to
`
`combine references, some motivation to combine or modify prior art references must
`
`exist in the references or the knowledge generally available to a POSITA, otherwise
`
`the challenged claims would not have been obvious. I also understand that the
`
`“teaching, suggestion, or motivation” test can be a useful guide in establishing a
`
`rationale for combining elements of the prior art. In addition, I have been informed
`
`and I understand that there must be a reasonable expectation of success in combining
`
`prior art teachings to arrive at the claimed invention.
`
`30.
`
`I understand that the relevant time for considering whether a claim
`
`11
`
`Page 14 of 99
`
`

`

`would have been obvious to a POSITA is the time of the effective filing of the ’581
`
`Patent.
`
`C. Claim Interpretation
`
`31.
`
`I understand that a claim term is interpreted according to its ordinary
`
`and customary meaning as a POSITA would have understood the term in light of the
`
`surrounding claim language, other claims, the specification, and the patent’s
`
`prosecution history, which are referred to as intrinsic evidence. I also understand that
`
`prior art references cited in the patent’s prosecution history are considered intrinsic
`
`evidence. I further understand that evidence outside the patent and its prosecution
`
`history (e.g., dictionaries and technical articles, as well as the POSITA’s general
`
`knowledge and experience) may inform the context in which a POSITA would have
`
`understood the claims of a patent. I understand this ordinary and customary meaning
`
`applies absent unique circumstances, such as where a patent clearly expresses an
`
`intent to set forth a special meaning for a term.
`
`V. TASK SUMMARY
`
`32.
`
`I have been asked to review the challenged U.S. patent: the ’581 Patent.
`
`I have been asked to provide my opinions from the perspective of a POSITA, having
`
`knowledge of the relevant art, as of August 5, 2011, and the opinions stated in this
`
`declaration are from that perspective. The qualifications and abilities of such a
`
`person are described in Section VII below.
`
`12
`
`Page 15 of 99
`
`

`

`33.
`
`In preparing this declaration, I have considered this patent in its entirety
`
`and the general knowledge of those familiar with the field of graphical user
`
`interfaces, as of August 5, 2011. My opinions are based on my education, training,
`
`and experience, as well as the references that form the basis for Petitioner’s
`
`challenge to the ’581 Patent, which I have reviewed in their entirety, including the
`
`publications listed in the following table:
`
`Exhibit
`1001
`1002
`1003
`1004
`
`1005
`
`1006
`
`1007-
`1019
`1020
`
`1021
`
`1022
`1023
`1024
`1025
`1026
`1027
`1028
`
`1029
`
`Description
`U.S. Patent No. 10,725,581 (“’581 Patent”)
`U.S. Patent No. 10,725,581 File History (“’581 File History”)
`Declaration of Dr. Andrew Cockburn (“Cockburn Decl.”)
`U.S. Patent Publication No. 2008/0259041 to Blumenberg et al.
`(“Blumenberg”)
`U.S. Patent Publication No. 2008/0235594 to Bhumkar et al.
`(“Bhumkar”)
`U.S. Patent Publication No. 2011/0248948 to Griffin et al.
`(“Griffin”)
`RESERVED
`
`Newton MessagePad, Models: original (OMP) and 100 (MP100),
`https://web.archive.org/web/19990202045806/
`https://msu.edu/~luckie/gallery/mp100.htm (1999) (“Newton”)
`
`Of mice and menus: designing the user-friendly interface, IEEE
`Spectrum (1999)
`Newton 2.0 User Interface Guidelines (1996)
`US 7,479,949
`US 8,954,887
`US 2009/0244023
`US 4,885,565
`Active Click: Tactile Feedback for Touch Panels (2001)
`Snap-Crackle-Pop: Tactile Feedback for Mobile Touch Screens
`(2006)
`Tactile Feedback for Mobile Interactions (2007)
`
`13
`
`Page 16 of 99
`
`

`

`1030
`
`1031
`1032
`1033
`1034
`1035
`1036
`1037
`
`1038
`
`1051
`1052
`1053-
`1096
`1097
`
`1098
`
`1099
`
`
`
`Design and Evaluation of Identifiable Key-Click Signals for
`Mobile Devices (2011)
`US 2006/0197753
`Issues and Techniques in Touch-Sensitive Tablet Input (1985)
`US 5,612,719
`US 2008/0036743
`US 8,830,181
`US 9,760,241
`Android Developers Guide, Creating Menus,
`https://web.archive.org/web/20110216140646/
`http://developer.android.com/guide/topics/ui/menus.html (2011)
`The State of the Art of Mobile Application Usability Evaluation
`(2012)
`Usability Heuristics for Touchscreen-based Mobile Devices (2012)
`1039
`1040 Web Usability: A Review of the Research (1999)
`1041
`US 11/367,749
`1042
`Evolution of Mobile Devices and Operating Systems (2013)
`1043
`US 2008/0094368
`1044
`US 2008/0168405
`1045
`iPhone User Guide: For iPhone OS 3.1 Software (2009)
`1046-
`RESERVED
`1049
`1050
`
`Xlib—C Language X Interface, X Window System Standard
`(2002) (excerpts)
`US 2013/0007665
`US 2011/0252380
`RESERVED
`
`United States District Courts — National Judicial Caseload Profile
`(Dec. 31, 2023) | United States Courts (uscourts.gov),
`https://www.uscourts.gov/sites/default/
`files/fcms_na_distprofile 1231.2023_0.pdf
`(last visited June 25, 2024)
`Memorandum, Interim Procedure for Discretionary Denials in AIA
`Post-Grant Proceedings with Parallel District Court Litigation
`(USPTO June 21, 2022) (“Guidance”)
`Case Management Order, Smith Interface Technologies, LLC v.
`Apple Inc., 3-23-cv-01187, S.D. Cal. (June 20, 2024)
`
`14
`
`Page 17 of 99
`
`

`

`VI. OVERVIEW OF THE ’581 PATENT
`
`34. The ’581 Patent “generally relate[s] to touch screens.” EX1001, 1:37–
`
`38. Specifically, the ’581 Patent focuses on improving user interfaces for
`
`increasingly complex touchscreen devices. EX1001, Abstract. By the time of the
`
`’581 Patent’s December 2011, the priority date of the ’581 Patent, touch screen
`
`mobile devices were becoming increasingly capable of handling more complex
`
`tasks, necessitating more advanced control methods than basic input methods could
`
`provide. Due to limited screen space, input and output techniques that had been used
`
`on desktop computers were not feasible for use on these new mobile devices. The
`
`’581 Patent resolves this issue by enabling users to operate the mobile devices and
`
`feature-rich applications through multi-part gestures, allowing for successive user
`
`actions and feedback at each stage. EX1001, 14:26-26:21, Abstract.
`
`A. Multi-Part-Pattern Based Gestures
`
`35. The ’581 Patent refers to multi-part gestures as “pattern-based
`
`gestures” that “may be recognized in the form of an ordered sequence,” such as
`
`“using a decision tree.” EX1001, 73:45-47; EX2011, ¶[00581]. These gestures are
`
`further defined as “a path through a phase space of one or more dimensions.”
`
`EX1001, 71:48-50; EX2011, ¶[00564]. The patent also notes that “probabilistic
`
`pattern recognition algorithms may output a probability…[where] this probability
`
`may be used to alter or modify a selection.” EX1001, 74:15-22; EX2011, ¶[00585].
`
`15
`
`Page 18 of 99
`
`

`

`For instance, the patent explains that “pattern-based gesture 2302” is “described by
`
`a phase space path which represents the time evolution of the touch state associated
`
`with a contact point” on the touchscreen. EX1001, 71:62-65; EX2011, ¶[00566].
`
`
`
`EX1001, FIG. 23; EX2011, FIG. 28.
`
`36. Each state and the transitions between them can be defined by various
`
`factors, including “the time spent on each leg of the path through phase space.”
`
`EX1001, 72:1-28; EX2011, ¶¶[00567]–[00570]. For example, if the touch duration
`
`does not exceed the time threshold t1, FIG. 23 shows that the device remains in the
`
`same state, S0. EX1001, 72:1-9; EX2011, ¶[00567]. However, if the touch duration
`
`surpasses threshold t1, the device transitions to a different state, S1. Id. Likewise, if
`
`the duration exceeds threshold t2, the device proceeds to state S2. Id.
`
`1.
`
`Figure 16
`
`37. The ’581 Patent describes a mobile device configured to detect pattern-
`
`based gestures using various components, including a touch screen that receives
`
`signals generated by contact with the screen. EX1001, 13:61–67, 14:7, 14:31–53,
`
`16
`
`Page 19 of 99
`
`

`

`Claim 1 Limitations [1.pre]–[1.c]; EX2011, ¶¶[0077]–[0078], [0082]. In response to
`
`these touch signals, the device may alter its output or perform processing actions.
`
`EX1001, 15:39–51, FIG. 3; EX2011, ¶[0089]. To recognize whether a user’s contact
`
`corresponds to a predefined gesture, the ’581 Patent explains the use of decision-
`
`tree-based programming logic, as illustrated in FIG. 16. This logic enables the device
`
`to detect a particular gesture and execute the action associated with it. EX1001,
`
`73:45–47, 83:63–65, FIG. 26; EX2011, ¶¶[00507], [00581], FIG. 24. Figure 16
`
`presents decision-tree-based programming logic that allows the device to determine
`
`whether a user’s contact with the touchscreen corresponds to “a selection made
`
`within a user interface.” EX1001, 53:7–54:34, FIG. 16. As shown in Figure 16, the
`
`logic begins with detecting a touch event and identifying the associated “touch event
`
`attributes.” EX1001, 53:7–27, Claim 1 Limitations [1.h]–[1.l]; EX2011, ¶¶[00355]–
`
`[00357]. These “touch event attributes” may include any measurable characteristic
`
`of the user’s interaction with the touchscreen, such as touch state, contact area,
`
`contact point, and attributes tracked over time. EX1001, 53:28–34, Claim 1
`
`Limitations [1.h]–[1.l]; EX2011, ¶[00359]; see also EX1001, 56:45–16 (“Possible
`
`gesture inputs may include, but are not limited to, touch event attributes (both initial
`
`and over time)”); EX2011, ¶[00386].
`
`17
`
`Page 20 of 99
`
`

`

`
`
`EX1001, FIG. 16; EX2011, FIG. 16. As shown in Figure 16, once the attributes of
`
`the initiating touch event are determined, the system proceeds to step 1606 to assess
`
`whether it is “in a selection mode.” EX1001, 54:35–36. If so, “a selection is created
`
`and / or modified as a function of the touch event attributes [and] then displayed.”
`
`Id., 54:53–55. However, “if it is determined in 1606 that the system is not in a
`
`selection mode,” the system moves to step 1610 to determine whether “a gesture has
`
`been detected.” EX1001, 54:12–15, Claim 1 Limitations [1.m.ii]–[1.m.iii]; EX2011,
`
`¶[00368]. If no gesture is detected at step 1610, the system continues looping
`
`through gesture detection without executing an associated action. EX1001, 54:12–
`
`18
`
`Page 21 of 99
`
`

`

`18, FIG. 16, Claim Limitations [1.m.ii]–[1.m.iii]; EX2011, ¶[00368]. The ’581
`
`Patent further states that “[i]f a gesture is detected, then an associated action is
`
`performed (step 1612).” EX1001, 54:19–20. It explains that a detected gesture may
`
`correspond to a specific command or function within the device’s programming
`
`logic, such as “scrolling a display or user interface element, zooming in or out,
`
`modifying the display brightness,” or “placing the system in a selection mode.”
`
`EX1001, 54:20–34, Claim Limitation [1.n.ii]-[1.n.vi]; EX2011, ¶¶[00369]–[00370].
`
`The patent also notes, that “[i]f the system is in a selection mode, then a selection is
`
`created and/or modified as a function of the touch event attributes…” EX1001,
`
`53:53–55.
`
`38. With regard to determining whether a touch event qualifies as a gesture,
`
`the ’918 Provisional—incorporated by reference into the ’581 Patent (EX1001,
`
`1:24–26)—describes a process called “gesture differentiation,” which goes beyond
`
`merely identifying a “basic user interaction (i.e., button press, basic application
`
`interaction, etc.).” EX2011, ¶¶[00498], [00502]. This process begins by assessing
`
`“whether there is any meaningful user input,” which may involve applying a
`
`“threshold above which user interaction may rise before it may be deemed
`
`meaningful.” Id., ¶¶[00501]–[00502]. Gesture detection can also take into account
`
`“gesture subject matter context,” such as “objects displayed at or ne