`
`UNITED STATES PATENT AND TRADEMARK OFFICE
`______________
`
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`______________
`
`APPLE INC.,
`
`Petitioner
`
`
`v.
`
`
`SMITH INTERFACE TECHNOLOGIES, LLC
`Patent Owner
`
`______________
`
`Case IPR2024-01090
`
`Patent No. 10,725,581
`______________
`
`
`DECLARATION OF MARC DAVIS, Ph.D.
`
`
`
`
`
`
`
`Apple Inc. v. Smith Interface Techs., LLC
`IPR2024-01090 | Smith EX2001
`Page 1 of 80
`
`

`

`
`
`TABLE OF CONTENTS
`INTRODUCTION ................................................................................................................... 1
`I.
`II. PROFESSIONAL BACKGROUND ....................................................................................... 1
`III. COMPENSATION .................................................................................................................. 7
`IV. LEGAL CONSIDERATION ................................................................................................... 7
`A. Anticipation.......................................................................................................................... 7
`B. Claim Interpretation ............................................................................................................. 8
`C. Overview of Task & Basis for Opinions.............................................................................. 8
`D. Level of Skill in the Relevant Art ...................................................................................... 10
`V. OVERVIEW OF THE ’581 PATENT .................................................................................. 10
`A. Independent Claims 1 and 42 ............................................................................................. 12
`VI. OVERVIEW OF THE PRIOR ART ..................................................................................... 16
`A. U.S. Patent App. No. 2008/0259041 (“Blumenberg”)....................................................... 16
`1.
`Inadvertent Touch Functionality .................................................................................... 16
`2. Hyperlink Preview Functionality ................................................................................... 18
`B. U.S. Patent App. No. 2008/0259041 (“Bhumkar”) ........................................................... 26
`C. U.S. Patent App. No. 2011/0248948 (“Griffin”) ............................................................... 29
`VII. GROUND 1: THE PETITION FAILS TO SHOW THE BLUMENBERG-BHUMKAR-
`GRIFFIN COMBINATION TEACH THE ELEMENTS OF CLAIMS 1 AND 42 ..................... 31
`A. [1.m.iii] – avoid display, on the first web page, a result of any operation based on the first
`contact ....................................................................................................................................... 32
`1. Dr. Cockburn’s Theory is Absent From and Contradicted by Blumenberg................... 33
`2. Griffin Fails to Cure the Deficiencies of Blumenberg ................................................... 38
`B. [1.o] .................................................................................................................................... 41
`1. Dr. Cockburn’s Combinations and Claim Interpretations .............................................. 43
`2. Dr. Cockburn’s Initial Modification Does Not Disclose the Claimed Limitations
`([1.o.vii]-[1.o.viii]) ................................................................................................................ 48
`3. Dr. Cockburn’s Alternative Modification is Internally Inconsistent and Cannot Meet All
`the Claim Elements ................................................................................................................ 50
`4. A POSITA Would Not Have Made Dr. Cockburn’s Alternative Modification – Link
`Information ............................................................................................................................ 51
`5. A POSITA Would Not Have Made Dr. Cockburn’s Alternative Modification – Menu 53
`
`- i -
`
`Page 2 of 80
`
`

`

`
`
`C. A POSITA Would Not Have Been Motivated to Modify Blumenberg’s Link Information
`to Have a Menu ......................................................................................................................... 58
`1. Petitioner’s Modification is Illogical and Inconsistent with a POSITA’s Understanding
`
`59
`2. Petitioner’s Motivations Do Not Resolve the Incongruities in the Purported
`Modification .......................................................................................................................... 66
`VIII. CONCLUSION .................................................................................................................... 77
`
`
`
`
`
`- ii -
`
`Page 3 of 80
`
`

`

`
`
`I.
`
`INTRODUCTION
`1. My name is Dr. Marc Davis, Ph.D. I have been retained by Smith
`
`Interface Technologies, LLC as an independent expert consultant in this inter partes
`
`review (“IPR”) proceeding before the United States Patent and Trademark Office
`
`(“PTO”).
`
`II.
`
`PROFESSIONAL BACKGROUND
`All of my opinions stated in this Declaration are based on my own
`2.
`
`personal knowledge and professional judgment. In forming my opinions, I have
`
`relied on my knowledge and experience in designing, developing, researching, and
`
`teaching the technology referenced in this Declaration.
`
`3.
`
`I am over 18 years of age and, if I am called upon to do so, I would be
`
`competent to testify as to the matters set forth herein. I understand that a copy of
`
`my current curriculum vitae, which details my education and professional and
`
`academic experience, is being submitted as EX2002. The following provides a brief
`
`overview of some of my experience that is relevant to the matters set forth in this
`
`Declaration.
`
`4.
`
`In 1984, I received my Bachelor of Arts degree in College of Letters at
`
`Wesleyan University in Middletown, Connecticut. I then received my Master of
`
`Arts degree in Literary Science and Philosophy in 1987 at the University of
`
`Konstanz in Germany. During my time at the University of Konstanz, I also worked
`
`IPR2023-01442
`
`- 1 -
`
`Patent No. 7,020,667
`
`Page 4 of 80
`
`

`

`
`
`as a Research Coordinator in the Literary Science department where I initiated and
`
`managed computerization of a large scale, multiyear Intertextuality Research
`
`Project. Later, in 1990, I pursued my doctorate in Media Arts and Sciences at the
`
`Massachusetts Institute of Technology (MIT) Media Laboratory. This program
`
`focuses on the invention, study, and creative use of new technologies and draws on
`
`a number of other disciplines, including computer science, cognitive sciences,
`
`communications, design, and the expressive arts. For 5 years, I also worked as a
`
`research assistant in the MIT Media Laboratory, where I worked on various
`
`multimedia, user interface, and AI technologies in digital applications with a focus
`
`on iconic visual language interfaces and systems for video annotation, retrieval, and
`
`repurposing. I received my Doctor of Philosophy in Media Arts and Sciences from
`
`MIT in 1995.
`
`5.
`
`For 5 years, partially overlapping my PhD program, I served as a
`
`Project Coordinator for Interval Research Corporation, a research laboratory created
`
`by Paul Allen, the Co-Founder of Microsoft. My role included leading and
`
`contributing to the technical and user interface development of digital media
`
`technologies,
`
`including media automation applications, visual
`
`language
`
`programming tools, and media interfaces and systems. During this time, my
`
`responsibilities also included researching other digital projects, including various
`
`IPR2024-01090
`
`- 2 -
`
`Patent No. 10,725,581
`
`Page 5 of 80
`
`

`

`
`
`interactive applications. It was at Interval Research where I first began patenting
`
`my inventions, leading to over 240 patents in my career.
`
`6.
`
`I then served as Chief Technology Officer and Co-Founder of Amova,
`
`which focused on the development of media automation and personalization
`
`technology. In my role, I was responsible for the technical research and development
`
`of innovative personalized media systems and applications. My focus included the
`
`design of user interfaces for multimedia websites and location-based live interactive
`
`multimedia systems.
`
`7.
`
`From 2002 to 2006, I served as Assistant Professor at the UC Berkeley
`
`School of Information where I initiated and directed various media related research
`
`and development projects that involved user interfaces and user experiences among
`
`various technologies, in particular for mobile devices. My mobile research projects
`
`pioneered technologies and interfaces for capturing, tagging, and sharing
`
`photographs on mobile devices, including using the earliest smartphone with a
`
`camera introduced in the United States. The Mobile Media Metadata projects
`
`(MMM and MMM2) developed context-aware mobile media
`
`technology,
`
`applications, and user interfaces that leveraged contextual metadata—spatial,
`
`temporal, and social—to infer media content and support media tagging, sharing,
`
`and reuse, including breakthrough work in context-aware mobile face and place
`
`recognition. In addition, our project collaborated closely with social science and
`
`IPR2024-01090
`
`- 3 -
`
`Patent No. 10,725,581
`
`Page 6 of 80
`
`

`

`
`
`design research to understand how and why people use mobile media technologies.
`
`These Mobile Media Metadata Data projects were sponsored by numerous
`
`commercial leaders in mobile and media technologies, including British Telecom,
`
`Hewlett-Packard, France Telecom, Nokia, Ricoh Innovations, AT&T, Opera
`
`Software, TEKES, and the UC Discovery Grant for Digital Media. Other projects
`
`that I directed at the UC Berkeley School of Information included: Media Streams
`
`Metadata Exchange (media metadata framework for annotating, retrieving, sharing,
`
`and remixing video on the Web); Active Capture (interactive cameras that use signal
`
`processing and computer-human interaction to capture high quality, reusable,
`
`annotated media assets); and Adaptive Media (adaptive media templates and
`
`automatic editing functions to mass customize and personalize media). In my time
`
`at UC Berkeley, I also served as Co-Founder and Executive Committee Member of
`
`the interdisciplinary UC Berkeley Center for New Media, an Advisory Board
`
`Member of the Art, Technology, and Culture Colloquium, and an Affiliated Faculty
`
`Member of the Berkeley Institute of Design. In addition to supervising doctoral and
`
`master’s degree students at the UC Berkeley School of Information, I taught courses
`
`in “Information Organization and Retrieval”, “Multimedia Information”, “Digital
`
`Media Design Studio”, and “Foundations of New Media”, all of which involved
`
`teaching and mentoring students in user interface design.
`
`IPR2024-01090
`
`- 4 -
`
`Patent No. 10,725,581
`
`Page 7 of 80
`
`

`

`
`
`8.
`
`In 2005, I founded and co-directed Yahoo! Research Berkeley, a joint
`
`research laboratory in partnership between UC Berkeley and Yahoo!, which
`
`produced a number of innovative public prototypes in mobile media and social
`
`media technology and applications. The development of these prototypes included
`
`extensive exploration of user and design interfaces, mobile computing, and media
`
`technology. From 2006-2009, I transitioned to the role of Chief Scientist and Vice
`
`President of Early Stage Products (ESP) for Yahoo! Connected Life (which included
`
`Yahoo!’s mobile and interactive television divisions), and I also served as Social
`
`Media Guru at Yahoo!, including working closely with Yahoo! CEO and Co-
`
`Founder Jerry Yang. My work at Yahoo! focused in particular on creating the
`
`technology, applications, and user interfaces for mobile devices and systems
`
`including mobile search, mobile social communications applications, and mobile
`
`advertising. In addition, I was very actively involved in patenting my inventions,
`
`producing over 100 patents while at Yahoo!.
`
`9.
`
`I then served as Chief Scientist for Invention Arts where I focused on
`
`new approaches to how people can better control, share, and benefit from their
`
`personal data as well as new methods of strategic intellectual property development.
`
`At Invention Arts, I worked with the World Economic Forum’s Rethinking Personal
`
`Data project. After my time at Invention Arts, I worked as a Partner Architect for
`
`Microsoft’s Online Services Division. I and my team continued to work on early-
`
`IPR2024-01090
`
`- 5 -
`
`Patent No. 10,725,581
`
`Page 8 of 80
`
`

`

`
`
`stage products and developed technology involving personal data, privacy, social
`
`computing, and personalized search. This technology included the development of
`
`user interface (UI) and the software architecture underlying the UI. I also worked
`
`as an elite inventor at Microsoft including co-inventing with Microsoft Co-Founder
`
`and Chairman Bill Gates and current Microsoft CEO Satya Nadella resulting in
`
`numerous patents.
`
`10.
`
`I am the inventor or coinventor on over two hundred and forty (240)
`
`granted patents, involving various computing environments, especially mobile and
`
`media systems, applications, and user interfaces. In my time at the MIT Media
`
`Laboratory, Interval Research, Amova, UC Berkeley, Yahoo!, Invention Arts, and
`
`Microsoft, I worked on innovative technologies, applications, and user interfaces for
`
`media automation, context-aware computing, computer vision, mobile computing,
`
`social and mobile media, social and mobile search, sociotechnical systems, mobile
`
`and social advertising, and personal data technologies and applications. During my
`
`time teaching and working with these entities, I was also a member of the Institute
`
`of Electrical and Electronics Engineers Computer Society (IEEE), Association for
`
`Computing Machinery (ACM), and American Association for Artificial Intelligence
`
`(AAAI).
`
`11. My curriculum vitae is attached as Attachment A and includes a listing
`
`of my research publications and patents on which I am an inventor. To help gauge
`
`IPR2024-01090
`
`- 6 -
`
`Patent No. 10,725,581
`
`Page 9 of 80
`
`

`

`
`
`the impact of my work, it is important to note that my patents and research
`
`publications have earned an h-index of 88, demonstrating significant impact, and an
`
`i10-index of 237, reflecting the wide scope of my contributions (Google Scholar).
`
`III. COMPENSATION
`12. My work on this matter is being billed at my customary rate of $400 an
`
`hour. Also, I am being reimbursed for reasonable expenses I incur in relation to my
`
`services. I have no pecuniary interest in the outcome of this proceeding. I
`
`understand I will be paid regardless of the outcome of any proceeding in which my
`
`work is used.
`
`IV. LEGAL CONSIDERATION
`I am not an attorney. My understanding of the law is based on
`13.
`
`information provided by counsel for Smith Interface Technologies, LLC.
`
`A. Anticipation
`I understand that a reference anticipates a claim if it discloses each and
`14.
`
`every element recited in the claim, arranged as in the claim, so as to enable a person
`
`of ordinary skill in the art (POSITA) to make and use the claimed invention without
`
`the need for undue experimentation in light of the general knowledge available in
`
`the art. I understand that a U.S. Patent document is presumed to have sufficient
`
`description to include sufficient detail for a person of ordinary skill in the art to make
`
`and use the subject matter that document describes.
`
`IPR2024-01090
`
`- 7 -
`
`Patent No. 10,725,581
`
`Page 10 of 80
`
`

`

`
`
`15.
`
`I also understand that a claim limitation may be present in an alleged
`
`prior art reference either expressly or inherently, but inherency may be established
`
`only if such limitation is necessarily present in the reference. I understand inherency
`
`cannot be established by probabilities or possibilities, and the mere fact that a certain
`
`thing may result from a given set of circumstances is not sufficient to establish
`
`inherency.
`
`B. Claim Interpretation
`I understand that a claim term is interpreted according to its ordinary
`16.
`
`and customary meaning as a POSITA would have understood the term in light of the
`
`surrounding claim language, other claims, the specification, and the patent’s
`
`prosecution history, which are referred to as intrinsic evidence. I also understand
`
`that prior art references cited in the patent’s prosecution history are considered
`
`intrinsic evidence. I further understand that evidence outside the patent and its
`
`prosecution history (e.g., dictionaries and technical articles) may inform the context
`
`in which a POSITA would have understood the claims of a patent. I understand this
`
`ordinary and customary meaning applies absent unique circumstances, such as
`
`where a patent clearly expresses an intent to set forth a special meaning for a term.
`
`C. Overview of Task & Basis for Opinions
`I have been asked to review U.S. Patent No. 10,725,581 (referred to as
`17.
`
`“the ’581 Patent”). I have been asked to provide opinions related to certain issues
`
`IPR2024-01090
`
`- 8 -
`
`Patent No. 10,725,581
`
`Page 11 of 80
`
`

`

`
`
`from the perspective of a person of ordinary skill, having knowledge of the relevant
`
`art, as of August 5, 2011, and—except where otherwise noted—the opinions stated
`
`in this declaration are from that perspective. The qualifications and abilities of such
`
`a person are described in paragraph 21 below.
`
`18. More particularly, I have been informed that this proceeding is
`
`currently at a preliminary stage during which the Patent Trial and Appeal Board (the
`
`Board) will decide whether the Petition establishes a reasonable likelihood of at least
`
`one claim of the ’581 Patent being invalid. At this preliminary stage, I have been
`
`asked to focus my analysis on the Petition’s proposed invalidity theories for the ’581
`
`Patent’s independent claims 1 and 42.
`
`19. My opinions are based on my education, training, and experience as
`
`well as items that I reviewed to prepare my opinions, including the Petition, the
`
`documents listed in the Exhibit List included with the Petition, and any other items
`
`that I reference in my below analysis.
`
`20.
`
`I understand that other issues may arise that require further explanation,
`
`and I will provide that explanation if appropriate. As a result, I respectfully reserve
`
`the right to update and supplement this declaration and the information and opinions
`
`provided herein.
`
`IPR2024-01090
`
`- 9 -
`
`Patent No. 10,725,581
`
`Page 12 of 80
`
`

`

`
`
`D. Level of Skill in the Relevant Art
`I understand that a patent must be analyzed from the perspective of a
`21.
`
`person of ordinary skill in the art (“POSITA”) as of the time of invention of the
`
`patent, which is typically considered to be the patent’s earliest filing date. While I
`
`do not necessarily agree with every aspect of Dr. Cockburn’s explanation of the level
`
`of ordinary skill in the art, I do not currently believe the aspects of that explanation
`
`I would change or clarify would impact the analysis set forth in this declaration. For
`
`purposes of this declaration, my opinions are therefore provided from the perspective
`
`of the POSITA described in paragraph 36 of Dr. Cockburn’s declaration (EX1003)
`
`as of August 5, 2011. This POSITA would have a Bachelor’s degree in computer
`
`science, computer engineering, or a comparable field and at least two years of
`
`professional experience working with graphical user interfaces and applications for
`
`computers. However, additional years of experience could substitute for the
`
`advanced-level degree. My education, technical expertise and personal knowledge
`
`discussed in Section II shows that I meet the qualifications of a POSITA.
`
`V. OVERVIEW OF THE ’581 PATENT
`22. The ’581 Patent is titled “Devices, methods and graphical user
`
`interfaces for manipulating user interface objects with visual and/or haptic feedback”
`
`and lists as its inventor Michael S. Smith. EX1001, Title. The ’581 Patent describes
`
`various user interface operations, such as those for portable electronic devices.
`
`IPR2024-01090
`
`- 10 -
`
`Patent No. 10,725,581
`
`Page 13 of 80
`
`

`

`
`
`EX1001, 13:51-14:11 (describing “hand-held computer, mobile phone, personal
`
`digital assistant (PDA), tablet computer” and depicting “a mobile device” in FIG.
`
`2). In operation, signals can be received in association with the touch interface (e.g.,
`
`touch screen) and can correspond to a signal indicative of a touch interface being
`
`touched. EX1001, 14:31-53. The user experience can then be altered utilizing these
`
`touch signals. EX1001, 15:39-51, FIG. 3.
`
`23. An illustration of this operation is shown in FIG. 18, where it is
`
`determined if a touch event has occurred (step 1802) and the touch event attributes
`
`are determined (step 1804). EX1001, 56:32-36. A gesture may then be identified
`
`(step 1806) from gesture inputs related to these touch event attributes (both initial
`
`and over time). EX1001, 56:37-60. The ’581 Patent explains that such inputs can
`
`include the duration of a gestures because often different gestures will be
`
`indistinguishable based on initial contact. EX1001, 56:37-60 (gestures “identified
`
`after the gesture has been performed for some period of time” and providing example
`
`of distinguishing between “a two finger pinch gesture and a two finger swipe”). In
`
`some embodiments, a contact pressure is also utilized to distinguish between
`
`gestures a well as to parametrize different intensities of the same gesture. EX1001,
`
`56:61-57:8. Once the gesture is identified, a corresponding operation can be
`
`performed (step 1808). EX1001, 57:8-10. Such operations can include the
`
`modification of one or more display attributes. EX1001, 57:35-36.
`
`IPR2024-01090
`
`- 11 -
`
`Patent No. 10,725,581
`
`Page 14 of 80
`
`

`

`
`
`24. The ’581 Patent also describes performing various operations based on
`
`a context sensitive embedded hyperlink in a web browser. EX1001, 30:44-49 (“once
`
`an appropriate [hyperlink] selection function is chosen, a selection may be defined
`
`based upon user input”). For example, a user can cause different operations to occur
`
`when selecting a hyperlink based on a parameter of a gesture (e.g., contact pressure
`
`level) being less than or greater than thresholds. EX1001, 31:39-52. (describing
`
`scenario where the contact pressure level surpasses a value of P1, but not P2). One
`
`such hyperlink operation can include utilizing a hyperlink contextual menu.
`
`EX1001, 31:39-52.
`
`A.
`25.
`
`Independent Claims 1 and 42
`Independent claim 1 of the ’581 Patent is directed to an “an electronic
`
`device” that has a display, a touch-sensitive interaction surface, an actuator, one or
`
`more processors, memory, and one or more programs. EX1001, 102:50-59. The
`
`programs are executed by the processor to cause various operations to be performed
`
`(via the execution of instructions within the program). EX1001, 102:50-59.
`
`26. These operations include to (1) display, on the display and via a user
`
`interface of a network browser application, a first web page including a hyperlink
`
`that identifies a second web page; (2) detect a first contact starting at a first contact
`
`point at a first contact time on the touch-sensitive interaction surface; (3) detect an
`
`end of the first contact at a second contact time; (4) determine a duration of the first
`
`IPR2024-01090
`
`- 12 -
`
`Patent No. 10,725,581
`
`Page 15 of 80
`
`

`

`
`
`contact as a difference between the first contact time and the second contact time;
`
`(5) determine whether there is a contact movement of the first contact between the
`
`first contact point and a second contact point; (6) and determine whether the first
`
`contact point of the first contact corresponds with a location of the hyperlink of the
`
`first web page. EX1001, 102:60-103:9. Based on the determinations and detections
`
`made during these operations, the programs will perform certain functions. In other
`
`words, the determinations and detections performed are utilized as prerequisite
`
`conditions to determine which resulting function will be executed. See e.g.,
`
`EX1001, 57:9-10 (“Once the gesture has been identified, the corresponding
`
`operation is performed”).
`
`27. Specifically, the claims include:
`
`(1) a temporal aspect; (Claim 1 – “when:”), (Claim 42 – “based on
`
`detected usage where:”);
`
`(2) a number of prerequisites elements; (Claim 1 – Limitations [1.m.i]-
`
`[1.m.ii], [1.n.i]-[1.n.v], [1.o.i]-[1.o.v]); and
`
`(3) functions that are executed when the prerequisites elements are met.
`
`(Claim 1 – Limitations [1.m.iiii], [1.n.iv], [1.o.vi]-[1.o.viii])
`
`EX1001, 102:50-103:55; 114:53-115:53.
`
`28. The declaration of Dr. Cockburn (EX1003) largely focuses on three of
`
`these functions, which are identified as [1.m] ([1.m.i – 1.m.iii]), [1.n] ([1.n.i –
`
`IPR2024-01090
`
`- 13 -
`
`Patent No. 10,725,581
`
`Page 16 of 80
`
`

`

`
`
`1.n.vi]), and [1.o] ([1.o.i – 1.o.viii]). EX1001, 103:10-55; EX1003, ¶¶33-35. As
`
`recited in claim 1, the three different claimed functionalities are executed “when”
`
`the prerequisite elements are met (described as three scenarios by Dr. Cockburn).
`
`EX1001, 102:50-103:55; EX1003, ¶¶33-35. These prerequisite elements are defined
`
`in the claims, but the most relevant of the scenarios concerns the duration of the first
`
`contact in relation to a first user-configurable predefined duration and a second user-
`
`configurable predefined duration. EX1001, 103:10-55. When mapping to the prior
`
`art Dr. Cockburn refers to the first user-configurable predefined duration as “the
`
`selection threshold” and refers to the second user-configurable predefined duration
`
`as “the activation threshold.” EX1003, ¶¶59-64. The below description of the
`
`claims is intended to be a summary of the claimed operation, and any omission of
`
`elements or use of different words are for brevity and consistency with Dr.
`
`Cockburn.
`
`29.
`
`In the first scenario (among other things), “the duration of the first
`
`contact is determined to be less than a first user-configurable predefined duration”
`
`and “the end of the first contact is detected” (Limitation [1.m.ii]). See EX1001,
`
`claim 1. When this prerequisite operation is satisfied, the programs (executed by the
`
`processor) cause the device to “avoid display, on the first web page, a result of any
`
`operation based on the first contact.” Id.
`
`IPR2024-01090
`
`- 14 -
`
`Patent No. 10,725,581
`
`Page 17 of 80
`
`

`

`
`
`30.
`
`In the second scenario (among other things), “the duration of the first
`
`contact is determined to be greater than the first user-configurable predefined
`
`duration and less than a second user-configurable predefined duration” (Limitation
`
`[1.m.ii]) and “the end of the first contact is detected” (Limitation [1.m.v]). See
`
`EX1001, claim 1. When this prerequisite operation is satisfied, the programs
`
`(executed by the processor) cause the device to “replace the display of the first web
`
`page with the second web page via the user interface.” Id.
`
`31.
`
`In the third scenario (among other things), “the duration of the first
`
`contact is determined to be greater than the second user-configurable predefined
`
`duration” (Limitation [1.o.ii]) and “the end of the first contact is not detected”
`
`(Limitation [1.o.v]). See EX1001, claim 1. When this prerequisite operation is
`
`satisfied, the programs (executed by the processor) cause the device to (1) “output,
`
`utilizing the actuator, a first feedback that is perceptible by touch;” (2) “display a
`
`menu including at least one option for […] performing an operation on a web address
`
`associated with the hyperlink of the first web page;” and (3) “further display at least
`
`a portion of the second web page […].” Id.
`
`32. Claim 42 includes similar limitations to those found in claim 1. See
`
`EX1004, 114:53-115:53. Claim 42 includes the same three scenarios described
`
`above and perform the resulting functionalities based on the detected usage where
`
`prerequisite operation is satisfied. Thus, both claims 1 and 42 recite execution of
`
`IPR2024-01090
`
`- 15 -
`
`Patent No. 10,725,581
`
`Page 18 of 80
`
`

`

`
`
`operations in response to the determination that various conditions are met. See
`
`EX1001, claims 1 and 42.
`
`VI. OVERVIEW OF THE PRIOR ART
`33. Dr. Cockburn primarily relies on three cited references for his invalidity
`
`analysis. See EX1003, ¶67 (the Blumenberg-Bhumkar-Griffin combination).
`
`A. U.S. Patent App. No. 2008/0259041 (“Blumenberg”)
`34. U.S. Patent App. No. 2008/0259041 is titled “Method, system, and
`
`graphical user interface for activating hyperlinks” and lists as its first inventor Chris
`
`Blumenberg. I will refer to this reference as “Blumenberg.”
`
`35. Blumenberg discloses a portable electronic device 200 that includes a
`
`touch screen 208 which can be contacted by a user’s fingers 212 to perform certain
`
`functions on the device. EX1004, ¶[0034]. I describe two of the most relevant
`
`functions cited by Dr. Cockburn below.
`
`Inadvertent Touch Functionality
`1.
`36. As depicted in Blumenberg’s FIG. 2 (inset below), Blumenberg
`
`describes the use of a keyboard 210 where “a user may select one or more of the
`
`icons, and thus, one or more of the corresponding symbols, by making contact or
`
`touching the keyboard 210.” EX1004, ¶[0034].
`
`IPR2024-01090
`
`- 16 -
`
`Patent No. 10,725,581
`
`Page 19 of 80
`
`

`

`
`
`
`
`As shown above, the user contacts the keyboard with their fingers 212.
`
`EX1004, ¶[0034]. Note that Blumenberg admits that the fingers depicted in FIG. 2
`
`are “(not drawn to scale in the figure)” EX1004, ¶[0034].1 This contact with the
`
`
`1 An average index finger length to mobile phone screen width ratio would be
`approximately 1:1, whereas in Blumenberg’s FIG. 2, the ratio appears to be more
`like 1:3. This is important to note because FIG. 2 does not accurately represent the
`difficulty in applying desktop user interface objects (like a menu with a series of
`small icons) to a mobile user interface context, where it can be difficult for a user to
`be able to easily and accurately press an intended small icon, virtual key, or virtual
`button with their finger, especially when these small icons, virtual keys, or virtual
`buttons are arrayed next to one another.
`
`IPR2024-01090
`
`- 17 -
`
`Patent No. 10,725,581
`
`Page 20 of 80
`
`

`

`
`
`user’s finger can correspond to a specific gesture that is associated with the selection
`
`of the icon, such as one or more taps, swipes, or a rolling of a finger. See id. By
`
`linking the selection of an icon with contact corresponding to a specific gesture,
`
`“inadvertent contact with an icon” can be avoided. See id. “For example, a swipe
`
`gesture that sweeps over an icon may not select a corresponding symbol if the
`
`gesture corresponding to selection is a tap gesture.” EX1004, ¶[0034].
`
`37. An illustration of this functionality can be seen by an example in which
`
`a user wishes to type “patent” on Blumenberg’s keyboard 210. In this example, the
`
`user must tap—initiate and break contact—on each icon/letter (“P” and then “A”
`
`and so forth) to spell the word “patent.” If the user does not tap the icons and,
`
`instead, attempts to swipe through the letters, Blumenberg explains that it will not
`
`select those letters. EX1004, ¶[0034]. Following the example above, if a user
`
`contacts the “P” icon and then drags their finger to the “A” icon without breaking
`
`contact, the device will consider this contact inadvertent. Therefore, the device will
`
`not register a selection on the icons (as opposed to selecting every icon between the
`
`“p” and “a” icons). See EX1004, ¶[0034].
`
`2. Hyperlink Preview Functionality
`38. Blumenberg also discloses a graphical user interface (“GUI”) for
`
`“activating hyperlinks on a portable electronic device.” EX1004, ¶[0002].
`
`Blumenberg’s GUI is designed to overcome the “small screen size” of portable
`
`IPR2024-01090
`
`- 18 -
`
`Patent No. 10,725,581
`
`Page 21 of 80
`
`

`

`
`
`devices that hinders the user’s ability to get information on and navigate using
`
`hyperlinks, because in mobile user interfaces “getting information about the
`
`hyperlinks and navigating to other webpages via the hyperlinks takes on a hit-or-
`
`miss character, which can frustrate users.” Id., ¶[0004].
`
`39. As depicted in FIG. 5C (inset below), Blumenberg seeks to mitigate the
`
`issue of navigating hyperlinks by introducing a hyperlink preview window (e.g., link
`
`information 510) that provides quick, easy to read information to help the user
`
`confirm correct f