UNITED STATES PATENT AND TRADEMARK OFFICE
`______________
`
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`______________
`
`APPLE INC.,
`
`Petitioner
`
`v.
`
`SMITH INTERFACE TECHNOLOGIES, LLC,
`Patent Owner
`
`______________
`
`Case IPR2024-01089
`10,671,212
`______________
`
`DECLARATION OF RAJEEV SURATI, Ph.D.
`
`Apple Inc. v. Smith Interface Techs., LLC
`IPR2024-01089 | Smith EX2001 | Page 1 of 58
`
`
`______________
`
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`______________
`
`APPLE INC.,
`
`Petitioner
`
`v.
`
`SMITH INTERFACE TECHNOLOGIES, LLC,
`Patent Owner
`
`______________
`
`Case IPR2024-01089
`10,671,212
`______________
`
`DECLARATION OF RAJEEV SURATI, Ph.D.
`
`Apple Inc. v. Smith Interface Techs., LLC
`IPR2024-01089 | Smith EX2001 | Page 1 of 58
`
`
`
`TABLE OF CONTENTS
`INTRODUCTION ............................................................................................... 1
`
`I.
`
`II. PROFESSIONAL BACKGROUND ................................................................... 1
`
`III. COMPENSATION ............................................................................................10
`
`IV. LEGAL CONSIDERATION .............................................................................10
`
`A. Obviousness .................................................................................................10
`
`B.
`
`Claim Interpretation ....................................................................................11
`
`V. Overview of Task & Basis for Opinions ...........................................................12
`
`VI. Level of Skill in the Relevant Art ......................................................................13
`
`VII.
`
`OVERVIEW OF THE ’212 PATENT ........................................................14
`
`A.
`
`Independent Claims 1, 16, and 84 ...............................................................15
`
`VIII. OVERVIEW OF THE PRIOR ART ...........................................................16
`
`A. U.S. Patent Application Pub. No. 2006/0197753 (“Hotelling”) .................16
`
`1.
`
`2.
`
`Figure 18: “an exemplary GUI for a main menu” ..................................19
`
`Figure 11: “an exemplary GUI for a media player” ...............................20
`
`B. U.S. Patent Application Pub. No. 2004/0051726 (“Martyn”) ....................23
`
`IX. GROUNDS 1 AND 2 FAIL BECAUSE PETITIONER’S HOTELLING-
`
`MARTYN-CHO COMBINATION LACKS EVIDENTIARY SUPPORT ............26
`
`- i -
`
`Page 2 of 58
`
`
`
`A.
`
`The Petition mixes distinct embodiments from Hotelling without asserting
`
`a motivation to combine them ..............................................................................26
`
`B. Dr. Cockburn’s asserted motivations to combine Hotelling and Martyn are
`
`also deficient .........................................................................................................33
`
`1.
`
`Dr. Cockburn’s first asserted motivation is a generalized assertion that
`
`lacks objective evidence ....................................................................................34
`
`2.
`
`Dr. Cockburn’s second asserted motivation manufactures a usability
`
`issue with Hotelling’s display that is unsupported by objective evidence ........37
`
`3.
`
`Dr. Cockburn’s third asserted motivation conflates Hotelling’s music
`
`player mode with its main menu GUI ...............................................................41
`
`4.
`
`Petitioner’s fourth asserted motivation ignores the difference between
`
`the user interface challenges of Hotelling and Martyn ......................................44
`
`X. GROUNDS 1 AND 2 ALSO FAIL BECAUSE, EVEN ACCEPTING THE
`
`PROPOSED COMBINATION, THAT COMBINATION DOES NOT MEET ALL
`
`ELEMENTS OF THE CHALLENGED CLAIMS ..................................................46
`
`A.
`
`Limitations [1.c], [16.h], [84.d]: “[when] / [based on] a second duration of
`
`the touch . . . [is] / [being] detected . . . output feedback that is perceptible by
`
`touch” ....................................................................................................................46
`
`1.
`
`First Theory .............................................................................................47
`
`- ii -
`
`Page 3 of 58
`
`
`
`2.
`
`Second Theory ........................................................................................50
`
`XI. CONCLUSION ..................................................................................................54
`
`
`
`
`
`
`
`
`- iii -
`
`Page 4 of 58
`
`
`
`
`
`I.
`
`INTRODUCTION
`1. My name is Dr. Rajeev Surati, Ph.D. I have been retained by Smith
`
`Interfaces Technologies, LLC as an independent expert consultant in this inter partes
`
`review (“IPR”) proceeding before the United States Patent and Trademark Office
`
`(“PTO”) regarding US Patent No. 10,671,212 (“the ’212 Patent”).
`
`II.
`
`PROFESSIONAL BACKGROUND
`All of my opinions stated in this Declaration are based on my own
`2.
`
`personal knowledge and professional judgment. In forming my opinions, I have
`
`relied on my knowledge and experience in designing, developing, and researching
`
`the technology referenced in this Declaration.
`
`3.
`
`I am over 18 years of age and, if I am called upon to do so, I would be
`
`competent to testify as to the matters set forth herein. I understand that a copy of
`
`my current curriculum vitae, which details my education and professional and
`
`academic experience, is being submitted as EX2002. The following provides a brief
`
`overview of some of my experience that is relevant to the matters set forth in this
`
`Declaration.
`
`4.
`
`I hold a Doctor of Philosophy (Ph.D.) degree in Electrical Engineering
`
`and Computer Science from the Massachusetts Institute of Technology (“MIT”)
`
`(awarded in 1999) with a Grade Point Average of 5.0/5.0. I obtained a Master of
`
`Science degree in Electrical Engineering and Computer Science from MIT (awarded
`
`IPR2024-01089
`
`- 1 -
`
`Patent No. 10,671,212
`
`Page 5 of 58
`
`
`
`
`
`in 1995) with a Grade Point Average of 5.0/5.0. I have a Bachelor of Science degree
`
`in Electrical Engineering from MIT (awarded 1992), graduating with a Grade Point
`
`Average of 4.9/5.0.
`
`5. My Ph.D. thesis was entitled “Scalable Self-Calibrating Technology for
`
`Large Scale Displays.” My master’s thesis was entitled “Practical Partial
`
`Evaluation.” My undergraduate thesis, for which I received MIT EECS’s William
`
`A. Martin thesis prize for best undergraduate thesis, was entitled “A Parallelizing
`
`Compiler based on Partial Evaluation.” I was also awarded the Department of
`
`Energy’s highly-selective Computational Science Fellowship in 1995, which funded
`
`my Ph.D. studies.
`
`6.
`
`In 1989, I began programming embedded systems and, in 1995, built
`
`one of the first publicly available Internet Coke Machines. During that time, I
`
`worked on various user interfaces. For example, one to augment the coke machine
`
`to dispense coke and another that became one of the early ecommerce mechanisms,
`
`using standards-based protocols for tracking user consumption, payments, and drink
`
`stock.
`
`7.
`
`In 1997, while at MIT pursuing my Ph.D., I started a company called
`
`Flash Communications. At Flash, I invented an instant messaging platform focused
`
`on enterprise needs. I had the idea while observing the popularity of the Zephyr
`
`Instant Messaging Service at MIT beginning in 1988. Attracted by Flash’s
`
`IPR2024-01089
`
`- 2 -
`
`Patent No. 10,671,212
`
`Page 6 of 58
`
`
`
`
`
`enterprise focus, Microsoft acquired Flash in 1998. With Microsoft, we built both a
`
`client and server product. The basic protocol we invented became the foundation
`
`for the well-known XMPP protocol that was widely used in the mid-2000s among
`
`instant messaging providers. I developed both the client and server products and
`
`dealt with many, if not all, of the issues one might face when implementing user
`
`interfaces for instant messaging systems. In fact, we helped to build Microsoft’s
`
`famous MSN product, which adopted various user interface elements we created.
`
`8.
`
`As a result of this experience, I became heavily involved with User
`
`Interface design on desktop machines, experience that I have continued to develop
`
`by building desktop, mobile, and web-based applications used by millions of people.
`
`9.
`
`After graduating, I joined Microsoft (as was required by Microsoft in
`
`its acquisition of Flash Communications), where I worked on both client and server
`
`technologies related to instant messaging. My work spanned both the Microsoft
`
`Exchange Instant Messaging product that was released in 2000 and MSN Messenger.
`
`10. From 2000 to 2007, I cofounded, led, and sold a business called
`
`photo.net to Namemedia, which is now part of GoDaddy.com. In early 2000,
`
`Photo.net was considered a top 1,000 website and it received large amounts of
`
`traffic. I built the site from its beginning as a single computer installed in a
`
`datacenter to a full rack of computers in that same data center. As a result, I became
`
`very familiar with the careful design and programming one must employ to build
`
`IPR2024-01089
`
`- 3 -
`
`Patent No. 10,671,212
`
`Page 7 of 58
`
`
`
`
`
`and maintain systems that enable both engaging user interfaces and experiences at
`
`scale.
`
`11. At photo.net, we prototyped many fundamental Internet community
`
`features, such as photo sharing, social networking, and memberships. The photo.net
`
`system was written on top of Oracle SQL Database and served up many millions of
`
`web pages each day (corresponding to millions of records) with high-volume inserts.
`
`The system also incorporated an ecommerce platform used to bill subscriptions,
`
`track users, etc. I spent seven years running the site, both writing new features driven
`
`by the Oracle SQL database and maintaining tables containing the many millions of
`
`records generated by the site. I also served as the database administrator, which
`
`provided me with extensive operational experience running such a system. I dealt
`
`with everything from database query speedup to the more mundane day-to-day
`
`issues, such as the maintenance of relational databases, including backups. In
`
`addition to the web interface, we also ran a mobile version of the product based on
`
`WAP, which I helped to develop.
`
`12. After developing one of the first photo sharing sites, we further
`
`increased user engagement by designing an innovative photo rating interface. Our
`
`photo rating system became a huge draw for the site. The rating system increased
`
`engagement by four orders of magnitude, ultimately resulting in Sergey Brin,
`
`cofounder of Google, asking photo.net to provide imagery to Google for a screen
`
`IPR2024-01089
`
`- 4 -
`
`Patent No. 10,671,212
`
`Page 8 of 58
`
`
`
`
`
`saver application. Joichi Ito, former head of the MIT Media Lab and a famous
`
`internet luminary called it “the best photo sharing site” in 2001.
`
`13.
`
`I also innovated user interface technology at Nexaweb, where I was an
`
`investor and technical advisor. Nexaweb designed User Interfaces that worked well
`
`over the web. At Nexaweb, I spent a great deal of time solving underlying issues
`
`related to implementing a UI framework both in Java and Javascript. We also
`
`innovated a mechanism using server push prior to the AJAX becoming widely
`
`adopted.
`
`14.
`
`In 2004, I started Scalable Display Technologies based on the
`
`technology in my Ph.D. thesis and the related patent (U.S. Patent No. 6,456,339)
`
`entitled “Super-resolution Display.” As part of my Ph.D. thesis, I designed a special
`
`display that allows one to create ultra-high-resolution displays composed of multiple
`
`projectors tiled with a slight overlap. The system uses camera-based feedback to
`
`create an inverse map that drives the system such that a person can only see a
`
`continuous, seamless display. There is no visible bezel or overlap. During this
`
`project, I became familiar with how and why to drive content onto the display. Thus,
`
`I also became familiar with content encoding/decoding, user interfaces for driving
`
`large displays, multimedia content storage, high resolution imagery, networks,
`
`recording, GPUs, storage of content, etc.
`
`IPR2024-01089
`
`- 5 -
`
`Patent No. 10,671,212
`
`Page 9 of 58
`
`
`
`
`
`15. At Scalable Display Technologies, I developed the core algorithms and
`
`implemented—with a team—the entire software system, which included the core
`
`user interface, machine vision and embedded components, and a multimedia
`
`playback system. Our first customer was Pixar. Today, the software is primarily
`
`used by the US military and commercial theme parks, such as Disney and Universal.
`
`Scalable is well-known in the industry for having some of the easiest and intuitive
`
`software to use.
`
`16.
`
`In parallel with my work at Scalable, I lectured at MIT on many
`
`subjects, including the Android operating system. In 2008, I worked with a group
`
`of MIT students on the development of mobile applications for Android with Rich
`
`Miner, a cofounder of Android. Since many students created applications involving
`
`Google Maps and localization, I became familiar with these technologies. I also
`
`served as a lecturer and mentor at MIT Play Labs, which was an incubator for
`
`augmented reality (AR) and virtual reality (VR) software for mobile handsets and
`
`headset applications. Notably, in addition to mobile handsets, Android runs on the
`
`Oculus headset. As part of that program, I worked with several startup companies
`
`on mobile applications, including a startup that developed applications for Telegram,
`
`which is a cross-platform, cloud-based instant messaging system.
`
`17. Since 2014, I have worked as an independent consultant for several
`
`companies including NEC, Hitachi, Hi Marley, and Estee Lauder.
`
`IPR2024-01089
`
`- 6 -
`
`Patent No. 10,671,212
`
`Page 10 of 58
`
`
`
`
`
`18.
`
`In 2018, I became a senior partner at nCent Labs. In that role, I
`
`consulted on the development of an incentive-market-based platform for block
`
`chains and cryptocurrency. Another part of my work at nCent Labs focused on the
`
`development of SMS messaging applications for the nCent platform. In that part, I
`
`developed a full stack application around the protocol.
`
`19. From 2019 to 2020, I served as a Technical Lead of the Skunkworks at
`
`Hydrow, which is a startup company that develops indoor rowing machines. In this
`
`role, I worked on special projects, including the development of a VR experience
`
`using Magic Leap and Oculus that immersed users in a team-based rowing crew on
`
`a scull using VR cinematography. I leveraged my experience with user interfaces
`
`and Human-Computer Interaction (“HCI”) to develop a system that gives people a
`
`sense of rowing on the water with others, while in reality, they were rowing on a
`
`rowing machine.
`
` We accomplished our goal through a clever use of
`
`cinematography. The result was quite compelling and increased user engagement
`
`with their exercise program.
`
`20.
`
`In 2020, I started a company called Skyline Nav AI Inc. that develops
`
`technology that uses a visual location (i.e., the skyline) to geo-locate the place a
`
`picture was taken. At Skyline, I built and now maintain a full user experience
`
`centered around a workflow for people to snap a picture and use the image’s skyline
`
`and downloaded 3D mode to localize a person. I have written applications for both
`
`IPR2024-01089
`
`- 7 -
`
`Patent No. 10,671,212
`
`Page 11 of 58
`
`
`
`
`
`Android and iOS that communicate with a backend server running the machine
`
`vision algorithms.
`
`21. Over the past decade, I have served as a technical consultant and expert
`
`witness on patent litigation matters. In the course of this work, I have provided
`
`consulting services to a wide variety of technology companies, including
`
`BritishTelecom, Apple,
`
`IBM, Philips, Shopify, Zillow, Polaris Powered
`
`Technologies, Amazon, Salesforce, Hitachi, Slack, Harris Teeter, and others.
`
`22. During the course of my career, I was the inventor of the subject matter
`
`for approximately 15 U.S. Patents. I have also received additional patents,
`
`including: U.S. Patent No. 8,817,111, entitled “System and method of calibrating a
`
`display system free of variation in system input resolution,” U.S. Patent No.
`
`8,994,757, “System and method for providing improved display quality by display
`
`adjustment and image processing using optical feedback,” U.S. Patent No.
`
`9,215,455, “System and method of calibrating a display system free of variation in
`
`system input resolution,” U.S. Patent No. 9,369,683, “System and method for
`
`calibrating a display system using manual and semi-manual techniques,” and U.S.
`
`Patent No. 9,497,447, “System and method for color and intensity calibrating of a
`
`display system for practical usage.”
`
`23.
`
`I am also a co-inventor of patented technology related to instant
`
`messaging, including U.S. Patent No. 5,943,478, which describes associated
`
`IPR2024-01089
`
`- 8 -
`
`Patent No. 10,671,212
`
`Page 12 of 58
`
`
`
`
`
`technology that I developed related to pop-up, two-way messaging over the Internet.
`
`While at Microsoft, I was an inventor on several patents, including: U.S. Patent No.
`
`6,260,148 relating to methods and systems for message forwarding and property
`
`notifications using electronic subscriptions; and U.S. Patent Nos. 6,415,318 and
`
`6,604,133 relating to inter-enterprise messaging systems using bridgehead servers.
`
`Aspects of these patents relate specifically to messaging and notification technology
`
`in telecommunications systems.
`
`24.
`
`I serve on the advisory boards of several technology companies,
`
`including Paneve, which develops general purpose ASICs coupled with compiler
`
`technology; Nexaweb, which develops real-time web application frameworks using
`
`HTTPS; Antix Labs, which develops compiler technology for a universal gaming
`
`platform; and Permabit, which develops content-addressable storage.
`
`25.
`
`In addition to my above experience, I have published papers on subjects
`
`relating to computing systems, computer network communications, databases, and
`
`other subjects within the realm of electrical and computer engineering. I have also
`
`received several awards for my contributions as an inventor and entrepreneur,
`
`including the Global Indus. Technovator Award 2009 and Laureate of 2009
`
`Computer World Honors Program. My publication and award history are listed in
`
`more detail on my CV.
`
`IPR2024-01089
`
`- 9 -
`
`Patent No. 10,671,212
`
`Page 13 of 58
`
`
`
`
`
`III. COMPENSATION
`26. My work on this matter is being billed at my customary rate of $700 an
`
`hour. Also, I am being reimbursed for reasonable expenses I incur in relation to my
`
`services. I have no pecuniary interest in the outcome of this proceeding. I
`
`understand I will be paid regardless of the outcome of any proceeding in which my
`
`work is used.
`
`IV. LEGAL CONSIDERATION
`I am not an attorney. My understanding of the law is based on
`27.
`
`information provided by counsel for Smith Interface Technologies, LLC.
`
`A. Obviousness
`I understand that a claimed invention is obvious and, therefore, not
`28.
`
`patentable if the subject matter claimed would have been obvious to a person of
`
`ordinary skill in the art at the time that the invention of the ’212 Patent (a
`
`“POSITA”), which I have been asked to treat as August 5, 2011.
`
`29.
`
`I understand that a claim can be obvious in view of a single prior art
`
`reference (e.g., via modification of that prior art reference) or multiple prior art
`
`references (e.g., via a combination of two or more prior art references), if such a
`
`modification or combination was within the skill of a POSITA. I understand that
`
`there must be some articulated reasoning with some rational underpinning to support
`
`a conclusion of obviousness. I also understand that to establish a finding of
`
`obviousness one must show that one of ordinary skill in the art would have had a
`IPR2024-01089
`- 10 -
`Patent No. 10,671,212
`
`Page 14 of 58
`
`
`
`
`
`motivation to combine the prior art references to produce the claimed invention and
`
`a reasonable expectation that the combination would be successful.
`
`30.
`
` I further understand that exemplary rationales that may support a
`
`conclusion of obviousness include: (1) simply arranging old elements in a way in
`
`which each element performs the same function it was known to perform, and the
`
`arrangement yields expected results, (2) merely substituting one element for another
`
`known element in the field, if the substitution yields no more than a predictable
`
`result, (3) combining elements in a way that was “obvious to try” because of a design
`
`need or market pressure, where there was a finite number of identified, predictable
`
`solutions, (4) that design incentives or other market forces in a field would have
`
`prompted variations in a work that were predictable to a person of ordinary skill in
`
`the art, and (5) that there was some teaching, suggestion, or motivation in the prior
`
`art that would have led a POSITA to modify or combine prior art references to arrive
`
`at the claimed invention.
`
`B. Claim Interpretation
`I understand that a claim term is interpreted according to its ordinary
`31.
`
`and customary meaning as a POSITA would have understood the term in light of the
`
`surrounding claim language, other claims, the specification, and the patent’s
`
`prosecution history, which are referred to as intrinsic evidence. I also understand
`
`that prior art references cited in the patent’s prosecution history are considered
`
`IPR2024-01089
`
`- 11 -
`
`Patent No. 10,671,212
`
`Page 15 of 58
`
`
`
`
`
`intrinsic evidence. I further understand that evidence outside the patent and its
`
`prosecution history (e.g., dictionaries and technical articles), may inform the context
`
`in which a POSITA would have understood the claims of a patent. I understand this
`
`ordinary and customary meaning applies absent unique circumstances, such as
`
`where a patent clearly expresses an intent to set forth a special meaning for a term.
`
`V. Overview of Task & Basis for Opinions
`I have been asked to review the ’212 Patent. I have been asked to
`32.
`
`provide opinions related to certain issues from the perspective of a person of ordinary
`
`skill, having knowledge of the relevant art, as of August 5, 2011, and—except where
`
`otherwise noted—the opinions stated in this declaration are from that perspective.
`
`The qualifications and abilities of such a person are described in paragraph 38 below.
`
`33. More particularly, I have been informed that this proceeding is
`
`currently at a preliminary stage during which the Patent Trial and Appeal Board (the
`
`Board) will decide whether the Petition establishes a reasonable likelihood of at least
`
`one claim of the ’212 Patent being invalid. I also understand that the Board may
`
`review the Petition under a stricter “compelling merits” standard that requires the
`
`Petition to make a stronger showing of invalidity; however, my analysis is not
`
`performed using this higher standard. At this preliminary stage, I have been asked
`
`to focus my analysis on the Petition’s proposed invalidity theories for the ’212
`
`Patent’s independent claims 1, 16, and 84.
`
`IPR2024-01089
`
`- 12 -
`
`Patent No. 10,671,212
`
`Page 16 of 58
`
`
`
`
`
`34. My opinions are based on my education, training, and experience as
`
`well as items that I reviewed to prepare my opinions, including the Petition, the
`
`documents listed in the Exhibit List included with the Petition, and any other items
`
`that I reference in my below analysis.
`
`35.
`
`I understand that other issues may arise that require further explanation,
`
`and I will provide that explanation if appropriate. As a result, I respectfully reserve
`
`the right to update and supplement this declaration and the information and opinions
`
`provided herein.
`
`VI. Level of Skill in the Relevant Art
`I understand that a patent must be analyzed from the perspective of a
`36.
`
`person of ordinary skill in the art (“POSITA”) as of the time of invention of the
`
`patent, which is typically considered to be the patent’s earliest filing date. While I
`
`do not necessarily agree with every aspect of Dr. Cockburn’s explanation of the level
`
`of ordinary skill in the art, I do not currently believe the aspects of that explanation
`
`I would change or clarify would impact the analysis set forth in this declaration. For
`
`purposes of this declaration, my opinions are therefore provided from the perspective
`
`of the POSITA described in paragraph 34 of Dr. Cockburn’s declaration (EX1003),
`
`which is produced below:
`
`a Bachelor of Science degree in computer science, computer
`engineering, or a comparable field and at least two years of professional
`experience working with graphical user interfaces and applications for
`
`IPR2024-01089
`
`- 13 -
`
`Patent No. 10,671,212
`
`Page 17 of 58
`
`
`
`
`
`computers. Such experience could be obtained through research and
`study in a graduate program or through comparable exposure through
`industry employment, and additional years of experience could
`substitute for the advanced-level degree.
`
`VII. OVERVIEW OF THE ’212 PATENT
`37. The ’212 Patent relates to a “gesture-equipped touch screen system,”
`
`where “a signal is received in association with a touch interface” and “a user
`
`experience may be altered, utilizing the signal.” EX1001, Title, 15:9-10, 15:57-58.
`
`The patent discloses various gestures, patterns, or multi-touch inputs that can be used
`
`to improve the functionality of user interfaces for touch screen devices. See e.g.,
`
`EX1001, 1:42-2:5, 2:6-31, 2:32-59, 2:60-3:15. Figure 18 depicts a high-level
`
`flowchart of
`
`the
`
` operation of one
`
`embodiment of
`
`the patent:
`
`IPR2024-01089
`
`- 14 -
`
`
`Patent No. 10,671,212
`
`Page 18 of 58
`
`
`
`
`
`38. EX1001, FIG. 18. Possible “gesture inputs” used to control the choice
`
`of operation can include “touch event attributes (both initial and over time), system
`
`and/or application flags, switches, buttons, states, sensor data,” or other relevant
`
`control inputs available to the user. Id., 57:19-22.
`
`A.
`39.
`
`Independent Claims 1, 16, and 84
`Independent claims 1, 16, and 84 are each directed to a multifunction
`
`response of “a mobile device including a touch screen” to “a touch.” “[When] /
`
`[based on]” “a first duration of a touch” “[is] / [being] detected as ceasing,”
`
`“perform[ing] an operation.” Limitations [1.b], [16.g], [84.c]. “[When] / [based
`
`on]” “a second duration of the touch, that is different than the first duration of the
`
`touch” “[is] / [being] detected” “after the first duration is detected without the
`
`ceasing”: (a) “output[ting] feedback that is perceptible by touch” and (b)
`
`“display[ing] at least one menu including a plurality of particular actions.” is
`
`displayed. Limitations [1.c], [1.d], [16.h], [16.i], [84.d], [84.e].
`
`40.
`
`Independent claims 1, 16, and 84 further recite recognizing and acting
`
`upon additional touches in connection with “the at least one menu.” First, “[when]
`
`/ [based on] a selection touch applied to the touch screen [is] / [being] detected in
`
`connection with at least one of the particular actions of the at least one menu,”
`
`“perform[ing] the at least one particular action.” Limitations [1.e], [16.j], [84.f].
`
`Second, “[when] / [based on] a slide or swipe gesture applied to the touch screen [is]
`
`IPR2024-01089
`
`- 15 -
`
`Patent No. 10,671,212
`
`Page 19 of 58
`
`
`
`
`
`/ [being] detected,” “chang[ing] at least one aspect of the display of the at least one
`
`menu.” Limitations [1.f], [16.k], [84.g].
`
`VIII. OVERVIEW OF THE PRIOR ART
`41. Dr. Cockburn’s analysis falters primarily with respect to two of the
`
`cited references, each of which are directed to distinct kinds of user interfaces. Each
`
`is discussed in more detail below.
`
`A. U.S. Patent Application Pub. No. 2006/0197753 (“Hotelling”)
`42. U.S. Patent Application Publication 2006/0197753 is entitled “Multi-
`
`functional hand-held device” and lists its first inventor as Steven P. Hotelling. See
`
`EX1004. I will refer to this reference as “Hotelling.”
`
`43. Hotelling discloses a “multi-functional hand-held device capable of
`
`configuring user inputs based on how the device is to be used.” EX1004, Abstract,
`
`[0009]. The goal of the multi-functional hand-held device of Hotelling is to
`
`“integrate[] at least two [traditional] devices” (such as a PDA, cell phone, camera,
`
`music player, etc.) into a single device. EX1004 [0048]. In 2006, at the time of
`
`Hotelling’s invention, there was a growing interest in these sorts of multi-functional
`
`devices that would allow a user to merge each of their five or ten mobile devices into
`
`a single device that could easily be carried in their pocket. However, each of these
`
`five to ten (or more) devices featured different user interfaces, typically using
`
`IPR2024-01089
`
`- 16 -
`
`Patent No. 10,671,212
`
`Page 20 of 58
`
`
`
`
`
`traditional physical buttons. This created a user interface challenge for developers
`
`who realized that a hundred-button multi-functional device was impractical.
`
`44. Hotelling recognized this challenge. See EX1004, [0006-0007]
`
`(“traditionally separate hand-held electronic devices have begun
`
`to be
`
`combined…[i]t is a challenge to design a merged device with limited input devices
`
`without adversely affecting the dedicated inputs for each device.”). Further, “there
`
`is not enough space on the [merged] device to house all the necessary buttons and
`
`switches” to accommodate the “particular set of input mechanisms for providing
`
`inputs” used by each separate legacy device. Id. In alignment with user interface
`
`design fundamentals, Hotelling recognized that “it is preferable, not to overload the
`
`hand-held devices with a large number of input mechanisms as this tends to confuse
`
`the user.” Id.
`
`45.
`
`Instead of overwhelming the user, Hotelling proposes “virtually
`
`incorporating the physical inputs for each functionality into the GUI,” which
`
`“adapt[s] to whatever device is selected.” EX1004, [0069]. Then, the device has “a
`
`GUI for each functionality.” Id. By replacing physical buttons with virtual buttons
`
`on a touch screen, Hotelling’s design has the added benefit that the “display size can
`
`be substantially increased.” EX1004, [0009]. While it was not the main thrust of
`
`Hotelling’s invention, it is significant that Hotelling’s virtual button design allowed
`
`for larger screens. Devices before Hotelling were heavily restricted by the amount
`
`IPR2024-01089
`
`- 17 -
`
`Patent No. 10,671,212
`
`Page 21 of 58
`
`
`
`
`
`of space required for buttons, including dial pads, keyboards, scroll wheels, and
`
`more. Larger screens like the one disclosed in Hotelling substantially expand the
`
`design space available to user interface developers.
`
`46. Hotelling also discloses the use of a “haptics mechanism” to “enhance
`
`the feel of actuating a GUI element” or to provide “alert[s]” to the user. EX1004,
`
`[0161-0163]. Hotelling teaches that “[t]his particular feature enhances the users
`
`experience and makes the virtual UI feel more like a physical UI.” EX1004, [0161].
`
`Feel is important for a user interface. By designing a user interface that is similar to
`
`other interfaces with which users are familiar, a designer can improve ease-of-use.
`
`“For example, in response to touching a virtual button of the GUI, the haptics may
`
`provide force feedback in the form of vibration and the speaker may provide audio
`
`feedback in the form of a click.” EX1004, [0163]. This feedback tells the user that
`
`“they actually implemented an input” by “simulat[ing] the audio and tactile feel of
`
`a button or switch.” Id. Alternatively, the feedback “may be used to simulate a
`
`button click” through a “click” and “clock” output “when a hard touch is made.” Id.
`
`47. Since Dr. Cockburn focuses specifically on two of the user interface
`
`designs described in Hotelling—one shown in Figure 18 and another in Figure 11—
`
`I will analyze them in more detail.
`
`IPR2024-01089
`
`- 18 -
`
`Patent No. 10,671,212
`
`Page 22 of 58
`
`
`
`
`
`Figure 18: “an exemplary GUI for a main menu”
`1.
`In one user interface design, Hotelling discloses “an exemplary main
`
`48.
`
`menu GUI 260 of a multi-functional device.” EX1004, [0085], FIG. 18 (inset
`
`below).
`
`
`
`EX1004, FIG. 18. When the user selects one of the various “virtual buttons” (e.g.,
`
`“a music player button 262C”) shown in the Figure, “the main page for the selected
`
`functional[ity]” is shown on the display. EX1004, [0085]. If the user wishes to
`
`select a different functionality, “the user simply selects a soft home button…to return
`
`to the main menu page 260.” Id. This design ensures that, “[b]y keeping different
`
`IPR2024-01089
`
`- 19 -
`
`Patent No. 10,671,212
`
`Page 23 of 58
`
`
`
`
`
`functionalities distinct, user confusion may be reduced.” EX1004, [0091]. Similar
`
`to its mimicking of physical buttons, Hotelling again leverages user background
`
`experience with this design. Before the advent of mu
Accessing this document will incur an additional charge of $.
After purchase, you can access this document again without charge.
Accept $ Charge
Still Working On It
This document is taking longer than usual to download. This can happen if we need to contact the court directly to obtain the document and their servers are running slowly.
Give it another minute or two to complete, and then try the refresh button.
A few More Minutes ... Still Working
It can take up to 5 minutes for us to download a document if the court servers are running slowly.
Thank you for your continued patience.
This document could not be displayed.
We could not find this document within its docket. Please go back to the docket page and check the link. If that does not work, go back to the docket and refresh it to pull the newest information.
Your account does not support viewing this document.
You need a Paid Account to view this document. Click here to change your account type.
Your account does not support viewing this document.
Set your membership
status to view this document.
With a Docket Alarm membership, you'll
get a whole lot more, including:
- Up-to-date information for this case.
- Email alerts whenever there is an update.
- Full text search for other cases.
- Get email alerts whenever a new case matches your search.
One Moment Please
The filing “” is large (MB) and is being downloaded.
Please refresh this page in a few minutes to see if the filing has been downloaded. The filing will also be emailed to you when the download completes.
Your document is on its way!
If you do not receive the document in five minutes, contact support at support@docketalarm.com.
Sealed Document
We are unable to display this document, it may be under a court ordered seal.
If you have proper credentials to access the file, you may proceed directly to the court's system using your government issued username and password.
Access Government Site
