Petition for Inter Partes Review of
`U.S. Patent No. 8,585,476 B2
`
`
`
`
`
`UNITED STATES PATENT AND TRADEMARK OFFICE
`
`
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`
`
`META PLATFORMS, INC.,
`Petitioner
`
`v.
`
`MULLEN INDUSTRIES LLC,
`Patent Owner.
`
`
`Case IPR2025-00737
`Patent 8,585,476 B2
`Issue Date: November 19, 2013
`
`Title: LOCATION-BASED GAMES AND
`AUGMENTED REALITY SYSTEMS
`
`
`PETITION FOR INTER PARTES REVIEW
`OF U.S. PATENT NO. 8,585,476 B2
`
`
`
`
`

`

`Table of Contents
`
`
`Page
`
`
`
`I.
`
`MANDATORY NOTICES UNDER §42.8(A)(1) ........................................ 1
`A.
`Real Party-In-Interest under §42.8.(b)(1) ........................................... 1
`B.
`Related Matters under §42.8(b)(2) ..................................................... 1
`C.
`Lead and Back-Up Counsel under §42.8(b)(3) ................................... 1
`D.
`Service Information ........................................................................... 2
`FEE PAYMENT .......................................................................................... 2
`II.
`III. REQUIREMENTS UNDER §§ 42.104 AND 42.108 AND
`CONSIDERATIONS UNDER §§ 314(A) AND 325(D) .............................. 3
`A. Grounds for Standing ......................................................................... 3
`B.
`Identification of Challenge and Statement of Precise Relief
`Requested .......................................................................................... 3
`Considerations Under §§ 314(a) and 325(d) ....................................... 3
`C.
`IV. OVERVIEW OF THE PATENT ................................................................. 5
`A.
`Level of Ordinary Skill ...................................................................... 5
`B.
`Specification Overview ...................................................................... 6
`CLAIM CONSTRUCTION ......................................................................... 8
`V.
`VI. THE CHALLENGED CLAIMS ARE UNPATENTABLE .......................... 9
`A. Overview of Grounds ......................................................................... 9
`B.
`Prior Art Status of Relied-Upon References ..................................... 11
`C.
`Ground 1: Claims 1, 4-5 and 13 Are Obvious Over Jaszlics in
`view of Piekarski ............................................................................. 12
`Independent Claim 1: “An augmented reality game
`
`system comprising:” (1[pre]) ................................................ 12
`(a)
`“a head-mounted display that overlays virtual
`indicia onto a physical playfield;” (1[a]) ..................... 16
`“memory comprising video game logic that
`provides a video game;” (1[b]) .................................... 23
`
`(b)
`
`
`
`
`
`-i-
`
`
`
`

`

`Table of Contents
`(continued)
`
`Page
`
`(c)
`
`(d)
`
`(e)
`
`
`
`
`
`“a wearable processor that utilizes said video game
`logic to provide video game indicia to said head-
`mounted display based on said video game logic,
`wherein said processor is coupled to said memory
`and said head-mounted display;” (1[c]) ....................... 24
`“a detector that determines landscape
`characteristics of said physical playfield, wherein
`said video game logic utilizes said landscape
`characteristics in providing said video game; and”
`(1[d]) ........................................................................... 33
`“a locating device that determines the physical
`location of said locating device on the physical
`playfield, wherein said video game logic utilizes
`the physical location of said locating device in
`providing said video game.” (1[e]) .............................. 39
`Claim 4: “The augmented reality game system of claim
`1, wherein said landscape characteristics are utilized to
`control a computer-controlled video game character.” ........... 42
`Claim 5: “The augmented reality game system of claim
`1, further comprising a wireless transmitter and receiver.” ..... 45
`Independent Claim 13 ............................................................ 47
`
`D. Ground 2: Claim 2 Is Obvious Over Ground 1 Prior Art in
`View of Morihira ............................................................................. 49
`Claim 2: “The augmented reality game system of claim
`
`1, wherein a virtual object is overlayed by said head-
`mounted display on said physical playfield and said
`virtual object is displayed at different transparencies
`depending on the perceived distance of said virtual object
`from said head-mounted display.” .......................................... 49
`Ground 3: Claim 4 Is Obvious Over Ground 1 Prior Art in
`View of Ohshima ............................................................................. 54
`
`E.
`
`-ii-
`
`
`
`
`
`
`
`
`
`

`

`Table of Contents
`(continued)
`
`Page
`
`
`
`
`
`
`
`F.
`
`Claim 4: “The augmented reality game system of claim
`1, wherein said landscape characteristics are utilized to
`control a computer-controlled video game character.” ........... 54
`Ground 4: Claims 6-7 Are Obvious Over Ground 1 Prior Art in
`View of Robarts ............................................................................... 59
`Claim 6: “The augmented reality game system of claim
`
`1, wherein a computer-controlled video game character is
`overlayed by said head-mounted display on said physical
`playfield and said computer-controlled video game
`character is limited to visibility within a certain area.” ........... 59
`Claim 7: “The augmented reality game system of claim
`1, wherein a computer-controlled video game character is
`overlayed by said head-mounted display on said physical
`playfield, said computer-controlled video game character
`is limited to visibility within a certain area, and said
`computer-controlled video game character provides a
`reaction to said physical location when said physical
`location corresponds to entering said certain area.” ................ 69
`G. Ground 5: Claim 17 Is Obvious Over Ground 1 Prior Art in
`View of Muendel ............................................................................. 71
`Claim 17: “The system of claim 13, wherein said
`
`locating device comprises: a positioning system; and an
`inertial movement system.” .................................................... 71
`VII. CONCLUSION ......................................................................................... 76
`CERTIFICATE OF SERVICE ............................................................................. 78
`
`
`
`
`
`
`-iii-
`
`
`
`

`

`Petition for Inter Partes Review of
`U.S. Patent No. 8,585,476 B2
`
`List of Exhibits
`
`
`
`
`
`
`
`
`
`Exhibit
`Description of Document
`No.
`1001 U.S. Patent No. 8,585,476 B2 to Jeffrey D. Mullen (filed Nov. 16,
`2005, issued Nov. 19, 2013) (“’476” or “’476 patent”)
`1002 Declaration of Jeremy Cooperstock, Ph.D. (“Cooperstock”)
`1003 U.S. Patent No. 6,166,744 to Ivan J. Jaszlics and Sheila L. Jaszlics
`(filed Sep. 15, 1998; issued Dec. 26, 2000) (“Jaszlics”)
`1004 Wayne Piekarski et al., “Integrating Virtual and Augmented Realities in
`an Outdoor Application,” in Proceedings, 2nd IEEE and ACM
`International Workshop on Augmented Reality (1999) (“Piekarski”)
`1005 U.S. Patent No. 6,951,515 B2 to Toshikazu Ohshima and Kiyohide
`Satoh (filed Feb. 17, 2000; issued Oct. 4, 2005) (“Ohshima”)
`1006 U.S. Patent No. 6,361,438 B1 to Shigeki Morihira (filed Jul. 24, 1998;
`issued Mar. 26, 2002) (“Morihira”)
`1007 U.S. Patent App. Pub. No. 2004/0002843 A1 to James O. Robarts and
`Cesar A. Alvarez (filed May 13, 2003; published Jan. 1, 2004)
`(“Robarts”)
`Int’l Patent App. Pub. No. WO 2001/042809 A2 to Martin Muendel
`(filed Dec. 6, 2000; published Jun. 14, 2001) (“Muendel”)
`1009 Wayne Piekarski et al., “Tinmith - Augmented Reality With Wearable
`Computers Running Linux,” in Linux Conf Au 2001 - 2nd Australian
`Linux Conference (2001) (“Piekarski 2001”)
`1010 Paul Milgram and Fumio Kishino, “A Taxonomy of Mixed Reality
`Visual Displays,” in IEICE Transactions on Information Systems, Vol.
`E77-D, No.12 (December 1994) (“Milgram”)
`1011 U.S. Provisional Patent App. No. 60/066,904 (“Jaszlics Provisional”)
`1012 Excerpts from Microsoft Computer Dictionary (5th ed. 2002)
`1013
`J. P. Rolland et al., “A Comparison of Optical and Video See-Through
`Head-Mounted Displays,” SPIE Vol. 2351 Telemanipulator and
`Telepresence Technologies (1994), pp.293-307 (“Rolland”)
`
`1008
`
`-i-
`
`
`
`

`

`Petition for Inter Partes Review of
`U.S. Patent No. 8,585,476 B2
`
`List of Exhibits
`
`
`
`
`
`
`
`Exhibit
`Description of Document
`No.
`1014 Excerpts from The Legend of Zelda: A Link to the Past: Prima’s
`Official Strategy Guide (2002)
`1015 Excerpts from New Penguin Dictionary of Computing (2001)
`1016
`’476 Patent File History
`1017 Declaration of Ingrid Hsieh-Yee, Ph.D. (“Hsieh-Yee”)
`1018 Proof of Service of Complaint
`
`-ii-
`
`
`
`

`

`Petition for Inter Partes Review of
`U.S. Patent No. 8,585,476 B2
`
`I. MANDATORY NOTICES UNDER §42.8(A)(1)
`A. Real Party-In-Interest under §42.8.(b)(1)
`Meta Platforms, Inc. is the real party-in-interest to this IPR petition.
`
`B. Related Matters under §42.8(b)(2)
`The ’476 patent is the subject of pending litigation involving Petitioner:
`
`Mullen Industries LLC v. Meta Platforms, Inc., Case No. 1:24-cv-00354-DAE (W.D.
`
`Tex.). Petitioner was served on April 5, 2024. (EX1018, p.003.)
`
`Petitioner is filing IPR petitions against twelve related patents asserted by
`
`Patent Owner in the pending litigation involving Petitioner:
`
`U.S. Patent No.
`8,585,476 B2
`9,744,448 B2
`10,179,277 B2
`10,828,559 B2
`10,967,270 B2
`11,033,821 B2
`11,376,493 B2
`11,904,243 B2
`11,947,716 B2
`12,019,791 B2
`9,662,582 B2
`10,974,151 B2
`
`IPR Case
`IPR2025-00737
`IPR2025-00738
`IPR2025-00739
`IPR2025-00740
`IPR2025-00702
`IPR2025-00703
`IPR2025-00741
`IPR2025-00742
`IPR2025-00743
`IPR2025-00744
`IPR2025-00745
`IPR2025-00746
`
`
`
`
`
`
`
`C. Lead and Back-Up Counsel under §42.8(b)(3)
`Petitioner provides the following designation of counsel.
`-1-
`
`
`

`

`Petition for Inter Partes Review of
`U.S. Patent No. 8,585,476 B2
`
`
`LEAD COUNSEL
`
`BACK-UP COUNSEL
`
`Heidi L. Keefe (Reg. No. 40,673)
`hkeefe@cooley.com
`
`COOLEY LLP
`ATTN: Patent Group
`1299 Pennsylvania Ave. NW, Suite 700
`Washington, DC 20004
`Tel: (650) 843-5001
`Fax: (650) 849-7400
`
`Phillip E. Morton (Reg. No. 57,835)
`pmorton@cooley.com
`
`Andrew C. Mace (Reg. No. 63,342)
`amace@cooley.com
`
`Mark R. Weinstein (Admission pro hac
`vice to be requested)
`mweinstein@cooley.com
`
`Lowell D. Mead (Admission pro hac vice
`to be requested)
`lmead@cooley.com
`
`Patrick Lauppe (Admission pro hac vice to
`be requested)
`plauppe@cooley.com
`
`COOLEY LLP
`ATTN: Patent Group
`1299 Pennsylvania Ave. NW, Suite 700
`Washington D.C. 20004
`
`
`
`Service Information
`D.
`This Petition is being served by Federal Express to the attorney of record for
`
`the ’476 patent, 32733 - JEFFREY D. MULLEN, 2212 Hassinger Lane, Glenshaw,
`
`PA 15116. Petitioner consents to electronic service at the addresses provided above
`
`for lead and back-up counsel.
`
`FEE PAYMENT
`Petitioner requests review of 7 claims, with a $51,875 payment.
`
`-2-
`
`
`
`II.
`
`
`
`
`
`

`

`Petition for Inter Partes Review of
`U.S. Patent No. 8,585,476 B2
`
`42.104 AND
`§§
`III. REQUIREMENTS UNDER
`CONSIDERATIONS UNDER §§ 314(A) AND 325(D)
`A. Grounds for Standing
`Petitioner certifies that the ’476 patent is available for IPR and that Petitioner
`
`42.108 AND
`
`is not barred or otherwise estopped.
`
`B.
`
`Identification of Challenge and Statement of Precise Relief
`Requested
`Petitioner requests IPR institution based on:
`
`Ground
`
`Claims
`
`Basis for Challenge under §103
`
`1
`
`2
`
`3
`
`4
`
`5
`
`1, 4, 5, 13
`
`2
`
`4
`
`6-7
`
`17
`
`Jaszlics (EX1003) in view of Piekarski (EX1004)
`Ground 1 Prior Art + Morihira (EX1006)
`Ground 1 Prior Art + Ohshima (EX1005)
`Ground 1 Prior Art + Robarts (EX1007)
`Ground 1 Prior Art + Muendel (EX1008)
`
`Submitted with this Petition is the Declaration of Jeremy Cooperstock, Ph.D.
`
`(EX1002) (“Cooperstock”), a qualified technical expert. (EX1002, ¶¶1-18,
`
`Appendix A.)
`
`C. Considerations Under §§ 314(a) and 325(d)
`§314(a): The General Plastic factors are not relevant as this is the first and
`
`only IPR petition filed by Petitioner with respect to the ’476 patent.
`
`Nor do the Fintiv factors support discretionary denial under §314(a). The
`
`pending litigation against Petitioner is in relatively early stage, with little substantive
`
`-3-
`
`
`
`
`

`

`Petition for Inter Partes Review of
`U.S. Patent No. 8,585,476 B2
`
`discovery having taken place to date and claim construction proceedings ongoing.
`
`Trial is scheduled to start October 5, 2026—over eighteen months from now—and
`
`a motion to transfer the case by Petitioner to the Northern District of California is
`
`currently pending, which if granted would likely result in an even later trial date.
`
`Petitioner also intends to move to stay the litigation pending resolution of the IPRs.
`
`Additionally, to allay concerns over duplication of efforts, Petitioner represents that,
`
`in the event of institution, it will not pursue in litigation any invalidity defense that
`
`was or could have been raised in IPR.
`
`§325(d): Advanced Bionics does not apply to either reference relied upon in
`
`Ground 1—Jaszlics (EX1003), Piekarski (EX1004)—because neither reference
`
`was presented during prosecution. For the same reason, Advanced Bionics does not
`
`apply to Muendel (EX1008).
`
`With respect to Ohshima (EX1005), the Examiner repeatedly rejected
`
`pending claims as anticipated by and obvious over Ohshima. (EX1016, 00203-
`
`00209, 00249-00255.) The Examiner ultimately concluded, however, that Ohshima
`
`failed to disclose limitations recited in claim 1. (EX1016, 00299.) But as shown
`
`below Part VI.C, Petitioner does not rely on Ohshima for claim 1—the limitations
`
`that the Examiner concluded were not disclosed in Ohshima are disclosed by and
`
`obvious in view of Jaszlics.
`
`
`
`
`
`-4-
`
`
`
`

`

`Petition for Inter Partes Review of
`U.S. Patent No. 8,585,476 B2
`
`With respect to Morihira (EX1006), the Examiner cited it solely in connection
`
`with claim 2, which was not disputed by the applicant. (See EX1016, 00156-00157
`
`(Oct. 2, 2009 Non-Final Rejection, pp.7-8) (“Morihira teaches a video game
`
`transparency control system (abstract) which varies the transparency of game images
`
`based on perceived distances through varying the amount of pixels displayed. It
`
`would have been obvious to one having ordinary skill in the art at the time the
`
`invention was made to modify game images transparencies as taught by Morihira in
`
`order to convey virtual object distance.”), 00186-00187 (Apr. 2, 2010 Applicant
`
`Response, pp.18-19) (not disputing Morihira).)
`
`With respect to Robarts (EX1007), it appears that a similar published
`
`application to Robarts (US 2005/0009608 A1) was listed among approximately fifty-
`
`five other references on a post-allowance applicant-submitted IDS. (EX1016,
`
`00349-00352.) It was not otherwise cited or discussed during prosecution.
`
`For these reasons, Petitioner respectfully submits that the cited prior art does
`
`not raise § 325(d) issues that would warrant discretionary denial of the Petition.
`
`IV. OVERVIEW OF THE PATENT
`A. Level of Ordinary Skill
`A person of ordinary skill would have possessed a bachelor’s degree in
`
`electrical engineering, computer science, or similar field, with two years combined
`
`experience in designing and/or developing interactive location-based computer
`
`
`
`
`
`-5-
`
`
`
`

`

`Petition for Inter Partes Review of
`U.S. Patent No. 8,585,476 B2
`
`systems/software, such as video games or other simulations incorporating location
`
`information (such as GPS information associated with a user’s physical location),
`
`and in designing and/or developing computer systems/software involving graphical
`
`virtual and/or augmented reality. A person could also have qualified as a person of
`
`ordinary skill in the art with some combination of (1) more formal education (such
`
`as a master’s of science degree) and less technical experience, or (2) less formal
`
`education and more technical or professional experience. (EX1002, ¶¶21-25.)
`
`Specification Overview
`B.
` The ’476 patent states that it provides “a location-based game system in
`
`which the physical location (or physical movement) of a user on a physical playfield”
`
`obtained by, for example, a GPS device, “determines the virtual location (or virtual
`
`movement) of a virtual character on a virtual playfield.” (’476, 7:6-12.) The system
`
`may include a head-mounted display so that “video game indicia may be overlaid
`
`onto the user’s physical playfield,” such as the user’s “physical environment.”
`
`(’476, 3:41-43; see also id., 1:51-55 (listing “the physical world, a physical
`
`environment, or a defined physical playfield” as examples of a “physical
`
`playfield”).) “The physical environment can then be manipulated by a processor
`
`such that, for example, virtual indicia (e.g., a video game character or component is
`
`added) is added to the physical environment.” (’476, 3:50-53.)
`
`The ’476 patent further states that “[a] landscape detector may be provided
`
`
`
`
`
`-6-
`
`
`
`

`

`Petition for Inter Partes Review of
`U.S. Patent No. 8,585,476 B2
`
`with a location-based game system such that information on the physical terrain of
`
`the user’s physical environment may be utilized by the gaming system.” (’476, 4:8-
`
`11.) Figure 4, reproduced below, “is an illustration of landscape detection
`
`constructed in accordance with the principles of the present invention”:
`
`
`
`(’476, Fig. 4, 5:66-67; see also id., 14:57-66.) “The information provided by the
`
`landscape detector may be utilized, for example, to position where portions of a
`
`video game playfield, objects, and characters may be positioned on a display (e.g.,
`
`on a head-mounted display).” (’476, 4:14-17.) “Such information may also be used,
`
`for example, to control the movement characteristics of computer-controlled video
`
`game characters and indicia.” (’476, 4:17-20.) For example, Figure 6, reproduced
`
`below, “is an illustration of computer-controlled character positioning and
`
`movement based on detected landscape objects constructed in accordance with the
`
`
`
`
`
`-7-
`
`
`
`

`

`Petition for Inter Partes Review of
`U.S. Patent No. 8,585,476 B2
`
`principles of the present invention”:
`
`
`
`(’476, Fig. 6, 6:5-8.) Referring to Figure 6, the ’476 patent states that “[i]n
`
`environment 650, actual environmental object may be present and stored in memory
`
`as virtual objects 651 and 652.” (’476, 17:50-51.) “If the stored location of virtual
`
`objects 651 impedes on the planned movement of virtual characters, then program
`
`logic (e.g., game code) may manipulate the movement path of virtual characters
`
`around these virtual objects.” (’476, 17:56-60.)
`
`V. CLAIM CONSTRUCTION
`Petitioner does not believe express claim construction is necessary at this
`
`time. The prior art cited herein, as demonstrated below, renders the challenged
`
`claims obvious under any reasonable construction. Petitioner therefore respectfully
`
`submits that, for purposes of this IPR, express constructions are not required.
`
`
`
`
`-8-
`
`
`
`

`

`Petition for Inter Partes Review of
`U.S. Patent No. 8,585,476 B2
`
`VI. THE CHALLENGED CLAIMS ARE UNPATENTABLE
`A. Overview of Grounds
`The challenged claims attempt to lay claim to techniques for location-based
`
`video games in which a user sees their real-world surroundings combined with
`
`virtual objects. But these techniques were well-known before the ’476 patent.
`
`All grounds rely on Jaszlics (EX1003), which as its title demonstrates,
`
`describes a “system for combining virtual images with real-world scenes.” The
`
`augmented reality gaming and simulation system it describes is remarkably similar
`
`to the ’476 patent. (Cooperstock, ¶52.) Jaszlics is a strong reference that discloses
`
`the majority of the claim limitations. Focusing on war gaming applications, but also
`
`making clear that the system is broadly applicable to outdoor location-based
`
`augmented reality gaming, Jaszlics provides a “combined reality” experience for a
`
`user by combining real-time, real-world imagery of a user’s physical surroundings
`
`with images of virtual entities, such as tanks. (Id. (citing Jaszlics, 1:64-67, 2:60-
`
`63).) For example, Jaszlics describes how, using data collected with a range
`
`scanning device (such as a modified infrared surveyor’s ranging laser), the distance
`
`and shape of real-world physical features around the user are detected and used to
`
`more realistically overlay virtual entities in the real-world scene. (Id. (citing
`
`Jaszlics, Fig. 1, 4:12-23).) The combined imagery—for example, as shown by
`
`Figure 1d below—can then be presented via head-mounted display to the user.
`
`
`
`
`
`-9-
`
`
`
`

`

`Petition for Inter Partes Review of
`U.S. Patent No. 8,585,476 B2
`
`
`
`
`(Id. (citing Jaszlics, 9:36, 9:48-57, 2:10-13).)
`
`All grounds also cite to Piekarski (EX1004),
`
`which describes an actual implementation of a
`
`wearable, outdoor
`
`augmented
`
`reality
`
`system.
`
`(Cooperstock, ¶¶57-58 (citing Piekarski, e.g., pp.006
`
`(“Abstract”), 009 (Fig. 3 (at right))).) Piekarski
`
`confirms the obviousness of a “wearable” processor,
`
`and of using an optical see-through head-mounted
`
`display device that “overlays virtual indicia onto a
`
`physical playfield.” (Cooperstock, ¶¶58-59.)
`
`The additional references are cited for ancillary features cited in dependent
`
`claims. Morihira (EX1006) is cited in Ground 2 for dependent claim 2, Ohshima
`
`-10-
`
`
`
`
`

`

`Petition for Inter Partes Review of
`U.S. Patent No. 8,585,476 B2
`
`(EX1005) in Ground 3 for an alternate mapping against dependent claim 4, Robarts
`
`(EX1007) in Ground 4 for dependent claims 6-7, and Muendel (EX1008) in Ground
`
`5 for dependent claim 17.
`
`Prior Art Status of Relied-Upon References
`B.
`Pre-AIA law applies to the challenged claims, and each reference cited in the
`
`grounds listed in Part III.B qualifies as prior art to the ’476 patent. Jaszlics and
`
`Morihira qualify as prior art under § 102(b) because they issued more than one year
`
`before the priority date.1 Piekarksi also qualifies as prior art under § 102(b) because
`
`it was published more than one year before the priority date, as evidenced by public
`
`library availability. (See EX1017, ¶¶26-58.) Muendel also qualifies as prior art
`
`under § 102(b) because it is a PCT application published more than one year before
`
`
`1 The ’476 patent claims priority to a provisional application (no. 60/628,475) filed
`
`November 16, 2004. For purposes of this Petition, Petitioner has assumed a priority
`
`date of November 16, 2004. Petitioner at this time expresses no opinion on whether
`
`the ’476 patent is actually entitled to the benefit of the provisional application filing
`
`date. This issue is not relevant here because all of the prior art references relied upon
`
`predate November 16, 2004. In the event entitlement to the provisional filing date
`
`or another earlier priority date may later become relevant, Petitioner reserves the
`
`right to respond at that time.
`
`
`
`
`
`-11-
`
`
`
`

`

`Petition for Inter Partes Review of
`U.S. Patent No. 8,585,476 B2
`
`the priority date. Ohshima and Robarts qualify as prior art under § 102(e) because
`
`they issued or were published from applications filed before the priority date.
`
`C. Ground 1: Claims 1, 4-5 and 13 Are Obvious Over Jaszlics in view
`of Piekarski
`Independent Claim 1: “An augmented reality game system
`
`comprising:” (1[pre])
`Assuming the preamble provides a claim limitation, Jaszlics discloses it.
`
`To start, Jaszlics discloses an “augmented reality…system,” which Jaszlics
`
`refers to as “combined reality.” (Jaszlics, col. 21, Table 7 (“Glossary of Terms”
`
`defining “combined reality” as “Apparent reality, with virtual elements projected
`
`into the image of the real world. Virtual and live elements may interact through
`
`simulation.”); Cooperstock, ¶85.) In particular, Jaszlics discloses “systems for
`
`combining virtual images with real-world scenes” where “[t]he resulting combined
`
`image is presented in [a display]” to a user. (Jaszlics, 1:13-15, 2:40-56.) Figure 1d
`
`of Jaszlics illustrates an example of a combined image, where a virtual image of a
`
`tank is rendered in a real-world scene to appear as though it is behind a tree:
`
`
`
`
`
`-12-
`
`
`
`

`

`Petition for Inter Partes Review of
`U.S. Patent No. 8,585,476 B2
`
`
`
`
`(Jaszlics, Fig. 1; see also id., 3:13-16 (“FIGS. 1a through 1d are examples of the
`
`real-world scene, a virtual image, the resulting masked virtual image, and combined
`
`images, respectively.”).) The display can take the form of “electronic display
`
`googles,” where “three-dimensional virtual elements” are injected “into the view of
`
`the real world.” (Jaszlics, 2:10-13.)
`
`A person of ordinary skill in the art would have understood that Jaszlics’
`
`disclosure of combining virtual images with real-world scenes describes “augmented
`
`reality” (or “AR”). (Cooperstock, ¶86.) For example, like Jaszlics’ “combined
`
`reality,” the ’476 patent similarly states that “Augmented Reality (AR) systems are
`
`provided in which video game indicia are overlaid onto a user’s physical
`
`environment.” (’476, Abstract.) More generally, Jaszlics’ description of “combined
`
`
`
`
`
`-13-
`
`
`
`

`

`Petition for Inter Partes Review of
`U.S. Patent No. 8,585,476 B2
`
`reality” is consistent with the ordinary understanding in the art of “augmented
`
`reality.” (Cooperstock, ¶86 (citing EX1010, p.007).)
`
` Jaszlics further discloses and renders obvious an “augmented reality game
`
`system.” (Cooperstock, ¶87.) For starters, Jaszlics specifically describes using its
`
`system for virtual war games, and related game play, such as engaging in a virtual
`
`tank battle. (Jaszlics, e.g., 10:15-25, 11:4-6, 15:15-30.) This is just like the ’476
`
`patent, which similarly states that “an AR game system may be utilized in the
`
`military training of soldiers” and specifically recognizes this sort of military
`
`simulation as “game play.” (’476, 5:10-11; see also id., 5:11-14 (“game is played.”),
`
`15:12-14 (“game play.”).)
`
`More specifically, Jaszlics explains that “[t]he present invention can be used
`
`in these war games to generate virtual tanks that appear to move about the terrain.
`
`This can completely eliminate the need, for example, for live tanks representing the
`
`enemy force in a war gaming exercise. Virtual vehicles and real vehicles can appear
`
`in the same scene.” (Jaszlics, 1:53-58.) Jaszlics describes military simulation game
`
`play that includes a player/user (which Jaszlics refers to as an “observer”2) engaging
`
`a virtual combat vehicle and taking damage or getting “killed.” (Jaszlics, 10:15-25
`
`
`2 See Jaszlics, 10:11-17 (“The observer 113 of the display is assumed to be the user
`
`of the invention….”).
`
`
`
`
`
`-14-
`
`
`
`

`

`Petition for Inter Partes Review of
`U.S. Patent No. 8,585,476 B2
`
`(“An example of such direct interaction is the engagement of a virtual opponent from
`
`a combat vehicle, such as a battle tank.”), 11:59-63 (“For example, if one of the
`
`virtual tanks represented in this simulation engage the avatar of the observer’s tank,
`
`the resulting damage information will activate damage displays, or a ‘killed’
`
`indicator in the observer’s sensors and controls suite.”); see also id., 11:4-6 (“virtual
`
`battle damage”), 15:53-56 (“The virtual entities implemented by the virtual
`
`simulation and control software are capable of engaging the avatar of the observer’s
`
`vehicle, depending on the scenario loaded in.”), 15:25-30 (“engaging the virtual T-
`
`80s operated by the virtual simulation”), 10:39-44 (engaging virtual tank).) Jaszlics’
`
`disclosure of using its augmented reality system for simulated war games therefore
`
`discloses an “augmented reality game system.” (Cooperstock, ¶88.)
`
`Putting aside its military-specific gaming applications, Jaszlics expressly
`
`teaches that its system can be used as “an interactive, recreational device,” and is
`
`broadly applicable to gaming generally, explaining that, for example, “[a]nother
`
`possible field of use is in games, such as laser tag, so that virtual players, objects,
`
`and special effects can be combined with real players and real objects in the playing
`
`field.”3 (Jaszlics, 1:64-67, 18:48-55; see also id., 15:21-25 (“Other scenarios not
`
`related to tanks, or military training, can be selected by a user….”).) Jaszlics
`
`
`3 All underlining added to quoted material unless noted.
`
`-15-
`
`
`
`
`

`

`Petition for Inter Partes Review of
`U.S. Patent No. 8,585,476 B2
`
`therefore discloses and renders obvious an “augmented reality game system” in
`
`which users can play laser tag-like and other games involving the user’s physical
`
`surroundings and virtual elements. (Cooperstock, ¶89.)
`
`(a)
`
`“a head-mounted display that overlays virtual indicia
`onto a physical playfield;” (1[a])
`Jaszlics’ augmented reality system includes “a head-mounted display,” such
`
`as “electronic display goggles,” “virtual reality goggles or visors worn by the
`
`observer [i.e., the user of the system].” (Jaszlics, 9:48-57; see also id., 2:10-13 (“The
`
`present invention can be used with electronic display goggles to greatly extend this
`
`entertainment medium by injecting, in real time, three-dimensional virtual elements
`
`into the view of the real world.”), 9:34-39 (“The display 111 may be any display
`
`system capable of displaying the combined image, such as…an electronic goggle or
`
`visor….”), 10:52-57 (“The position and orientation of the observer (including the
`
`movement and attitude of the vehicle, if any, housing the observer) is sensed by
`
`observer sensors and controls 115 such as a magnetic or infrared head tracker system
`
`used in conjunction with virtual reality goggles.”).) Jaszlics therefore discloses “a
`
`head-mounted display,” as claimed. (Cooperstock, ¶90.)
`
`With respect to “overlay[ing] virtual indicia onto a physical playfield,” the
`
`’476 patent describes “physical playfield” as broadly including “the physical world,
`
`a physical environment, or a defined physical playfield[.]” (’476, 1:51-55.)
`
`
`
`
`
`-16-
`
`
`
`

`

`Petition for Inter Partes Review of
`U.S. Patent No. 8,585,476 B2
`
`Jaszlics teaches that the “electronic display goggles” or “visors” (“head-
`
`mounted display”) “overlay[] virtual indicia onto a physical playfield,” by for
`
`example, displaying virtual tanks (“virtual indicia”) overlaid within a physical
`
`landscape surrounding the user (“physical playfield”) for participating in a virtual
`
`war games exercise, as illustrated in Figure 1:
`
`
`(Jaszlics, Fig. 1; see also id., 4:19-23 (“The masked virtual image and the image of
`
`the real world are then combined in such a manner that in the resulting combined
`
`image (FIG. 1d) the virtual object will appear to exist and move in the real world at
`
`its virtual position.”), 8:53-56 (“In this step 110 of FIG. 2, the virtual images
`
`constructed in the virtual image generation step 109 are combined with the real-
`
`world image 114 originating from the video camera 100.”), 8:7-16 (“three battle
`
`tanks are animated.”); Cooperstock, ¶92.)
`
`-17-
`
`
`
`
`
`

`

`Petition for Inter Partes Review of
`U.S. Patent No. 8,585,476 B2
`
`As another example, Figure 14 of Jaszlics (below) illustrates virtual tanks 162
`
`(“virtual indicia”) overlaid on hills 160 of the user’s physical environment
`
`(“physical playfield”):
`
`
`(Jaszlics, Fig. 14; see also id., 3:49-51, 9:60-62 (describing Fig. 14); EX1011
`
`(Jaszlics Provisional), 00152, Fig. 14 (identifying 160 as “Hills” and 162 as “Virtual
`
`Entities”); Cooperstock, ¶93.)
`
`As a third illustration, referring to Figure 15 (below), Jaszlics explains how
`
`the overlaid “virtual indicia” can be a variety of virtual elements, such as a path
`
`marker 172, a unit boundary 173, a navigational marker 174, and a phase line 175,
`
`which are overlaid on terrain 170:
`
`
`
`
`
`-18-
`
`
`
`

`

`Petition for Inter Partes Review of
`U.S. Patent No. 8,585,476 B2
`
`
`
`
`(Jaszlics, Fig. 15; see also id., 9:64-10:9 (“The virtual entities generated by a virtual
`
`entity simulation do not necessarily represent a real-world object. For example, they
`
`may represent fictional characters or entities, or views of conceptual objects. An
`
`example of conceptual virtual entities is shown in FIG. 15. In FIG. 15, the real-
`
`world objects 170 include the terrain, a tree, and a battle tank, in which the observer
`
`or observers 171 of a combined reality