JDM/018 CON4
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`N THE UNITED STATES PATENT AND TRADEMARK OFFICE
`
`
`
`
`
`Jeffrey David Mullen
`
`Not Yet Assigned
`
`Not Yet Assigned
`
`
`
`
`
`
`Herewith
`
`
`
`
`
`
`
`
`
`
`AUGMENTED REALITY VIDEO GAME
`
`
`
`
`SYSTEMS
`
`
`Not Yet Assigned
`
`
`
`Not Yet Assigned
`
`:
`
`:
`
`:
`
`:
`
`;
`
`:
`
`:
`
`Applicant
`
`Application No.
`
`Confirmation No.
`
`
`
`Filed
`
`For
`
`Group Art Unit
`
`Examiner
`
`
`
`
`
`Hon. Commissioner for Patents
`
`P.O. Box 1450
`Alexandria, VA 22313-1450
`
`
`
`
`
`Pittsburgh, PA 15116
`January 4, 2022
`
`
`
`
`
`
`PRELIMINARY AMENDMENT
`
`Sir:
`
`
`Prior to examination of the application,
`
`
`applicant hereby amends the application as follows:
`
`
`Amendments to the Specification are reflected on
`
`
`page 2 of this paper.
`
`
`
`Amendments to the Claims are reflected in the
`
`
`listing of claims which begins on page 3 of this paper.
`
`
`
`
`Remarks/Arguments begin on page 5 of this paper.
`
`Meta Exhibit 1007
`Meta v. Mullen - Page 001
`
`Meta Exhibit 1007
`Meta v. Mullen - Page 001
`
`

`

`Amendments to Specii
`
` fication
`
`
`Applicant hereby replaces paragraph 1 of the
`
`
`specification with the following paragraph
`
`showing
`
`
`
`amendments.
`
`—-
`
`This
`
`
`
`
`application is a continuation of U.S. Patent
`
`
`No. 16/684,357, filed November 14, 2019, wh
`
` Application No. 15/975,585
`continuation of
`
`2019, which is a continuation of U.S. Paten
`
`No. 15/618,090
`
`
`filed on June 8, 2017, which
`
`is a
`
`ich is a
`
`filed on May 9,
`
` t Application
`
`
`
`continuation o
`
`f U.S.
`
`Patent Application No.
`
`12/249,925
`
`
`
`
`filed on October 11 2008, which claims the benefit of U.S.
`
`
`
` reference herein in their +s entirety.--
`
`Provisional Patent Application No. 60/979,3
`
`
`79 filed on
`
`October 11, 2007, which are +s hereby incorporated by
`
`
`
`Meta Exhibit 1007
`Meta v. Mullen - Page 002
`
`Meta Exhibit 1007
`Meta v. Mullen - Page 002
`
`

`

`Amendments to the Claims:
`
`
`This listing of claims will
`
`replace all prior
`
`
`versions, and listings, of claims in the application:
`
`
`Listing of Claims:
`
`1-3.
`
`(cancelled)
`
`4,
`
`(new) A system comprising:
`
`
`a head-mounted display having a video camera for
`
`
`obtaining images in view of said head-mounted display and
`
`
`
`utilizing said images to detect a surface of a physical
`
`object included in said images, wherein said head-mounted
`
`display is operable to display a virtual object associated
`
`
`with an application, and a physical object type of said
`
`
`physical object is operable to be determined from several
`
`physical object types; and
`
`
`a hand-held controller for providing a first
`
`object and a second three-dimensional control signal to
`
`change the movement of said virtual object
`
`from the
`
`
`
`three-dimensional control signal to move said virtual
`
`
`
`
`
`
`
`
`
`
`movement provided from said first three-dimensional control
`
`Signal, wherein said first three-dimensional control signal
`
`
`
`moves said virtual object three-dimensionally over at least
`
`
`three axis of movement
`
`
`in a first manner and said second
`
`three-dimensional control signal moves said virtual object
`
`
`
`
`
`
`
`
`three-dimensionally over at least three axis of movement
`
`
`
`in
`
`a second manner causing said virtual object to interact
`
`
`with said detected surface.
`
`Meta Exhibit 1007
`Meta v. Mullen - Page 003
`
`Meta Exhibit 1007
`Meta v. Mullen - Page 003
`
`

`

`
`
`5.
`
`
`(new) The system of claim 4, wherein said
`
`head-mounted display comprises a non-transparent display.
`
`6.
`
`
`(new) The system of claim 4, wherein said
`
`application is a video game having a
`
`selectable playing
`
`area.
`
`7.
`
`
`
`(new) The system of claim 4, wherein said
`
`
`
`game controller includes buttons.
`
`8.
`
`
`(new) The system of claim 4, wherein said
`
`wherein said application is a video game that is a board
`
`game.
`
`9.
`
`
`(new) The system of claim 4, wherein said
`
`wherein said application is a video game that is a sports
`
`game.
`
`10.
`
`
`
`(new) The system of cl aim 4, wherein said
`
`head mounted display is powered wirelessly.
`
`
`
`11.
`
`
`(new) The system of cl
`
`aim 4, wherein said
`
`controller is powered wirelessly.
`
`12.
`
`
`
`(new) The system of claim 4, further
`
`comprising a second camera.
`
`13.
`
`
`(new) The system of claim 4, further
`
`comprising a second camera and a third camera.
`
`
`
`Meta Exhibit 1007
`Meta v. Mullen - Page 004
`
`Meta Exhibit 1007
`Meta v. Mullen - Page 004
`
`

`

`14.
`
`
`
`(new) The system of claim 4, further
`
`
`
`comprising a second camera and a third camera and the views
`
` of said second and third camera overlap.
`
`
`
`
`
`
`
`(new) The system of claim 4, further
`
`15.
`
`
`
`
`
`comprising a telephonic communications component.
`
`16.
`
`
`
`(new) The system of claim 4, further
`
`comprising a selected virtual environment.
`
`17.
`
`
`
`(new) The system of claim 4, further
`
`comprising a switch.
`
`18.
`
`
`
`(new) The system of claim 4, further
`
`comprising a microphone.
`
`19.
`
`
`
`(new) The system of claim 4, further
`
`
`comprising a first microphone and a second microphone.
`
`20.
`
`
`
`
`(new) The system of claim 4, wherein a first
`
`
`
`type of said several physical object types is a floor anda
`
`
`second type of said several types of physical object types
`
`
`
`is a wall.
`
`21.
`
`
`
`(new) The system of claim 4, further wherein
`
`said head-mounted display comprises a transparent display.
`
`22.
`
`
`
`(new) The system of claim 4, further
`
`comprising a microphone and a second camera.
`
`Meta Exhibit 1007
`Meta v. Mullen - Page 005
`
`
`
`
`
`Meta Exhibit 1007
`Meta v. Mullen - Page 005
`
`

`

`23.
`
`
`(new) The system of claim 4, further
`
`
`
`comprising a microphone and a second camera, wherein said
`
`head-mounted display comprises a transparent display.
`
`Meta Exhibit 1007
`Meta v. Mullen - Page 006
`
`Meta Exhibit 1007
`Meta v. Mullen - Page 006
`
`

`

`ve)
`
`EMARKS /ARGUMENTS
`
`
`Summary of Applicant's Amendments
`
`App licant has cancelled c]l aims 1-3 without
`
`prejudice in order to expedite prosecution.
`
` App
`
`
`
`licant has added new claims 4-23 in order to
`
`more particularly point out and distinctly claim the
`
`subject matter that applicant regards as the invention.
`
`
`
`
`
`is in condition
`
`Conclusion
`
`Applicant respect
`
`fully submits that this
`
`application,
`
`including the pending claims,
`
`for allowance. Reconsidera
`
` this application are respec
`
`tion and prompt allowance of
`
`
`full
`
`y reguested.
`
`The
`Director is hereby authorized to charge any
`
`
`fees required in conjunction with this
`
` filing,
`
`or credit
`
`
`any overpayment of
`
`the same
`
`t
`
`to
`
` Deposit Account No. 50-
`
`3855, Order No. J
`DM/018 CON3.
`
`
`Respectfully Submitted,
`
`
`
`
`rey D.
`/Jeft
`
`
`Je
`D.
`
` tomer
` licant
` Frey
`
`
`
`
`22
`
`Mullen/
`Mullen
`(Reg. No.
`App
`Cus
`No. 32733
`12 Hassinger Lane
`PA 15116
`Glenshaw,
`Mobile:
`(914) 837-7741
`
`
`
`52,056)
`
`Meta Exhibit 1007
`Meta v. Mullen - Page 007
`
`Meta Exhibit 1007
`Meta v. Mullen - Page 007
`
`

`

`
`
`Meta Exhibit 1007
`Meta v. Mullen - Page 008
`
`

`

`FIG.2
`
`Meta Exhibit 1007
`Meta v. Mullen - Page 009
`
`Meta Exhibit 1007
`Meta v. Mullen - Page 009
`
`

`

`
`
`\ hs
`
`oy
`
`FIG.3
`
`Meta Exhibit 1007
`Meta v. Mullen - Page 0010
`
`Meta Exhibit 1007
`Meta v. Mullen - Page 0010
`
`

`

`
`
`4/16
`400
`
`io Wn]
`Nay, =
`
`PLAYER1WON!
`
`Zz
`=== —_
`T16
`—-“aq
`owSwi
`~ 2 a
`$58
`o o
`
`FIG.4
`
`Meta Exhibit 1007
`Meta v. Mullen - Page 0011
`
`Meta Exhibit 1007
`Meta v. Mullen - Page 0011
`
`

`

`5/16
`500
`
`(o)
`
`St4
`
`oY WSS=
`
`FIG.5
`
`Meta Exhibit 1007
`Meta v. Mullen - Page 0012
`
`Meta Exhibit 1007
`Meta v. Mullen - Page 0012
`
`

`

`
`
`—
`
`Q KROSS
`
`>
`
`z
`
`=a
`
`ie
`tow
`fo
`wo <
`$
`
`*
`
`\—an
`SO SSN
`
`N
`Pe
`
`SS
`
`SCS
`
`0
`
`BE
`
`LL
`of
`©
`
`co
`
`:
`
`LL
`
`0
`
`ay|
`PK
`
`2
`2
`os
`O& wi
`<<
`
`©
`
`™N
`
`————
`
`fog §
`“hy psNK— \
`Lp9
`
`60
`
`Meta Exhibit 1007
`Meta v. Mullen - Page 0013
`
`Meta Exhibit 1007
`Meta v. Mullen - Page 0013
`
`

`

`7/16
`£00
`
`
`
`YL
`
`OSos4h (__we
`CARRACING
`
`7
`
`0
`
`FIG.7
`
`
`
`*paUSEDPICKANOTHERGAME?
`
`70
`
`
`
`AIRPLANEBATTLE
`
`MONOPOLY
`
`BATTLESHIP
`
`Meta Exhibit 1007
`Meta v. Mullen - Page 0014
`
`Meta Exhibit 1007
`Meta v. Mullen - Page 0014
`
`

`

`0
`
`8/16
`800
`
`c08
`
`8‘Sid
`
`Meta Exhibit 1007
`Meta v. Mullen - Page 0015
`
`Meta Exhibit 1007
`Meta v. Mullen - Page 0015
`
`
`

`

`9/16
`900
`
`su ie
`zy
`
`IG7°
`
`N —Wh
`
`FIG.9
`
`Meta Exhibit 1007
`Meta v. Mullen - Page 0016
`
`Meta Exhibit 1007
`Meta v. Mullen - Page 0016
`
`

`

`x
`
`10/16
`1000
`
`FIG.10
`
`+xo
`
`O a> +—o>
`
`Meta Exhibit 1007
`Meta v. Mullen - Page 0017
`
`Meta Exhibit 1007
`Meta v. Mullen - Page 0017
`
`

`

`11/16
`1100
`
`
`
`Http://www.JeffreyMullen.com
`
`WELCOMETOJEFFREYMULLEN.COM
`
`FIG.11
`
`Wu
`Fe
`
`o=
`
`at|io
`
`a a=a
`
`moW
`
`w
`
`oO
`<oo
`
`Meta Exhibit 1007
`Meta v. Mullen - Page 0018
`
`Meta Exhibit 1007
`Meta v. Mullen - Page 0018
`
`

`

`12/16
`
`
`
`Le
`
`
`waAVTdVIGAW|arooaYaLndwoo 6
`
`
`aWYSadyuvogAYWHYSITAlfxogdannr
`
`T1vaLaxsvgyoloaroudquvosiuva
`
`
`ANIHDVIN*1178348£aGhousYSLVSHLSIAOW
`LNAIWNOMIANAYW319|30v1d3y
`
`
`
`|A1avLYaMalAayPaiANOHdJSON
`dOOHa:
`
`
`SYVAACYYHTYNLYIALoalas
`
`waisas|aWOH|auwmMyos|ova Sccl
`
`6cl
`
`cb“Sls
`
`Meta Exhibit 1007
`Meta v. Mullen - Page 0019
`
`Meta Exhibit 1007
`Meta v. Mullen - Page 0019
`
`

`

`13/16
`
`Pre
`
`Sve
`
`SVE
`
`Lve
`
`87e
`
`NVEL
`
`a79norWwoowaLvdTvTVAINEYSpoNw|auosWwnthr
`
`
`
`
`VWNYOMAAYSVLSINHD6Shiay20>ANVL
`
`
`
`ANWNoWLBosh.yuvoLldW305ANW1d
`
`
`LNAWNOUIANAYvaI0|mainayd|asn|6El616ob
`
`
`NVISYaavowtHovaa
`3Yynloa1Evel
`waisas|3woH|aywmyos|ova
`
`SLNAWNOYIANATYNLYIALOA14S
`
`
`
`6re
`
`OSe
`
`WuvdSWAHL
`
`eb‘Sid
`
`Meta Exhibit 1007
`Meta v. Mullen - Page 0020
`
`Meta Exhibit 1007
`Meta v. Mullen - Page 0020
`
`
`

`

`
`
` %.
`
`
`
`Meta Exhibit 1007
`Meta v. Mullen - Page 0021
`
`

`

`wauyyoo,|!|VARYHOOTSSLvayyooT4!rSSl
`~~~"SggGf—561
`
`CaNINYALaa
`
`
`
`
`
`WauAV1dSId117M
`
`GaNIWYyaLlad
`
`|||||
`
`
`
`||||OANIWYaLaarTaonmnraaSer§S}__-
`
`iFTTTTTT
`rWauyyoold
`
`
`15/16
`1500
`
`aunlolid
`
`Meta Exhibit 1007
`Meta v. Mullen - Page 0022
`
`Meta Exhibit 1007
`Meta v. Mullen - Page 0022
`
`
`
`

`

`16/16
`1600
`
`bLAW1dS10
`
`éAW1dSI0
`
`9b‘Sls
`
`Meta Exhibit 1007
`Meta v. Mullen - Page 0023
`
`Meta Exhibit 1007
`Meta v. Mullen - Page 0023
`
`

`

`JDM/018
`
`
`
`
`
`
`
`
`
`
`
`
`
`AUGMENTED REALITY VIDEO GAME SYSTEMS
`
`
`
`Cross-Reference to Related Applications
`
`
`
`claims the benefit o
`
`[0001]
`
`This application
`
`Provisional Patent Applica
`
`Oc
`
`re
`
`tober 11, 2007, which is
`
` ference herein in its ent
`
`
`
`
`tion No. 60/979,379, filed
`
`hereby incorporated by
`
` tirety.
`
`
`
`
`Background of the Invention
`
`[0002]
`
`10
`
`This invention relates to augmented reality.
`
`
`
`
`
`Summary of the Invention
`
`[0003]
`
`Augmented reality video game systems are
`
`provided.
`
`Video game indicia may be perceived to be
`
`located in
`
`a user’s physical environment
`
`through
`
`15
`
`virtual indicia that are placed on a head-mounted
`
`display.
`
`The head-mounted
`
`display may be transparent
`
`that the user can look through the transparent
`
`lay and see his/her physical environment. Video
`
`disp]
`
`such
`
`
`
`game indicia may then be provided on this transparent
`
`20
`
`screen such that video game indicia are introduced into
`
`
`
`the user’s perspective of
`
` f his/her environment.
`
`Alternatively,
`
`video game indicia may be interlaced
`
`Meta Exhibit 1007
`Meta v. Mullen - Page 0024
`
`Meta Exhibit 1007
`Meta v. Mullen - Page 0024
`
`

`

`back into image data that was captured
`
`(e.g., video)
`
`and this interlaced image data may be displayed t
`oT
`
`user
`
`(e.g
`
`via a head-mounted display)
`
`such that
`
`user can see an image of
`
`a non-transparent
`
`display may provide a
`
`Accordingly,
`
`for
`
`oa
`
`the
`
`the
`
`user
`
`terlaced video game indicia.
`
`
` his/her environment and
`
`
`
`
`example,
`
`in
`
` wi
`
`th a view o
`
`
`
`
`
`f his/her environment
`
`and interlaced
`
`virtual objects.
`
`Alternatively,
`
`for example,
`
`the
`
`video
`
`
`of a user’s environment may be interlaced with virtual
`
`10
`
`objects and displayed on a transparent display.
`
`A
`
`head-mount
`
`ted display may be provided and may include
`
`
`
`multiple
`
`example,
`
`transparent or non-transparent displays.
`
`
`
`ferent
`a dif
`
`
`
`display may be provided
`
`
`for each eye.
`
`transparent or non-transparent
`
`
`
` fferent
`Di
`
`For
`
`video
`
`15
`
`may be captured
`
`for each eye’s display
`
`
`
`(e.g.,
`
`
`fro
`
`m two
`
`or more video cameras)
`
`
`interlaced into each of
`
`to the displays
`
`in or near real
`
`the video streams and pro
`
`
`
`for ex
`If,
`
`time.
`
`and virtual indicia may be
`
`each display is transparent,
`
`20
`
`directly displayed on such a
`
`transparent display.
`
`transparent or non-transparent displays may be
`
`Fl
`
`
`
` virtual indicia may
`
`
`
`
`A camera may be att
`
`tached to the head-mounted
`
`display to
`
`continually capture images
`
`(e.g., avi
`
`
`
`displays.
`
`[0004]
`
`25
`
`camera)
`
`oO
`
`from this
`
` this way,
`
`example,
`
`
`
`vided
`
`ample,
`
`be
`
`such
`
`exible
`
`deo
`
`ted
`
`In
`
`
`
`f a user’s environment.
`
`The data collec
`
`<>
`for
`camera may be processed such tha
`
`
`
`the surfaces of objects may be de
`
`
`
`
`tected.
`
`as data is collected, processed,
`
`video game indicia may interact with objects
`
`
` affect
`and utilized to
`
`30
`
`video game indicia in real-time.
`
`[0005]
`
`Accordingly,
`
` for example, an augmented
`
`reality console may be placed underneath a table.
`
`Two
`
`users may then sit at a table and turn ON their head-
`
`Meta Exhibit 1007
`Meta v. Mullen - Page 0025
`
`Meta Exhibit 1007
`Meta v. Mullen - Page 0025
`
`

`

`mounted displays.
`
`The video camera
`
` from the head-
`
`mounted displays may detect
`
`The two users may then
`
`the pre
`
`
`
`
`sence of
`
`a table.
`
`
`ay a fighting game.
`
`select to pl
`
`
`
`
`
`A video game character may then be placed on the table
`
`for each person by the
`
`video game system by changing
`
`
`f the table
`
`by providing video
`
`
` the user’s perspective o
`
`
`game indicia onto
`
`the head-mounted display.
`
`Accordingly,
`
`the users may utilize
`
`their
`
`respec
`
`hand-held
`
`
`
`
`
`tive fighters.
`Ifa
`
`
`
`to the
`
`table like a large
`
`controllers to control
`
`10
`
`user adds a physical object
`
`
`
`book,
`
`the detection system
`
`(e.g.,
`
`Vv
`
`ideo camera or radar
`
`and associated processing)
`may dete
`
`the fighter
`
`the book. Accordingly,
`
`
`
`ct the surfaces of
`
`s may be able to,
`
`for example,
`
`jump onto the book or duck and hide behind
`
`15
`
`the book (e.g.,
`
`
`
`
`line-of-
`Fire of
`
`such
`
`that a fighter
`
`another
`
`fighter).
`
`is removed from the
`
`The components of a
`
`game console may be provided in a head-mounted display
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`such that
`
`video game console capabilities may be
`
`portable.
`
`Positioning systems
`
`(e.g
`
`systems or satellite-based position
`
`
`
`included in a head-mounted display,
`
`
`
`other device.
`
`other device.
`
`
`
`
`
`
`
`
`
`7 i
`
`local positioning
`
`ng systems) may be
`
`controller, or
`
`tection systems may
`
`controller, or
`
`may be used with
`
`20
`
`30
`
`Inertial movement de
`
`be included in a head-mounted display,
`
`25
`
`inertial
`
`
`accuracy of
`
`
`
`location determinations
`
`movement systems in order to increase the
`
`Positioning systems
`
`
`
` for a head-mounted
`
`Particularly,
`
`for
`
`
`
`
`
`
`
`display, controller or o1
`
`ther device.
`
`
`
`example,
`
`a positioning system may periodically
`
`determine the location of
`
`fF a user an
`
`d the inertial
`
`movement system may determine the location of
`
`a user
`
`between the periodic position determinations by the
`
`positioning system.
`
`An inertial system may include,
`
`Meta Exhibit 1007
`Meta v. Mullen - Page 0026
`
`Meta Exhibit 1007
`Meta v. Mullen - Page 0026
`
`

`

`
`
`for example, an array of one or more accelerometers
`
`and/or gyroscopes.
`
`[0006]
`
`An augmented reality game may be played
`
`remotely wit
`
`respective location,
`
`th another augmented reality system.
`
`
`for example, of characters to an
`
`
`
`The
`
`
`
`origin point may be transmitted to another augmented
`
`reality syst
`
`tem and this may be utilized to generate
`
`
`
`video game indicia at the
`
`other system. Additionally,
`
`
`control information (e.g.,
`
`
`ACTION button) may be
`
`10
`
`transmitted between syst
`
`tems.
`
`[0007]
`
`Gyroscopes and/or accelerometers (e€.g.,
`
`a head-mounted display to
`
`determine the viewing
`
`tion devices) may be utilized in
`
`
`
`
`
`inertial movement detec
`
`
`perspective of the user as well as the change in
`
`
`location of a user.
`
`For example,
`
`the user may be
`
`
`
`
` al movement determination
`
`
`
`looking at the ceiling in one location at one instance
`
`a second instance.
`
`systems may be utilized
`
`
`accel eration of an object
`
`
`
` floor in a second location at
`and may be looking at the
`
`
`Inerti
`
`
`to,
`for example, determine the
`
`disp]lay). This accelerati
`
`
`
`
`20
`
`25
`
`30
`
`(e.g., a head-mounted
`
`
`on information may be
`
`utilized,
`
`in turn,
`
`
`to determine the velocity of an
`
`
`object. This velocity information may be utilized,
`
`in
`
`turn,
`
`
`to determine the position of a device (e.g., with
`
`respect to an origin such
`
`as a periodic position
`
`
`
`
`
`determination from a positioning system such as a
`
`receiver
`
`for receiving global positioning system
`
`satellite signals.
`
`
`
`[0008]
`
`Computer-control
`
`
`
`
`provided as part of a video game.
`
`[led video game indicia may be
`
`A user may control,
`
`via,
`
` for example, hand-hel
`
`ld controls the movement and
`
`action of
`
`
`
`fF one or more video game indicia. Hand-held
`
`controls may include,
`
`
`for
`
`example, a controller such as
`
`Meta Exhibit 1007
`Meta v. Mullen - Page 0027
`
`Meta Exhibit 1007
`Meta v. Mullen - Page 0027
`
`

`

`
`
`a control object (e.g., cylinder) or glove.
`
`A user may
`
`utilize multiple controllers to play a game or utilize
`
`an augmented reality system
`
`(e.g.,
`
`two controllers —
`
`one for each hand).
`
`Augmented-reality environments may
`
`
`be utilized to augment th
`perspective of physical
`
`
`
`
`
`
`
`objects.
`
`For example,
`
`augmented wat
`
`ter may be placed on
`
`the
`
`
`top of a table such that an augmented reality boat
`
`racing,
`
`
`swimming, surfing,
`
`scuba diving, and/or
`
`
`fishing
`
`game may be played.
`
`10
`
`[0009]
`
`
`Inertial movement determination devices may
`
`also be utilized in controllers,
`
` outputs
`control signals may be based off
`
`from such
`
`inertial movement determination devices.
`
`[0010]
`
`A game console may provide wireless
`
`
` for example, and
`
`
`communication signals that each head-mounted device
`
`(HMD) device
`
`(and/or controllers)
`
`can receive and
`
`to assist in locating the
`
`location with respect to the console.
`
`HMDs’
`
`(and/or
`
`
`
`
`
`may also be determined by
`
`
`
`to look at the playfield
`
`
`
`
`
`
`and then location can be determined,
`
`
`
`off the user perspectives of
`
`
`in software,
`
`based
`
`
`the physical playfield
`
`
`
`(e.g.,
`
`the table).
`
`Devices may be placed around the
`
`
`
`
`
`
`room to provide
`
`wireless communications signals
`
`in a triangulation algorithm)
`
`that
`
`
`
`can be utilized (e.g.,
`
`the location of
`
`a HMD.
`
`
`
`The location of
`
`the playing
`
`
` the location
`
`
`
`
`
`
`
`
`
`
`area may be configured during a setup feature of a
`
`video game system and a user may control
`
`
`
`and size of the playfield during the setup
`
`feature.
`
`[0011]
`Games may be constructed to be played on
`
`
`different playfields.
`
`For example,
`
`a user may be
`
`prompted to
`
`go to their kitchen table and pl
`
`lay on their
`
`kitchen table.
`
`The user may be prompted to notiit
`
` fy the
`
`Meta Exhibit 1007
`Meta v. Mullen - Page 0028
`
`
`
` utilize
`
`
`
`
`
`controllers’ )
`
`The
`
`
`locations of
`
`
`
`two HMDs’
`
`instructing each of
`the HMDs
`
`15
`
`20
`
`25
`
`30
`
`Meta Exhibit 1007
`Meta v. Mullen - Page 0028
`
`

`

`system when the user is looking a
`
`t the kitchen table.
`
`
`
`
`The user may also be prompted
`
`looking at,
`
`system may detect the kitchen
`
`the kitchen table.
`
`
`
`
`
`
`
` that a game
`
`while
`
`the
`
`
`
`to walk around,
`
`In this manner,
`
`table such
`
`
`
`may be played on the kitchen tabl
`
`e. Similarly,
`
`a game
`
`wall. Similarly
`
`may request that a user look ata
`
`
`for example,
`still,
`
`
`
`
`
`
`
`
`a game may request that the user
`
`
`sits at the bottom of a stairwell and looks up in order
`
`to play a game.
`
`A single game may have levels that
`
`
`
`
`utilize different types of physical surfaces (e.g.,
`
`
`tabletop, wall, stairwell, floor, sink, bed, couch, TV
`
`screen).
`
`[0012]
`
`A head-mounted display may also be utilized,
`
` for example,
`wn uch that a user cannot see any o
`
`to immerse a user in
`
`f his/her
`
` utilized,
`
`surroundings.
`
`Such a virtual rea
`
`lity topology may be
`
`
`attained from transparent or non-t
`
`ransparent display.
`
`[0013]
`
`An augmented reality game console may be
`
` for example, as an augme
`
`nted reality
`
` to a virtual world
`
`10
`
`15
`
`20
`
`computing
`
`
`platform. Accordingly,
`
`
`
`may utilize an augmented reality console
`
`for example, a user
`
`(which may be
`
`located inside a head-mounted display)
`
`to display
`
`
`
`virtual objects such as virtual hardware.
`
`Accordingly,
`
`a user may create virtual hardware such as virtual
`
`
`
`25
`
`projectors,
`
`television sets, cellu
`
`game devices
`
`(e.g., virtual instru
`
`lar phones, virtual
`
`ments such as virtual
`
`drums,
`
`delete,
`
`virtual gaming playmats).
`
`A user may add,
`
`and modi
`
` fy virtual hardwar
`
`e to his/her physical
`
`environment utilizing a virtual
`
`gr
`
`aphical user
`
`30
`
`
`interface.
`
`
`
`
`
`[0014]
`
`allow,
`
`
`
`environment. Accordingly,
`
`for example, a user to chan
`
`ge his/her
`
`An augmented reality computing platform may
`
`
`
`
`
`an augmented reality
`
`Meta Exhibit 1007
`Meta v. Mullen - Page 0029
`
`Meta Exhibit 1007
`Meta v. Mullen - Page 0029
`
`

`

`
`computing platform may allow a user to select a theme
`
`
`for his/her physical environment.
`
`A theme may be,
`
`for
`
`
`
`example, an Americana theme. Another theme may be,
`
`
`for
`
`example,
`
`a Japanese theme. Accordingly, virtual
`
`Americana objects may be placed around a user’s
`
`physical environment when an Americana theme is
`
`
`
`selected.
`
`These virtual objects may be stationary or
`
`may move in a pre-determined or customized manner.
`
`A
`
`user may select,
`
`
`for example,
`
`10
`
`desires and then add associated virtual hardware
`
` fferent
`Virtual hardware objects may have di
`
`objects.
`
` the environment the user
`
`
`
`
`
`themes depending on
`
`the environment that is chosen.
`
`For example,
`
`1,000 x 1,000
`
`a beach
`
`
`environment and a selection of a
`
`
`ft virtual drive
`
` the movie-theater screen
`
`
`in movie-theater screen
`
`
`
`15
`
`over a virtual ocean may cause
`
`to be provided in a beach theme
`
`(e.g., virtualized in
`
`summer colors). Accordingly,
`
`for example, an augmented
`
`reality computing platform
`
`purchase a small number of
`
`
`
`would allow users to
`
`augmented reality computing
`
`20
`
`devices (@e.g., a single)
`
`head-mounted display to
`
`
`
`virtualize numerous physical objects instead of
`
`
`
`
`
` virtual hardware objects such
`
`
`
`
`manufacturers may provide
`
`the same virtual hardware
`
`(e.g.,
`
`[0015]
`
`different programs of
`
`Third-party
`
`
`
`
`
`
`
`different virtual television sets)
`
`that are each
`
`branded with
`
`the logo and appearances programmed by the
`
`third-parties and,
`
`accordingly,
`
`
` fF the augmented reality
`The manufacturer of
`
`may each charge their
`
`own cost.
`
`system may,
`
`in turn, allow users to purchase and
`
`Meta Exhibit 1007
`Meta v. Mullen - Page 0030
`
`purchasing the numerous physical objects individually.
`
`
`
`
`
`
`fferent
`Different costs may be associated with di
`
`that a user can purchase
`
`such virtual objects
`
`graphical user interface.
`
`
`from an augmented reality
`
`
`
`
`
`25
`
`30
`
`Meta Exhibit 1007
`Meta v. Mullen - Page 0030
`
`

`

`download the virtual hardware code from a third-party
`
`manufacturer for a set-fee,
`
`a commission, or any other
`
`pricing structure.
`
`The same virtual hardware objects
`
`
`
`may be provided with different functionality and such
`
`
`
`different virtual objects may be associated with
`
`
`
`different costs.
`
`For example, one virtual projector
`
`may be able to project in a particular resolution with
`
`
`colors up to a particular maximum
`
`a particular set of
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`size.
`
`A second virtual projector may be able to
`
`
`
`
`
`
`project at a different resolution, with a different set
`
`
`
` of colors, up to a different maximum size.
`
`[0016]
`
`
`Users of virtual hardware may,
`
`
`for example,
`
`share the virtual hardware in a physical environment
`
`with another user. Accordingly,
`
`a user may request
`
`10
`
`15
`
`20
`
`25
`
`permission to see another user’s augmented environment.
`
`The requested user may grant permission to the
`
`
`
`requesting user. This permission may be associated with
`
`
` different access levels and an access level may be
`
` selected by a user.
`
`For example, a permission access
`
`level may be to allow a requesting user to watch a
`
`user’s virtual
`television set, but not see the user’s
`
`virtual browser.
`
`
`The location and visual information
`
`
` to the requesting user’s head-mounted display.
`
`
`
`
`to display such virtual hardware (or other virtual
`
`objects) may be communicated wireless, or via a wire,
`
`A user
`
`may add,
`
`remove, or edit, augmented indicia access
`
`rights (e.g., viewing rights) at any time.
`
`
`
`
`Brief Description of the Drawings
`
`30
`
`[0017]
`
`The principles and advantages of the present
`
`
`
`
`
`invention can be more clearly understood from the
`
` following detailed description considered in
`
`
`conjunction with the following drawings,
`
`in which the
`
`Meta Exhibit 1007
`Meta v. Mullen - Page 0031
`
`Meta Exhibit 1007
`Meta v. Mullen - Page 0031
`
`

`

`same rel
`
`
`
`ference numerals denote the same structural
`
`elements throughout,
`
`[0018]
`
`FI G.
`
`and in which:
`
`
`1 is an ill ustral
`tion of!fF an augmented
`
`reality video game systen
`
`[0019]
`
`with
`
`
`the principles of
`
`FI G.
`
`2 is an
`
`reality video game systen
`
`
`
`the present invention;
`
`ill
`
`ustral
`tion of!
`fF an augmented
`
`n and associated video
`
`game
`
`nm COns! tructed in accordance
`
` the
`
`
`
`constructed in accordance with the principles of
`
`the
`
`present invention;
`
`10
`
`[0020]
`
`FI G.
`
`3 is an illust
`
`tration of
`
`fF an augmented
`
`reality video game system and associated video game
`
`played in an environment wit
`
`th a physical obstacle
`
`
`
`constructed in accordance with the principles of
`
`the
`
`present invention;
`
`15
`
`[0021]
`
`FI G.
`
`
`4 is an illustration of!
`
`fF an augmented
`
`reality video game system and associated video game and
`
`virtual indicia construct
`
`ted in accordance with
`
`
`principles of
`
`the present
`
`invention;
`
`
`
`[0022]
`
`FI G.
`
`5 is an illustration oj
`
`
`
`fF an augmented
`
`20
`
`reality video game system and associated video game
`
`constructed in accordance with the principles of
`
`the
`
`present invention;
`
`[0023]
`
`FI G.
`
`6 is an illustration oj
`
` fF an augmented
`
`reality video game system and associated video game
`
`25
`
`with virtual
`
`indicia constructed in accordance with the
`
`
`principles of
`
`[0024]
`
`FI G.
`
`the present invention;
`
`ill
`tion of!
`ustral
`fF an augmented
`
`7 is an
`
`reality
`
`video
`
`game systen
`
`nm COns!
`
`tructed in accordance
`
`with
`
`
`the principles of
`
`30
`
`FI G.
`
`the present invention;
`
`
`8 is an ill
`fF an augmented
`
`reality video game systen n and associated video
`ustral tion of!
`[0025]
`
`
`
`game
`
`being played on a physical environment constructed in
`
`
`
`Meta Exhibit 1007
`Meta v. Mullen - Page 0032
`
`Meta Exhibit 1007
`Meta v. Mullen - Page 0032
`
`

`

`
`accordance with the principles of
`
`the
`
`present
`
`invention;
`
`[0026]
`
`
`FIG.
`
`9 is an
`
`
`illustration of
`
`a video game
`
`system constructed in accordance with
`
`
`the principles of
`
`the present invention;
`
`[0027]
`
`
`
`
`
`hardware
`FIG. 10 is an illustration of virtual
`
`constructed in accordance with the principles of
`
`the
`
`present invention;
`
`[0028]
`
`
`
`
`
`FIG. 11 is an illustration of a virtual
`
`browser constructed in accordance with the principles
`
`
`of the present invention;
`
`
`
`
`FIG. 12 is an illustration of a virtual
`
`[0029]
`
`
`
`
`graphical user interface for virtual hardware selection
`
`constructed in accordance with the principles of the
`
`present invention;
`
`
`
`Cra
`
`
`13 is an ill
`tion of
`a virtual
`FIG.
`[0030]
`
`graphical user interface for virtual environment
`
`selection constructed in accordance with the principles
`
`
`
`us
`
`
`
`
`
`10
`
`15
`
`
`
`
` of
`
`the present invention;
`
`20
`
`[0031]
`
`
`FIG.
`
`14 is an illus
`
`constructed in accordance wi
`
`present invention;
`
`[0032]
`
`
`FIG. 15 is an illus
`
`
`Cra
`
`tion of
`
`controllers
`
`the
`
`
`the principles of
`
`
`
`
`
`Cra tion of a physical
`
`
`
`environment determination scheme constructed in
`
`25
`
`
`accordance with the principles of the present
`
`invention; and
`
`[0033]
`
`
`FIG.
`
`
`16 is an illustration of
`
`a head-mounted
`
`display constructed in accordance with
`
`the principles
`
` of the present invention.
`
`30
`
`Invention
`
`
`the
`Detailed Description of
`
`[0034]
`
`
`FIG.
`
`that may include users 101 and 102 utilizing table 103
`
`1 shows augmented reality setups
`
`Meta Exhibit 1007
`Meta v. Mullen - Page 0033
`
`Meta Exhibit 1007
`Meta v. Mullen - Page 0033
`
`

`

`
`
`
`
`face
`as a playing sur
`for an augmented reality video
`
`game system.
`
`The users may sel
`
`playing areas
`
` for an augmented
`
`users may select to pl
`
`
`lect one of
`
`
`a number of
`
`reality video game.
`
`For
`
` lay a game within the
`
`example,
`
`confines
`
`
`
`of area 120,
`
`130,
`
`140,
`
`and 110. Users may
`
`also change
`
`the size and shape
`
` of a three-dimensional
`
`area manually.
`
`An area may be
`
`shown to a user ina
`
`
`
`(e
`
`-g., green)
`
`such that a user can visually see
`
`the area will be.
`
`During game play,
`
`
`a border of
`
`
`
`(e.g., a user controlled object) attempts to leave the
`
`area,
`
`the game may stop the object from leaving the
`
`area. At any time,
`
`a game may be paused and a play
`
`15
`
`area may
`
`be modified.
`
`
`
`Similarly,
`
`for example, when a
`
`game is saved,
`
`area dimension
`
`
`information may be saved
`
`
`
`color
`
`where
`
`10
`
`
`
`a light color may be provided such
`
`
`the system are aware of
`the game area.
`
`that the users of
`
` If an object
`
`
`
`
`
`
`
`along with a game.
`
`When a user returns to play the
`
`saved game at a later time,
`
`the user may be
`
`
`firm the saved area
`automatically requested to cont
`
`20
`
`dimensions
`
`(and
`
`the user may be provided with a visual
`
`
` of the area to confirm).
`
`An object
`
`representation
`
`
`
`leaving
`
`
`
`the confines of
`
`be destroyed and a game
`
`For example,
`
`a
`
`25
`
`(e.g., plane,
`
`airplane)
`
`into
`
`user that drives a
`
`
`
`space shuttle,
`
`area may be perform an animation
`
`
`
`car,
`
`
`three-dimensional play
`the boundary of a
`
` for example,
`a game area may,
`
`
` FFected.
`may be according]
`ya
`
`
`
`virtual
`
`vehicle
`
`
`
`robot, boat,
`
`
`
`(e.g., explode) when a
`
`
` the art will
`
`boundary is hit
`
`t and points/charac
`
`reduced for a
`
`user.
`
`ter lives may be
`
`for
`
`A game may alternativel
`Yr
`
`30
`
`example, end.
`
`Persons skilled in
`
`
`
`
`
`appreciate tha
`
`
`t a game may be provided with a default
`
`boundary
`
`(e.g.,
`
`a dome-shaped boundary as in play area
`
`110).
`
`A user may be provided with the ability to scale
`
`Meta Exhibit 1007
`Meta v. Mullen - Page 0034
`
`Meta Exhibit 1007
`Meta v. Mullen - Page 0034
`
`

`

`the area. When a play area is scaled (e.g.,
`
`increased
`
`in size or decreased in size)
`
`
`the virtual objects of
`
`
` the game may be proportionally scaled by the video game
`
`
`
`
`(e.g.,
`
`increased in size or decreased in size).
`
`
`for example, a game may be
`Alternatively,
`
`
`provided in a user-selected play area of varying size
`
`[0035]
`
`
`with virtual objects of
`
`the same size.
`
`A user-
`
`controllable virtual object
`
`(@e.g., a virtual vehicle)
`
`may be provided at the center of a play area and user-
`
`
`
`
`
`
`
` the same location within a virtual environment.
`
`controls may cause the environment to move with respect
`
`to the virtual vehicle such that
`the vehicle stays in
`
`Such a
`
`
`
`
`
`10
`
`15
`
`20
`
`25
`
`play methodology may be user-selectable (e.g., an
`
`environment moving methodology or a vehicle moving
`
`methodology). Accordingly, a user may be provided with
`
`
`the flexibility to play the same game in different
`
`
`
`
`
`
`
`
`
`in a car on a user’s lap
`physical environments (e€.g.,
`
`versus in a room with a large floor). Similarly,
`for
`
`
`
`
`
`
`example, virtual environment indicia may be stationary
`
`and a user-controlled object may explore the virtual
`
`environment indicia until,
`for example,
`
`
`
`the user-
`
`
`
`controlled object reaches a boundary threshold (e.g., a
`
`boundary or a proximity to a boundary). At this point,
`
`for example,
`
`the environment may scroll with the user-
`
`
`controls such that if an user-controlled object reaches
`
`
` the threshold then the vehicle would stop but the
`
`
`
`
`
`environment would move such that the user perceives th
`
`vehicle to be moving at the same speed and in the same
`
`direction with respect to the virtual environment
`
`30
`
`indicia.
`
`[0036]
`
`A game console may process a game or a head-
`
`mounted display may process a game
`
`(and,
`
`in multiplayer
`
`
`modes may share information with another head-mounted
`
`Meta Exhibit 1007
`Meta v. Mullen - Page 0035
`
`Meta Exhibit 1007
`Meta v. Mullen - Page 0035
`
`

`

`display).
`
`A game area may be,
`
`
`for example,
`
`dome
`
`shaped, pyramid shaped,
`
`cubed shaped,
`
`cylindrically
`
`shaped,
`
`rectangular shaped,
`
`or spherically shaped.
`
`
`
` Edges may be rounded.
`
`Heights,
`
`width’s,
`
`and length’s
`
`
`
`A user may store his/her
`may be user adjusted.
`
`
`
`
`
`
`list of
`preferred
`(or a
`ay area parameters
`
`preferred pl
`
`parameter play areas)
`
`on a consol
`
`
`
`
`
`le or head-mounted
`
`ly utilize a user’s
`
`device and games may automaticall
`
`
`stored area preferenc
`(e.g.,
`
`
`for a location).
`
`10
`
`[0037]
`
`A head-mounted display may
`
`include,
`
`
`for
`
`example,
`
`a device,
`
`such as a camera,
`
` for use in
`
`
`determining the landscape of
`
`a use’s environment.
`
`Such
`
`a device may provide s