Method and apparatus for tracking of a subject in a video game

Patent metadata

Field	Value
Publication number	US-9694283-B2
Application number	US-201313763720-A
Country	US
Kind code	B2
Filing date	Feb 10, 2013
Priority date	Feb 8, 2007
Publication date	Jul 4, 2017
Grant date	Jul 4, 2017

Abstract

Method, computer program and system for tracking movement of a subject. The method includes receiving data from a distributed network of camera sensors employing one or more emitted light sources associated with one or more of the one or more camera sensors to generate a volumetric three-dimensional representation of the subject, identifying a plurality of clusters within the volumetric three-dimensional representation that correspond to motion features indicative of movement of the motion features of the subject, presenting one or more objects on one or more three dimensional display screens, and using the plurality of fixed position sensors to track motion of the motion features of the subject and track manipulation of the motion features of the volumetric three-dimensional representation to determine interaction of one or more of the motion features of the subject and one or more of the one or more objects on the three dimensional display.

First claim

What is claimed:

1. A system for tracking movement of motion features of a subject operating a video game, comprising:
a plurality of fixed position sensors comprising a distributed network of camera sensors including one or more emitted light sources associated with one or more of the one or more camera sensors, the plurality of fixed position sensors capturing data and generating a volumetric three-dimensional representation of the subject;
a processor for identifying one or more motion features of the subject within the three-dimensional representation indicative of motion of one or more motion features of the subject relative to the fixed position sensors and one or more other portions of the subject, and
one or more three-dimensional displays allowing for the perception of a two-dimensional object in a three-dimensional space, and for presenting the generated volumetric three-dimensional representation of the subject;
the processor further using the plurality of fixed position sensors to track motion of the one or more motion features of the subject to determine interaction of one or more of the one or more motion features of the subject and one or more of the one or more objects on the one or more three-dimensional displays.

2. The system of claim 1 , wherein the processor further identifies one or more recognized events within the video game corresponding to the tracked motion of the subject.

3. The system of claim 2 , wherein the identifying one or more events by the processor further comprises matching the motion of the subject with an event stored in an event library associated with the video game.

4. The system of claim 3 , further comprising generating new events for the library associated with the video game taking into account anatomical constraints of the subject, sensor resolution, pixel depth and a total number of sensors.

5. The system of claim 1 , wherein an image of the face of the subject playing the video game is acquired by one or more of the plurality of fixed position sensors; and
a non-photo-real avatar resembling the acquired image of the user into a character in the video game is integrated by the processor and displayed on the display.

6. The system of claim 5 , wherein the non-photo-real avatar is warped to best match a non-photo-real color space associated with the video game.

7. The system of claim 5 , wherein the non-photo-real avatar changes according to one or more game conditions.

8. The system of claim 7 , wherein the one or more game conditions comprises a change in skill level of a character controlled by the subject.

9. The system of claim 7 , wherein the changing of the non-photo-real avatar represents a nonlinear transformation.

10. The system of claim 9 , wherein the nonlinear transformation comprises eigen faces fusion between the avatar and the subject.

11. The system of claim 9 , wherein the nonlinear transformation comprises landmarking between the avatar and the subject.

12. The system of claim 1 , wherein the one or more three-dimensional displays comprise autostereoscopic displays.

13. The system of claim 1 , wherein the one or more three-dimensional displays employ a three-dimensional viewing device.

14. The system of claim 13 , wherein the three-dimensional viewing device comprises three-dimensional viewing glasses.

15. The system of claim 1 , wherein the fixed position sensors are time-of-flight sensors.

16. A system for tracking motion of a subject operating a video game, comprising:
a distributed network of camera sensors employing one or more emitted light sources associated with one or more of the one or more camera sensors, at least data received from the camera sensors being used to generate a volumetric three-dimensional representation of a subject;
a processor for identifying one or more motion features of the subject within the three-dimensional representation indicative of motion of the one or more motion features of the subject relative to the fixed position sensors and, one or more other portions of the subject; and
one or more three-dimensional displays allowing for the perception of a two-dimensional object in a three-dimensional space, and for representing the three-dimensional representation of the subject and one or more objects thereon;
wherein the plurality of fixed position sensors is used to track motion of the one or more motion features of the subject to determine interaction of the one or more motion features of the subject and one or more of the one or more objects on one or more of the one or more three-dimensional displays within the video game.

17. The system of claim 16 , wherein the plurality of fixed position sensors comprises a distributed grid of sensors.

18. The system of claim 17 , wherein the distributed grid of sensors further comprises a plurality of light sources.

19. The system of claim 16 , wherein a representation of the volumetric three-dimensional representation for the subject is displayed on one or more of the three-dimensional displays, wherein the volumetric three-dimensional representation moves according to instructions generated by a software development kit that links the motion features to a skeletal structure.

20. The system of claim 19 , wherein the one or more displays display interaction between the representation of the volumetric three-dimensional representation and the one or more of the one or more objects within the video game.

21. The system of claim 16 , wherein the processor further identifies one or more recognized events within the video game corresponding to the tracked motion of the subject.

22. The system of claim 21 , wherein the identifying one or more events by the processor further comprises matching the motion of the subject with an event stored in an event library associated with the video game.

23. The system of claim 22 , wherein the processor generates new events for the library associated with the video game taking into account anatomical constraints of the subject, sensor resolution, pixel depth and a total number of sensors.

24. The system of claim 16 , wherein the processor acquires an image of the face of the subject playing the video game in accordance with at least data received from the camera sensor, and integrates a non-photo-real avatar resembling the acquired image of the subject into a character in the video game.

25. The system of claim 24 , wherein the non-photo-real avatar is warped by the processor to best match a non-photo-real color space associated with the video game.

26. The system of claim 24 , wherein the processor changes the non-photo-real avatar according to one or more game conditions.

27. The system of claim 26 , wherein the one or more game conditions comprises a change in skill level of a character controlled by the subject.

28. The system of claim 26 , wherein the changing of the non-photo-real avatar represents a nonlinear transformation.

29. The system of claim 28 , wherein the nonlinear transformation comprises eigen faces fusion between the avatar and the corresponding one of the first and second subjects.

30. The system of claim 28 , wherein the nonlinear transformation comprises landmarking between the avatar and the corresponding one of the first and second subjects.

31. The system of claim 16 , wherein the one or more three-dime

Assignees

• Microsoft Technology Licensing Llc

Inventors

• El Dokor Tarek
• King Joshua E
• Holmes James E
• Gigliotto Justin R
• Glomski William E

Classifications

• A63F2300/6607

  for animating game characters, e.g. skeleton kinematics · CPC title

• A63F13/56

  Computing the motion of game characters with respect to other game characters, game objects or elements of the game scene, e.g. for simulating the behaviour of a group of virtual soldiers or for path finding · CPC title

• A63F2300/208

  for storing personal settings or data of the player · CPC title

• A63F2300/1093

  using visible light · CPC title

• A63F2300/1087

  comprising photodetecting means, e.g. a camera · CPC title