User-driven three-dimensional interactive gaming environment

Patent metadata

Field	Value
Publication number	US-11010971-B2
Application number	US-201113282369-A
Country	US
Kind code	B2
Filing date	Oct 26, 2011
Priority date	May 29, 2003
Publication date	May 18, 2021
Grant date	May 18, 2021

Abstract

An invention is provided for affording a real-time three-dimensional interactive environment using a depth sensing device. The invention includes obtaining depth values indicating distances from one or more physical objects in a physical scene to a depth sensing device. The depth sensing device is configurable to be maintained at a particular depth range defined by a plane so that objects between the particular depth range and the depth sensing device are processed by the depth sensing device, wherein the particular depth range establishes active detection by the depth sensing device, as depth values of objects placed through the particular depth range and toward the depth sensing device are detected and depth values of objects placed beyond the particular depth range are not detected. The objects placed through the particular depth range are rendered and displayed in a virtual scene based on geometric characteristics of the object itself.

First claim

What is claimed is:

1. A computer implemented method having access to memory, the method providing a real-time three-dimensional interactive environment, comprising the operations of:
obtaining depth values indicating distances from one or more physical objects in a physical scene to a depth sensing device, the depth sensing device being maintained at a particular depth range defined by a plane that is substantially perpendicular to a view direction of the depth sensing device, so that objects within a field of view of the depth sensing device and between the particular depth range and the depth sensing device are processed by the depth sensing device, and objects within the field of view and beyond the particular depth range are not visible to, nor processed by, the depth sensing device, wherein depth values of surfaces of objects within the field of view and placed through the particular depth range toward, and unobstructed to, the depth sensing device are provided by the depth sensing device, and depth values of surfaces of objects within the field of view placed beyond the particular depth range that are unobstructed to the depth sensing device are not provided by the depth sensing device, wherein the objects placed through the particular depth range are rendered and displayed in a virtual scene based on geometric characteristics of the object itself;
wherein the depth sensing device captures depth values for pixels of the scene corresponding to surfaces placed through the particular depth range and towards the depth sensing device, and wherein the depth sensing device does not capture depth values for pixels of the scene corresponding to surfaces that are beyond the particular depth range;
initiating tracking of the objects when the objects are placed through the particular depth range and toward the depth sensing device, and terminating tracking of the objects when the objects are placed beyond the particular depth range.

2. A method as recited in claim 1 , further comprising,
detecting an interaction between only objects placed through the particular depth range and the virtual object based on coordinates of the virtual object and the obtained depth values of the objects placed through the particular depth range.

3. A method as recited in claim 1 , wherein the depth sensing device is a depth camera using controlled infrared lighting.

4. A method as recited in claim 1 , further comprising,
estimating three-dimensional volume information for each physical object within the particular depth range based on the obtained depth values.

5. A method as recited in claim 1 ,
wherein the objects include a hand of a user;
wherein tracking of the hand of the user is initiated when placed through the particular depth range and toward the depth sensing device, and wherein tracking of the hand of the user is terminated when placed beyond the particular depth range;
wherein the virtual scene is rendered to a display device.

6. A method as recited in claim 1 , further comprising:
inserting at least one virtual object into the virtual scene after obtaining the depth values, the virtual object being computer-generated and configured to be inserted within and beyond the particular depth range;

using a computer generated light source to project one or more virtual shadows in the virtual scene based on the depth values;
wherein the one or more virtual shadows are for one or more of the at least one virtual object or the objects placed through the particular depth range that are rendered and displayed in the virtual scene.

7. A method as recited in claim 6 ,
wherein the one or more virtual shadows includes a shadow of the at least one virtual object cast on one or more of the objects placed through the particular depth range that are rendered and displayed in the virtual scene.

8. A computer program embodied on a non-transitory computer readable medium for providing a real-time three-dimensional interactive environment, comprising:
program instructions that obtain depth values indicating distances from one or more physical objects in a physical scene to a depth sensing device, the depth sensing device being maintained at a particular depth range defined by a plane that is substantially perpendicular to a view direction of the depth sensing device, so that objects within a field of view of the depth sensing device and between the particular depth range and the depth sensing device are processed by the depth sensing device, and objects within the field of view and beyond the particular depth range are not visible to, nor processed by the depth sensing device, wherein depth values of surfaces of objects placed through the particular depth range toward, and unobstructed to, the depth sensing device are provided by the depth sensing device, and depth values of surfaces of objects within the field of view placed beyond the particular depth range that are unobstructed to the depth sensing device are not provided by the depth sensing device, wherein the objects placed through the particular depth range are rendered and displayed in a virtual scene based on geometric characteristics of the object itself;
wherein the depth sensing device captures depth values for pixels of the scene corresponding to surfaces placed through the particular depth range and towards the depth sensing device, and wherein the depth sensing device does not capture depth values for pixels of the scene corresponding to surfaces that are beyond the particular depth range;
program instructions that initiate tracking of the objects when the objects are placed through the particular depth range and toward the depth sensing device, and terminate tracking of the objects when the objects are placed beyond the particular depth range.

9. A computer program as recited in claim 8 , further comprising,
program instructions that detect an interaction between only objects placed through the particular depth range and the virtual object based on coordinates of the virtual object and the obtained depth values of the objects placed through the particular depth range.

10. A computer program as recited in claim 8 , wherein the depth sensing device is a depth camera using controlled infrared lighting.

11. A computer program as recited in claim 8 , further comprising,
program instructions that estimate three-dimensional volume information for each physical object within the particular depth range based on the obtained depth values.

12. A computer program as recited in claim 8 ,
wherein the objects include a hand of a user;
wherein tracking of the hand of the user is initiated when placed through the particular depth range and toward the depth sensing device, and wherein tracking of the hand of the user is terminated when placed beyond the particular depth range;
wherein the virtual scene is rendered to a display device.

13. A system for providing a real-time three-dimensional interactive environment, comprising:
a depth sensing device capable of obtaining depth values indicating distances from one or more physical objects in a physical scene to a depth sensing device, the depth sensing device being maintained at a particular depth range defined by a plane that is substantially perpendicular to a view direction of the depth sensing device, so that objects within a field of view of the depth sensing device and between the particular depth range and the depth sensing device are processed by the depth sensing device, and objects within the field of view and beyond the particular depth range are not visible to, nor processed by the depth sensing device, wherein depth values of surfaces of objects within the fiel

Assignees

• Marks Richard
• Sony Interactive Entertainment Inc

Inventors

• Marks Richard

Classifications

• A63F13/65

  automatically by game devices or servers from real world data, e.g. measurement in live racing competition · CPC title

• A63F2300/1087

  comprising photodetecting means, e.g. a camera · CPC title

• A63F13/213

  comprising photodetecting means, e.g. cameras, photodiodes or infrared cells (A63F13/219 takes precedence) · CPC title

• A63F2300/69

  Involving elements of the real world in the game world, e.g. measurement in live races, real video · CPC title

• G06T19/006 Primary

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title