Information processing device and information processing method

Patent metadata

Field	Value
Publication number	US-10279249-B2
Application number	US-201515516182-A
Country	US
Kind code	B2
Filing date	Aug 27, 2015
Priority date	Oct 8, 2014
Publication date	May 7, 2019
Grant date	May 7, 2019

Abstract

On a play field 20 set in real space, a target region 102 having the play field 20 as a bottom surface and having a predetermined height h is set virtually. A target region image that expresses a distance of the target region 102 from an imaging surface as a pixel value on an image plane is generated, and compared with a depth image corresponding to an actual captured image 100 ; thus, only images of blocks 3 c and 3 b inside the target region 102 are extracted as detection and tracking targets.

First claim

The invention claimed is:

1. An information processing device comprising:
a non-transitory computer-readable storage medium configured to store content data including model data of a virtual object; and
at least one processor configured to
acquire a captured image of a real space from an imaging device,
specify a planar surface of a first real object in the captured image,
virtually set a three-dimensional target region on the planar surface of the first real object,
determine whether a second real object is present in the three-dimensional target region,
acquire a first audio signal uttered by a user from a microphone,
recognize a first word included in the first audio signal,
record the first word as label data of the second real object in the storage medium, when it is determined that the second real object is present in the three-dimensional target region,
determine whether the label data of the second real object corresponds to the model data of the virtual object, and
control a display device to display the virtual object at a depth position of the second real object with respect to the imaging device, when it is determined that the second real object is present in the three-dimensional target region and that the label data of the second real object corresponds to the model data of the virtual object.

2. The information processing device according to claim 1 , wherein the three-dimensional target region is set to have a predetermined height corresponding to the first real object.

3. The information processing device according to claim 2 , wherein the first real object comprises a play mat that is placed in the real space and has a predetermined size.

4. The information processing device according to claim 1 , wherein the at least one processor is further configured to
generate a depth image based on the captured image, and
control the display device to display a boundary of the three-dimensional target region to correspond to the depth image.

5. The information processing device according to claim 1 ,
wherein the content data includes first sound data for the virtual object, and
wherein the at least one processor is further configured to
acquire a second audio signal uttered by the user from the microphone,
recognize a second word included in the second audio signal, the second word being different from the first word,
determine whether the second word corresponds to the first sound data, and
control a speaker to output a sound corresponding to the first sound data, when it is determined that the second real object is present in the three-dimensional target region and that the second word corresponds to the first sound data.

6. The information processing device according to claim 1 ,
wherein the content data includes motion data representing a predetermined movement and second sound data corresponding to the predetermined movement, and
wherein the at least one processor is further configured to
determine, based on the captured image, whether a movement of the second real object in the three-dimensional target region corresponds to the predetermined movement, and
control a speaker to output a sound corresponding to the second sound data, when it is determined that the second real object is present in the three-dimensional target region and that the movement of the second real object in the target region corresponds to the predetermined movement.

7. The information processing device according to claim 1 ,
wherein the content data includes a predetermined condition and third sound data corresponding to the predetermined condition, and
wherein the at least one processor is further configured to
determine a position of the second real object in the three-dimensional target region, and
control a speaker to output a sound corresponding to the third sound data, when it is determined that a positional relation between the second real object and at least one other real object in the three-dimensional target region satisfies the predetermined condition.

8. The information processing device according to claim 1 ,
wherein the at least one processor is further configured to
generate a depth image based on the captured image, and
represent the depth image as a pixel value on a captured image plane.

9. An information processing method executed by an information processing device, the method comprising:
acquiring a captured image of a real space from an imaging device;
specifying a planar surface of a first real object in the captured image;
virtually setting a three-dimensional target region on the planar surface of the first real object;
determining whether a second real object is present in the three-dimensional target region;
acquiring a first audio signal uttered by a user from a microphone;
recognizing a first word included in the first audio signal;
recording the first word as label data of the second real object in the storage medium, when it is determined that the second real object is present in the three-dimensional target region;
determining whether the label data of the second real object corresponds to the model data of the virtual object; and
controlling a display device to display the virtual object at a depth position of the second real object with respect to the imaging device, when it is determined that the second real object is present in the three-dimensional target region and that the label data of the second real object corresponds to the model data of the virtual object.

10. A non-transitory computer-readable recording medium having embodied thereon a program, which when executed by a computer causes the computer to execute a method, the method comprising:
acquiring a captured image of a real space from an imaging device;
specifying a planar surface of a first real object in the captured image;
virtually setting a three-dimensional target region on the planar surface of the first real object;
determining whether a second real object is present in the three-dimensional target region;
acquiring a first audio signal uttered by a user from a microphone;
recognizing a first word included in the first audio signal;
recording the first word as label data of the second real object in the storage medium, when it is determined that the second real object is present in the three-dimensional target region;
determining whether the label data of the second real object corresponds to the model data of the virtual object; and
controlling a display device to display the virtual object at a depth position of the second real object with respect to the imaging device, when it is determined that the second real object is present in the three-dimensional target region and that the label data of the second real object corresponds to the model data of the virtual object.

Assignees

• Sony Corp
• Sony Interactive Entertainment Inc

Inventors

• Numaguchi Naoki
• Andre Alexis

Classifications

• G06V10/751

  Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching · CPC title

• G06V10/56

  relating to colour · CPC title

• G06V10/40

  Extraction of image or video features · CPC title

• G06K9/46

  Physics · mapped topic

• G06T7/215

  Motion-based segmentation · CPC title