Information processor, information processing method, and computer program

Patent metadata

Field	Value
Publication number	US-9785854-B2
Application number	US-201514802800-A
Country	US
Kind code	B2
Filing date	Jul 17, 2015
Priority date	Aug 11, 2014
Publication date	Oct 10, 2017
Grant date	Oct 10, 2017

Abstract

An information processor includes: a similarity data generation portion generating, for a position within the search range, similarity data that represents the calculated similarity to the image in the reference block in association with the position within the search range; a similarity correction portion smoothing the similarity data in a direction of space on a basis of similarity data; a result evaluation portion detecting a position with a maximum similarity value in each piece of the smoothed similarity data; a depth image generation portion generating a depth image by associating the position of the subject in the depth direction with an image plane; and an output information generation section performing given information processing on a basis of the subject position in a three-dimensional space using the depth image and outputting the result of information processing.

First claim

What is claimed is:

1. An information processor comprising:
an image acquisition section enabled to receive image data from an imaging device equipped with stereo cameras for shooting the same space from left and right viewpoints;
a position information generation section adapted to generate subject position information on the basis of images of a subject shot by the stereo-camera imaging device, the position information generation section including
a similarity data generation portion adapted to treat, as a reference block, each of a plurality of reference-image areas obtained by dividing one of either left or right shot images shot with the imaging-device stereo cameras, to set up, for each of the reference blocks in the one of the either left or right shot images, a search-range image area in the other of the either left or right shot images, and to generate similarity data representing by-position similarity of the search-range image area to the reference-image area, calculated in association with position within the search range,
a similarity correction portion adapted to space-directionally smooth the similarity data on the basis of similarity data generated for a given number of reference blocks surrounding the corresponding reference block,
a result evaluation portion adapted to detect a position with a maximum similarity value in each piece of the smoothed similarity data, and
a depth image generation portion adapted to find a parallax for each of the reference blocks using the detection result obtained by the result evaluation portion, calculate a position of the subject on the basis of the parallax, and generate a depth image that represents the position of the subject in the depth direction as pixel values; and

an output information generation section adapted to perform given information processing on the basis of the subject position in a three-dimensional space using the depth image and output the information processing result.

2. The information processor of claim 1 , wherein
the result evaluation portion further refers to similarity data before smoothing for the corresponding reference block and evaluates similarity before smoothing at the position detected in similarity data after smoothing so as to screen the detection result, and
the depth image generation portion finds the parallax using a detection result validated as a result of screening.

3. The information processor of claim 2 , wherein the result evaluation portion invalidates the detection result if the similarity before smoothing is equal to a given threshold or less and detects again the position with the maximum similarity value in the smoothed similarity data.

4. The information processor of claim 1 , wherein
the shot images are movie frames shot at a given frame rate, and
the similarity correction portion further compares the similarity data generated by the similarity data generation portion for a reference block in the shot image of a first frame, with the similarity data generated by the similarity data generation portion for an identical-position reference block in the shot image of a second frame immediately previous to the first frame, and time-directionally smooths the similarity data on the basis of comparing the similarity data for the reference blocks in the first and second frames.

5. The information processor of claim 1 , wherein the result evaluation portion further finds, by using a given calculation formula, sharpness that represents an extent to which the maximum similarity value stands out in the similarity data, and invalidates the detection result if the sharpness is equal to a given threshold or less.

6. The information processor of claim 1 , wherein the result evaluation portion further invalidates the detection result if the maximum similarity value of the similarity data is equal to a given threshold or less and changes the threshold on a basis of a luminance distribution of the corresponding reference block.

7. The information processor of claim 1 , wherein
the similarity data generation portion generates similarity data not only for reference blocks obtained by dividing one of the left and right shot images but also for reference blocks obtained by dividing the other image, and
the depth image generation portion generates left and right parallax images by finding the parallax in association with each of the reference blocks for each of the images in which the reference blocks have been set up and screens parallax data used to generate a depth image on a basis of the result of comparison of the parallax values at the corresponding positions of the left and right parallax images.

8. An information processing method comprising:
receiving image data from an imaging device equipped with stereo cameras for shooting the same space from left and right viewpoints;
treating, as a reference block, each of a plurality of reference-image areas obtained by dividing one of either left or right shot images shot with the imaging-device stereo cameras, setting up, for each of the reference blocks in the one of the either left or right shot images, a search-range image area in the other of the either left or right shot images, generating similarity data representing by-position similarity of the search-range image area to the reference-image area, calculated in association with position within the search range, and storing the similarity data in a memory;
reading each piece of the similarity data from the memory and space-directionally smoothing the similarity data on the basis of similarity data generated for a given number of reference blocks surrounding the corresponding reference block;
detecting a position with a maximum similarity value in each piece of the smoothed similarity data;
finding a parallax for each of the reference blocks using the detection result obtained by the detecting, calculating a position of a subject on the basis of the parallax, and generating a depth image that represents the position of the subject in the depth direction as pixel values; and
performing given information processing on the basis of the subject position in a three-dimensional space using the depth image and outputting the information processing result to an external device.

9. A non-transitory computer-readable recording medium having embodied thereon a computer program causing a central processing unit to execute steps of:
receiving image data from an imaging device equipped with stereo cameras for shooting the same space from left and right viewpoints;
treating, as a reference block, each of a plurality of reference-image areas obtained by dividing one of either left or right shot images shot with the imaging-device stereo cameras, setting up, for each of the reference blocks in the one of the either left or right shot images, a search-range image area in the other of the either left or right shot images, generating similarity data representing by-position similarity of the search-range image area to the reference-image area, calculated in association with position within the search range;
space-directionally smoothing the similarity data on the basis of similarity data generated for a given number of reference blocks surrounding the corresponding reference block;
detecting a position with a maximum similarity value in each piece of the smoothed similarity data;
finding a parallax for each of the reference blocks using the detection result obtained by the detecting, calculating a position of a subject on the basis of the parallax, and generating a depth image that represents the position of the subject in the depth direction as pixel values; and
performing given information processing on the basis of the subject position in a three-dim

Assignees

• Sony Corp
• Sony Computer Entertainment Inc
• Sony Interactive Entertainment Inc

Inventors

• Segawa Hiroyuki
• Ohba Akio
• Inada Tetsugo
• Okamoto Hirofumi
• Kimura Atsushi
• Kohashi Takashi
• Itou Takashi

Classifications

• G06F3/017 Primary

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title

• G06F18/22

  Matching criteria, e.g. proximity measures · CPC title

• G06V10/34

  Smoothing or thinning of the pattern; Morphological operations; Skeletonisation · CPC title

• G06T7/593

  from stereo images · CPC title

• G06K9/00771

  Physics · mapped topic