Image pickup apparatus, information processing system, mat, and image generation method

Patent metadata

Field	Value
Publication number	US-10477175-B2
Application number	US-201615547707-A
Country	US
Kind code	B2
Filing date	Jan 29, 2016
Priority date	Feb 6, 2015
Publication date	Nov 12, 2019
Grant date	Nov 12, 2019

Abstract

An image of a block or the like entering a space above a play mat is picked up by an image pickup apparatus, and an information processing apparatus detects and recognizes the block on the basis of a picture of the block in a picked up image and performs information processing. The play mat includes a play field that defines a detection space for the block, and a calibration chart including at least a plurality of regions of colors of different luminances. The image pickup apparatus adjusts an exposure time period, a gain value for each color component and a correction rule upon gamma correction on the basis of the picture of the calibration chart in the picked up image.

First claim

The invention claimed is:

1. An image pickup apparatus that outputs data of a picked up image to be used for recognition of a real object, comprising:
a computer having a microprocessor coupled to a storage medium containing a computer program, which when executed by the microprocessor, causes the computer to implement the following functional units:
an output image production unit configured to produce data of a picked up image to be outputted from data of a non-processed image picked up using an image pickup element;
an adjustment processing unit configured to evaluate the picked up image and adjust at least one of an image pickup condition in the image pickup element and a processing condition in the output image production unit; and
a data sending out unit configured to output the data of the picked up image to an apparatus that recognizes the real object, wherein
the adjustment processing unit adjusts the image pickup condition and the processing condition based on color information of a picture of a calibration chart that is provided in a space of a recognition target and is configured from a plurality of color regions having luminances different from each other.

2. The image pickup apparatus according to claim 1 , wherein the adjustment processing unit evaluates the luminance of a picture of the color regions of the calibration chart and adjusts the processing condition so as to expand the difference between a plurality of luminances corresponding to the luminance of the color of the real object.

3. The image pickup apparatus according to claim 2 , wherein the adjustment processing unit determines a luminance range within which the difference is to be expanded based on a luminance distribution of the picture of the real object in the picked up image.

4. The image pickup apparatus according to claim 1 , wherein the adjustment processing unit evaluates a color of a picture of a color region having a color same as a color of the real object in the calibration chart and adjusts the processing condition so as to expand a distance between a plurality of colors of different pictures in a color space.

5. The image pickup apparatus according to claim 2 , wherein the adjustment processing unit adjusts, as the adjustment of the processing condition, a correction rule upon gamma correction.

6. The image pickup apparatus according to claim 1 , wherein the adjustment processing unit adjusts a gain value of each color component such that a picture of a color region in the calibration chart, which has a color same as one of chromatic colors of the real object, has an achromatic color.

7. The image pickup apparatus according to claim 1 , wherein the adjustment processing unit evaluates a luminance of a picture of a color region in the calibration chart, which has a given luminance, and adjusts the image pickup condition and the processing condition in a direction in which the luminance is adjusted to a target luminance determined in advance.

8. The image pickup apparatus according to claim 1 , wherein the adjustment processing unit adjusts a white balance to a picture of a color region of an achromatic color in the calibration chart.

9. The image pickup apparatus according to claim 1 , wherein the data sending out unit parallelly outputs data of a picked up image produced in accordance with a processing condition adjusted by the adjustment processing unit and data of a picked up image produced in accordance with a processing condition that omits at least part of the adjustment by the adjustment processing unit.

10. The image pickup apparatus according to claim 1 , wherein the data sending out unit switches and outputs data of a picked up image produced in accordance with a processing condition adjusted by the adjustment processing unit and data of a picked up image produced in accordance with a processing condition that omits at least part of the adjustment by the adjustment processing unit in accordance with a request from an apparatus that recognizes the real object.

11. An information processing system including an image pickup apparatus that outputs data of a picked up image obtained by image pickup of a real object and an information processing apparatus that recognizes the real object using the picked up image, wherein
the image pickup apparatus includes a computer having a microprocessor coupled to a storage medium containing a computer program, which when executed by the microprocessor, causes the computer to implement the following functional units:
an output image production unit configured to produce data of a picked up image to be outputted from data of a non-processed image picked up using an image pickup element,
an adjustment processing unit configured to evaluate the picked up image and adjust at least one of an image pickup condition in the image pickup element and a processing condition in the output image production unit, and
a data sending out unit configured to output the data of the picked up image to the information processing apparatus, and
the adjustment processing unit adjusts the image pickup condition and the processing condition based on color information of a picture of a calibration chart that is provided in a space of a recognition target and is configured from a plurality of color regions having luminances different from each other, and
the information processing apparatus
recognizes a block by comparison between color information of the picture of the real object in the picked up image and color information of real objects registered already with each other and performs a process set for the block.

12. The image pickup apparatus according to claim 1 , further comprising a mat that is placed in an image pickup space, wherein:
the mat includes a play field that defines a bottom region of the space in which, when the real object is placed into the space, the real object is determined as the recognition target; and
the mat includes the calibration chart.

13. The image pickup apparatus according to claim 12 , wherein the calibration chart includes a color region having a luminance corresponding to a supposed color of the real object of the recognition target.

14. The image pickup apparatus according to claim 12 , wherein
the mat includes a first mat that includes the play field and a second mat that includes the calibration chart, and
the second mat is placed and removably mounted on the first mat.

15. An image production method, by an image pickup apparatus, of producing data of a picked up image to be used for recognition of a real object, comprising:
producing data of a picked up image to be outputted from data of a non-processed image picked up using an image pickup element;
evaluating the picked up image and adjusting at least one of an image pickup condition in the image pickup element and a processing condition in the producing; and
outputting the data of the picked up image to an apparatus that recognizes the real object, wherein
the adjusting adjusts the image pickup condition and the processing condition based on color information of a picture of a calibration chart that is provided in a space of a recognition target and is configured from a plurality of color regions having luminances different from each other.

16. A non-transitory, computer readable storage medium containing a computer program, which when executed by a computer, causes the computer to carry out actions to implement a function for producing data of a picked up image to be used for recognition of a real object, the computer program causing the computer to carry out the actions of:
produci

Assignees

• Sony Interactive Entertainment Inc

Inventors

• Ogasawara Hidehiko
• Nagatsuka Norio
• Numaguchi Naoki
• Segawa Hiroyuki
• Ohba Akio

Classifications

• A63F13/25 Primary

  Output arrangements for video game devices · CPC title

• G06V10/56

  relating to colour · CPC title

• H04N9/77 Primary

  Circuits for processing the brightness signal and the chrominance signal relative to each other, e.g. adjusting the phase of the brightness signal relative to the colour signal, correcting differential gain or differential phase (circuits for matrixing H04N9/67) · CPC title

• H04N23/70

  Circuitry for compensating brightness variation in the scene · CPC title

• H04N23/71

  Circuitry for evaluating the brightness variation · CPC title