Image capturing device and program to control image capturing device

Patent metadata

Field	Value
Publication number	US-9888226-B2
Application number	US-201314072340-A
Country	US
Kind code	B2
Filing date	Nov 5, 2013
Priority date	May 6, 2011
Publication date	Feb 6, 2018
Grant date	Feb 6, 2018

Abstract

With a conventional stereoimage capturing device, it has been difficult to simply determine the present of a depth. An image capturing device includes an image capturing element in which a first parallax pixel group and a second parallax pixel group that output a first parallax image and a second parallax image, respectively, to cause a parallax are arranged at mutually different positions. The image capturing device computes a differential pixel value of each of a plurality of second parallax output pixels in the second parallax image positioned around a target pixel that is a first parallax output pixel of the first parallax image, the differential pixel value being computed with reference to the target pixel, and generates differential data in which a direction of each of the plurality of second parallax output pixels with reference to the target pixel is associated with the corresponding differential pixel value.

First claim

What is claimed is:

1. An image capturing device comprising:
an image capturing element in which a first parallax pixel group and a second parallax pixel group that output a first parallax image and a second parallax image, respectively, to cause a parallax are arranged at mutually different positions;
an image processor configured to
compute a differential pixel value of each of a plurality of second parallax output pixels in the second parallax image positioned around a target pixel that is a first parallax output pixel of the first parallax image, the differential pixel value being computed with reference to the target pixel,
generate a differential data set including a plurality of pieces of differential data in which a direction of each of the plurality of second parallax output pixels is associated with the corresponding differential pixel value, and
arrange, for each of the differential data, the differential pixel value in an arrangement order based on directions of the plurality of second parallax output pixels with reference to the target pixel; and

a memory that stores the differential data set in a file format.

2. The image capturing device according to claim 1 , wherein the image processor is further configured to
set each of a plurality of first parallax output pixels in the first parallax image as the target pixel,
compute the differential pixel value with reference to each of the plurality of first parallax output pixels, and
generate the differential data for each of the plurality of first parallax output pixels, and
generate the differential data set including the generated differential data.

3. The image capturing device according to claim 2 , wherein the image processor is further configured to
associate positional information of the target pixel with the differential data for the target pixel to generate the differential data set.

4. The image capturing device according to claim 2 , wherein the image processor is further configured to
arrange the differential data in a predetermined arrangement order of the differential data to generate the differential data set.

5. The image capturing device according to claim 2 , wherein the image processor is further configured to
associate the differential data corresponding to a reference pixel in the first parallax image with a pixel value of the reference pixel to generate the differential data set.

6. The image capturing device according to claim 1 , wherein the image processor is further configured to
set, as a target area, an m×n pixel area (m and n are integers equal to or larger than three) centered around the target pixel and including the plurality of second parallax output pixels of the second parallax image; and
compute the differential pixel value of each of the plurality of second parallax output pixels included in the target area, the differential pixel value being computed with reference to the target pixel.

7. The image capturing device according to claim 6 , wherein the image processor is further configured to
set the target area according to array patterns of the first parallax pixel group and the second parallax pixel group in the image capturing element.

8. The image capturing device according to claim 6 , wherein the image processor is further configured to
set the target area including the plurality of second parallax output pixels of a color same as the color of the target pixel.

9. The image capturing device according to claim 1 , wherein the image processor is further configured to
compute an identical parallax differential value that is a differential pixel value for a single first parallax output pixel or each of a plurality of first parallax output pixels of the first parallax image positioned around the target pixel, the identical parallax differential value being computed with reference to the target pixel, and
generate identical parallax differential data in which a direction of the single first parallax output pixel or each of the plurality of first parallax output pixels with reference to the target pixel is associated with the identical parallax differential value.

10. The image capturing device according to claim 9 , wherein the image processor is further configured to
generate an identical parallax differential data set including a plurality of pieces of the identical parallax differential data.

11. The image capturing device according to claim 10 , wherein the image processor is further configured to
set each of the plurality of first parallax output pixels in the first parallax image as the target pixel, and computes the identical parallax differential value with reference to each of the plurality of first parallax output pixels, and
generate the identical parallax differential data for each of the plurality of first parallax output pixels, and
generate the identical parallax differential data set including the generated identical parallax differential data.

12. The image capturing device according to claim 10 , wherein the image processor is further configured to
store the differential data set and the identical parallax differential data set in the memory in the file format.

13. The image capturing device according to claim 9 , wherein the image processor is further configured to
set, as a target area, an m×n pixel area (m and n are integers equal to or larger than three) centered around the target pixel and including the plurality of second parallax output pixels of the second parallax image;
set, as an identical parallax target area, an i×j pixel area (i and j are integers equal to or larger than three) centered around the target pixel and including a single first parallax output pixel or a plurality of first parallax output pixels of the second parallax image;
compute the differential pixel value of each of the plurality of second parallax output pixels included in the target area, the differential pixel value being computed with reference to the target pixel; and
compute the identical parallax differential value of the single first parallax output pixel or the plurality of first parallax output pixels included in the identical parallax target area, the identical parallax differential value being computed with reference to the target pixel.

14. The image capturing device according to claim 1 , wherein
the image capturing element has:
photoelectric converting elements that convert an incident light into an electrical signal and are arrayed two-dimensionally; and
aperture masks each provided corresponding to each of the photoelectric converting elements,
an aperture of each of the aperture masks provided corresponding to at least two of the photoelectric converting elements among adjacent n photoelectric converting elements (n is an integer equal to or larger than three) of the photoelectric converting elements is positioned to transmit a luminous flux from a mutually different partial area in a cross-sectional area of the incident light,
groups of the photoelectric converting elements, each group consisting of a set of the n photoelectric converting elements, are arrayed periodically and continuously, and
two of the photoelectric converting elements among the at least two photoelectric converting elements output a first parallax output pixel of the first parallax image and a second parallax output pixel of the second parallax image.

15. A non-transitory computer-readable medium having stored thereon a program to control an image capturing device having an image capturing element in which a first parallax pixel group and a second parallax pixel group that output a fi

Assignees

• Nippon Kogaku Kk

Inventors

• Shibazaki Kiyoshige
• Mori Susumu
• Hamashima Muneki

Classifications

• H04N13/0285

  Electricity · mapped topic

• H04N13/0257

  Electricity · mapped topic

• G03B35/10

  having single camera with stereoscopic-base-defining system · CPC title

• H04N13/0207

  Electricity · mapped topic

• H04N13/0225 Primary

  Electricity · mapped topic