Image processing device, control method of an image processing device, and storage medium that stores a program to execute a control method of an image processing device

Patent metadata

Field	Value
Publication number	US-9936121-B2
Application number	US-201514722757-A
Country	US
Kind code	B2
Filing date	May 27, 2015
Priority date	Jun 4, 2014
Publication date	Apr 3, 2018
Grant date	Apr 3, 2018

Abstract

A first image generating unit configured to generate a first image having a predetermined depth of field from each of a plurality of light-field data each of which a focus state is changeable; a searching unit configured to search the light-field data that includes a predetermined object by analyzing the first image generated by the first image generating unit; and a second image generating unit configured to generate a second image that has a shallower depth of field than the first image and focused on the predetermined object, based on the light-field data detected by the searching unit are included.

First claim

What is claimed is:

1. An image processing device comprising:
at least one memory that stores a program; and
at least one processor for executing the program stored in the at least one memory to perform:
a first image acquiring step of acquiring a plurality of first images having a predetermined depth of field corresponding to pan-focus from a plurality of light-field data, a focus state of each light-field data being changeable;
a searching step of searching for and detecting, from the plurality of light-field data, light-field data that includes a predetermined object by analyzing each of the plurality of first images acquired in the first image acquiring step; and
a second image acquiring step of acquiring a second image that has a shallower depth of field than the predetermined depth of field and that is focused on the predetermined object, based on the light-field data detected in the searching step.

2. The image processing device according to claim 1 , wherein the at least one processor executes the program stored in the at least one memory to further execute a distance information generating step of generating distance information that indicates a depth in relation to the light-field data, and
wherein, in the second image acquiring step, the distance information corresponding to the predetermined object, generated in the distance information generating step, is obtained, and the second image focusing on the predetermined object is acquired.

3. The image processing device according to claim 1 , wherein the light-field data includes information about light rays and direction information of light rays.

4. The image processing device according to claim 1 , wherein, when the light-field data including a plurality of predetermined objects is detected in the searching step, a depth of field is set, in the second image acquiring step, that allows for focusing on a plurality of detected objects.

5. The image processing device according to claim 1 , wherein, when focusing on the predetermined object detected in the searching step is not possible, a maximum depth of field is set in the second image acquiring step.

6. The image processing device according to claim 1 , further comprising a display screen configured to display the second image acquired in the second image acquiring step,
wherein, when focusing on the predetermined object detected in the searching step is not possible, a notification image is generated in the second image acquiring step to provide a notification that focusing on the predetermined object is not possible, and a process of displaying the notification image on the display screen is performed.

7. The image processing device according to claim 1 , further comprising an imaging unit configured to receive light from an object in an image through an imaging optical system,
wherein the imaging unit (1) has a plurality of microlenses and a plurality of photoelectric converters that convert light incident respectively from the imaging optical system via the plurality of microlenses into an electric signal, and (2) outputs information about light rays indicating a light intensity in relation to the image and direction information of light rays, and
wherein each of the plurality of light-field data includes the information about light rays and the direction information of light rays.

8. A control method of an image processing device, the method comprising:
a first image acquiring step of acquiring a plurality of first images having a predetermined depth of field corresponding to pan-focus from a plurality of light-field data, a focus state of each light-field data being changeable;
a searching step of searching for and detecting, from the plurality of light-field data, light-field data that includes a predetermined object by analyzing each of the plurality of first images acquired in the first image acquiring step; and
a second image acquiring step of acquiring a second image that has a shallower depth of field than the predetermined depth of field and that is focused on the predetermined object, based on the light-field data detected in the searching step.

9. The control method of the image processing device according to claim 8 , further comprising a distance information generating step of generating distance information indicating a depth in relation to the light-field data,
wherein, in the second image acquiring step, the distance information corresponding to the predetermined object, generated in the distance information generating step, is obtained, and the second image focusing on the predetermined object is acquired.

10. The control method of the image processing device according to claim 9 , wherein the light-field data includes information about light rays and direction information of light rays.

11. The control method of the image processing device according to claim 8 , wherein, when the light-field data including a plurality of predetermined object is detected in the searching step, a depth of field is set, in the second image acquiring step, that allows focusing on a plurality of detected objects.

12. The control method of the image processing device according to claim 8 , wherein, when focusing on the predetermined object detected in the searching step is not possible, a maximum depth of field is set in the second image acquiring step.

13. The control method of the image processing device according to claim 8 , further comprising a displaying step of displaying the second image acquired in the second image acquiring step,
wherein when focusing on the predetermined object detected in the searching step is not possible, a notification image is generated in the second image acquiring step to provide a notification that focusing on the predetermined object is not possible, and a process of displaying the notification image in the displaying step is performed.

14. The control method of the image processing device according to claim 8 , further comprising an imaging step of capturing an image using an imaging unit that (1) has a plurality of microlenses and a plurality of photoelectric converters that convert light incident respectively from an imaging optical system via the plurality of microlenses into an electric signal, and (2) outputs information about light rays indicating a light intensity in relation to the image and direction information of light rays,
wherein each of the plurality of light-field data includes the information about light rays and the direction information of light rays.

15. A non-transitory computer-readable storage medium that stores a program for causing a computer to execute each step of a control method of an image processing device, the method comprising:
a first image acquiring step of acquiring a plurality of first images having a predetermined depth of field corresponding to pan-focus from a plurality of light-field data, a focus state of each light-field data being changeable;
a searching step of searching for and detecting, from the plurality of light-field data, light-field data that includes a predetermined object by analyzing each of the plurality of first images acquired in the first image acquiring step; and
a second image acquiring step of acquiring a second image that has a shallower depth of field than the predetermined depth of field and that is focused on the predetermined object, based on the light-field data detected in the searching step.

Assignees

• Canon Kk

Inventors

• Kawakami Takashi

Classifications

• G06T5/50 Primary

  using two or more images, e.g. averaging or subtraction · CPC title

• G02B27/0075

  with means for altering, e.g. increasing, the depth of field or depth of focus · CPC title

• G06T7/571

  from focus · CPC title

• G06T2207/10052

  Images from lightfield camera · CPC title

• H04N5/23229

  Electricity · mapped topic