Image processing apparatus, imaging device, and image processing method

Patent metadata

Field	Value
Publication number	US-2017195540-A1
Application number	US-201715467699-A
Country	US
Kind code	A1
Filing date	Mar 23, 2017
Priority date	Sep 29, 2014
Publication date	Jul 6, 2017
Grant date	—

Abstract

An image processing apparatus is intended to find, even when shooting a subject having a depth, respective reflection characteristics of portions of the subject with high accuracy, and includes a high-luminance region extraction unit which extracts high-luminance regions having a higher luminance than a predetermined threshold value in a plurality of image signals obtained by irradiating the subject with light emitted from a known light source and shooting reflected light from the subject at a plurality of viewpoints, a local region extraction unit which extracts local regions respectively most focused on in the portions of the subject using the plurality of high-luminance regions, and a reflection characteristic estimation unit which estimates a reflection characteristic of the subject using respective light source characteristics of the high-luminance regions in the local regions.

First claim

1 . An image processing apparatus comprising:
a high-luminance region extraction unit which extracts high-luminance regions serving as regions each having a higher luminance than a predetermined threshold value in a plurality of image signals obtained by irradiating a subject with light emitted from a known light source and by shooting reflected light from the subject at a plurality of viewpoints different from one another; a local region extraction unit which extracts local regions each of which is in the most in-focus state in its portion of the subject using the plurality of high-luminance regions extracted in the high-luminance region extraction unit; and a reflection characteristic estimation unit which estimates a reflection characteristic of the subject using respective light source characteristics of the high-luminance regions in the local regions extracted in the local region extraction unit.

2 . The image processing apparatus according to claim 1 , wherein the reflection characteristic estimation unit uses an edge gradient of the light source as the light source characteristic of the high-luminance region.

3 . The image processing apparatus according to claim 1 , wherein the reflection characteristic estimation unit estimates a point spread function of specular reflection as the reflection characteristic.

4 . The image processing apparatus according to claim 1 , further comprising a reproduced image generation unit which generates a reproduced image of the subject under a virtual light source based on the reflection characteristic estimated by the reflection characteristic estimation unit and an image of the virtual light source.

5 . The image processing apparatus according to claim 4 , wherein the reproduced image generation unit generates the reproduced image based on a distance from a focal point at each of points of the subject.

6 . An imaging device comprising:
an image signal acquisition unit which acquires a plurality of image signals obtained by irradiating a subject with light emitted from a known light source and by shooting reflected light from the subject at a plurality of viewpoints different from one another; and the image processing apparatus according to claim 1 which estimates a reflection characteristic of the subject from the plurality of image signals acquired by the image signal acquisition unit.

7 . The imaging device according to claim 6 , wherein the image signal acquisition unit acquires the image signals which are based on images obtained by combining a plurality of optical systems which differ in principal ray.

8 . The imaging device according to claim 7 , further comprising an imaging optical system which collects light from the subject,
wherein the plurality of optical systems divide an exit pupil of the imaging optical system.

9 . The imaging device according to claim 7 , wherein the plurality of optical systems respectively have optical axes different from one another.

10 . The imaging device according to claim 8 , wherein the image signal acquisition unit acquires the plurality of image signals by capturing images a plurality of times while changing a focal length of the imaging optical system.

11 . An image processing method comprising:
a high-luminance region extraction step of extracting high-luminance regions serving as regions each having a higher luminance than a predetermined threshold value in a plurality of image signals obtained by irradiating a subject with light emitted from a known light source and by shooting reflected light from the subject at a plurality of viewpoints different from one another; a local region extraction step of extracting local regions each of which is in the most in-focus state in its portion of the subject using the plurality of high-luminance regions extracted in the high-luminance region extraction step; and a reflection characteristic estimation step of estimating a reflection characteristic of the subject using respective light source characteristics of the high-luminance regions in the local regions extracted in the local region extraction step.

12 . The image processing method according to claim 11 , further comprising a reproduced image generation step of generating a reproduced image of the subject under a virtual light source based on the reflection characteristic estimated in the reflection characteristic estimation step and an image of the virtual light source.

Assignees

• Olympus Corp
• Nat Univ Corp Chiba Univ

Inventors

• Sambongi Masao
• Tsumura Norimichi
• Baba Kaori

Classifications

• H04N23/72

  Combination of two or more compensation controls · CPC title

• G01N21/57 Primary

  Measuring gloss · CPC title

• H04N23/743

  Bracketing, i.e. taking a series of images with varying exposure conditions · CPC title

• H04N23/69

  Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming · CPC title

• H04N23/12

  with one sensor only · CPC title