Image capturing apparatus, image capturing method, and image processing apparatus

Patent metadata

Field	Value
Publication number	US-11012633-B2
Application number	US-201916358923-A
Country	US
Kind code	B2
Filing date	Mar 20, 2019
Priority date	Mar 22, 2018
Publication date	May 18, 2021
Grant date	May 18, 2021

Abstract

An image capturing apparatus includes an image data acquirer configured to acquire pieces of image data at a plurality of relative movement positions to which an imaging surface of an imaging sensor is relatively moved with respect to an object light flux; an exposure changer configured to change an exposure when acquiring the pieces of image data at the plurality of relative movement positions; a dynamic range adjuster configured to adjust a dynamic range of the acquired pieces of image data; and an image data combiner configured to obtain composite image data of the pieces of image data, based on a positional shift amount between the pieces of image data and the dynamic range.

First claim

What is claimed is:

1. An image capturing apparatus comprising:
one or more hardware processors;
one or more hardware, memories containing instructions causing the one or more hardware processors to;
acquire pieces of image data including (i) a first piece of image data acquired at a first exposure and a first position relative to an image sensor, and (ii) a second piece of image data acquired at a second exposure and a second position; the second exposure different than the first exposure: the second position shifted an odd number of pixels relative to the first position;
adjust a dynamic range of the acquired pieces of image data; and
obtain composite image data of the pieces of image data, based on a positional shift amount between the pieces of image data, and the dynamic range;
wherein the composite image data is obtained by (i) for each pixel of the composite image data, obtaining red, green, or blue first color information from the first piece of image data acquired at the first exposure and the first position, (ii) for each pixel of the composite image data, obtaining red, green, or blue second color information from the second piece of image data acquired at the second exposure and the second position, and (iii) for each pixel of the composite image data, superimposing the first color information obtained from the first piece of image data and the second color information obtained from the second piece of image data.

2. The image capturing apparatus according to claim 1 , wherein the instructions cause the one or more hardware processors to adjust an overlapping amount of object luminance levels in the pieces of image data.

3. The image capturing apparatus according to claim 1 , wherein the instructions cause the one or more hardware processors to:
interpolate a first pixel in the first image data, among the pieces of image data, by a second pixel in the second image data among the pieces of image data, when the first pixel is included in the first image data.

4. The image capturing apparatus according to claim 3 , wherein the first pixel is a defective pixel and the second pixel is an interpolation pixel.

5. The image capturing apparatus according to claim 1 , wherein the instructions cause the one or more hardware processors to:
acquire a third piece of image data at a third position shifted with respect to the first position and the second position; the third piece of image data acquired at a third exposure that is the same as the second exposure;
acquire a fourth piece of image data at a fourth position shifted with respect to the first position, the second position, and the third position; the fourth piece of image data acquired at a fourth exposure different than the first exposure, the second exposure, and the third exposure;
wherein the step of obtaining the composite image data includes (iv) for each pixel of the composite image data, obtaining red, green, or blue first color information from the third piece of image data acquired at the third exposure and the third position, (v) for each pixel of the composite image data, obtaining red, green, or blue second color information from the fourth piece of image data acquired at the fourth exposure and the fourth position, and (vi) for each pixel of the composite image data, superimposing the third color information obtained from the third piece of image data and the fourth color information obtained from the fourth piece of image data over the superimposed first color information and second color information.

6. The image capturing apparatus according to claim 5 , wherein the first exposure range and the second exposure range overlap, and the second exposure range and the third exposure range overlap.

7. The image capturing apparatus according to claim 1 , wherein the second position is shifted only 1, 3, or 5 pixels relative to, the first position.

8. The image capturing apparatus according to claim 1 , wherein the second piece of image data is acquired by shifting a position of the imaging sensor relative to an optical axis of an imaging optical system.

9. The image capturing apparatus according to claim 1 , wherein the first piece of image data and the second piece of image data are acquired by analyzing pixel shift amounts of image data generated from photographing with camera shake.

10. An image capturing, method comprising:
acquiring pieces of image data including (i) a first piece of image data acquired at a first exposure and a first position relative to an image sensor, and (ii) a second piece of image data acquired at a second exposure and a second position; the second exposure different than the first exposure; the second position shifted an odd number of pixels relative to the first position;
adjusting a dynamic range of the acquired pieces of image data; and
obtaining composite image data of the pieces of image data, based on a positional shift amount between the pieces of image data and the dynamic range;
wherein the composite image data is obtained by (i) for each pixel of the composite image data, obtaining red, green, or blue first color information from the first piece, of image data acquired at the first exposure and the first position, (ii) for each pixel of the composite image data, obtaining red, green, or blue second color information from the second piece of image data acquired at the second exposure and the second position, and (iii) for each pixel of the composite image data, superimposing the first color information obtained from the first piece of image data and the second color information obtained from the second piece of image data.

11. An image processing apparatus comprising:
one or more hardware processors;
one or more hardware memories containing instructions causing the one or more hardware processors to:
detect a positional shift amount of a plurality of pieces of image data including (i) a first piece of image data acquired at a first exposure and a first position relative to an image sensor, and (ii) a second piece of image data acquired at a second exposure and a second position; the second exposure different than the first exposure; the second position shifted an odd number of pixels, relative to the first position;
adjust a dynamic range of the plurality of pieces of image data; and
obtain composite image data of the plurality of pieces of image data, based on the positional shift amount of the plurality of pieces of image data and the dynamic range;
wherein the composite image data is obtained by (i) for each pixel of the composite image data, obtaining red, green, or blue first color information from the first piece of image data acquired at the first exposure and the first position, (ii) for each pixel of the composite image data, obtaining red, green, or blue second color information from the second piece of image data acquired at the second exposure and the second position, and (iii) for each pixel of the composite image data, superimposing the first color information obtained from the first piece of image data and the second color information obtained from the second piece of image data.

Assignees

• Terauchi Masakazu
• Uehara Hiroyasu
• Miyata Masashi
• Kano Hiroshi
• Nishimura Takeyoshi
• Ricoh Co Ltd

Inventors

• Terauchi Masakazu
• Uehara Hiroyasu
• Miyata Masashi
• Kano Hiroshi
• Nishimura Takeyoshi

Classifications

• H04N25/68

  applied to defects · CPC title

• H04N25/48

  Increasing resolution by shifting the sensor relative to the scene · CPC title

• H04N23/951 Primary

  by using two or more images to influence resolution, frame rate or aspect ratio · CPC title

• H04N23/741 Primary

  by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors · CPC title

• H04N23/687

  by shifting the lens or sensor position · CPC title