Image processing apparatus, system, image processing method, and non-transitory computer-readable storage medium

Patent metadata

Field	Value
Publication number	US-11361408-B2
Application number	US-202016739652-A
Country	US
Kind code	B2
Filing date	Jan 10, 2020
Priority date	Jan 21, 2019
Publication date	Jun 14, 2022
Grant date	Jun 14, 2022

Abstract

An image processing apparatus comprising, a specifying unit configured to specify, in a visible light image, a region having color information similar to first color information at a position where the visible light image is designated, an extraction unit configured to extract second color information from the specified region in the visible light image, and, a composition unit configured to generate a composite image by superimposing the second color information on the specified region in an invisible light image synchronously captured by an optical system common to the visible light image.

First claim

What is claimed is:

1. An image processing apparatus comprising:
a specifying unit configured to specify, in a visible light image, a region having color information similar to first color information at a position where the visible light image is designated;
an extraction unit configured to extract second color information from the specified region in the visible light image; and
a composition unit configured to generate a composite image by superimposing the second color information on the specified region in an invisible light image synchronously captured by an optical system common to the visible light image,
wherein the second color information is a central value of the color information included in the specified region.

2. The apparatus according to claim 1 , wherein the specifying unit specifies, as the region having the similar color information, a region including a pixel having color information belonging to a predetermined color difference range based on the first color information.

3. The apparatus according to claim 1 , further comprising:
an image capturing apparatus including:
one optical system;
a visible light image sensor configured to receive visible light having passed through the one optical system and generate a visible light image; and
an invisible light image sensor configured to receive invisible light having passed through the one optical system and generate an invisible light image; and

a control unit configured to control an exposure time of the visible light image sensor.

4. The apparatus according to claim 3 , wherein
the invisible light is infrared light, and
the image processing apparatus further comprises an illumination device configured to illuminate, with infrared light, an object to be captured by the image capturing apparatus.

5. The apparatus according to claim 3 , further comprising an acquisition unit configured to acquire an instruction to change an intensity of the second color information,
wherein if the acquisition unit acquires the instruction, the control unit adjusts the exposure time in accordance with the change of the intensity.

6. The apparatus according to claim 5 , wherein if the control unit extends the exposure time, the extraction unit extracts the second color information from an extraction region narrower than the specified region.

7. The apparatus according to claim 6 , wherein the exposure time extended by the control unit is longer than an exposure time of the invisible light image sensor.

8. The apparatus according to claim 6 , wherein if the exposure time is extended, the composition unit generates the composite image by superimposing the second color information extracted from the extraction region on an invisible light image generated after the second color information is extracted from a visible light image captured with the extended exposure time.

9. The apparatus according to claim 8 , wherein the composition unit superimposes the second color information extracted from the extraction region on a region, corresponding to the specified region, in the generated invisible light image.

10. The apparatus according to claim 6 , further comprising an output unit configured to output a notification when a pixel value included in the specified region is saturated in the visible light image captured with the exposure time according to the change of the intensity of the second color information.

11. The apparatus according to claim 10 , further comprising a determination unit configured to determine image quality of the extraction region,
wherein the control unit further adjusts the exposure time in accordance with an image quality determination result by the determination unit.

12. The apparatus according to claim 11 , wherein if the determination unit determines that a noise amount of the extraction region exceeds a threshold, the control unit extends the exposure time.

13. The apparatus according to claim 12 , wherein if the determination unit determines that the extraction region includes a saturated pixel, the threshold is changed to a smaller value.

14. A system comprising:
an image processing apparatus including:
a specifying unit configured to specify, in a visible light image, a region having color information similar to first color information at a position where the visible light image is designated;
an extraction unit configured to extract second color information from the specified region in the visible light image; and
a composition unit configured to generate a composite image by superimposing the second color information on the specified region in an invisible light image synchronously captured by an optical system common to the visible light image,
wherein the second color information is a central value of the color information included in the specified region; and

an information processing apparatus configured to acquire the composite image by communicating with the image processing apparatus.

15. An image processing method comprising:
specifying, in a visible light image, a region having color information similar to first color information at a position where the visible light image is designated;
extracting second color information from the specified region in the visible light image; and
generating a composite image by superimposing the second color information on the specified region in an invisible light image synchronously captured by an optical system common to the visible light image,
wherein the second color information is a central value of the color information included in the specified region.

16. A non-transitory computer-readable storage medium storing one or more program including instructions that, when executed by a processor of an image processing apparatus, causes the processor to perform operations of:
specifying, in a visible light image, a region having color information similar to first color information at a position where the visible light image is designated;
extracting second color information from the specified region in the visible light image; and
generating a composite image by superimposing the second color information on the specified region in an invisible light image synchronously captured by an optical system common to the visible light image,

wherein the second color information is a central value of the color information included in the specified region.

Assignees

• Canon Kk

Inventors

• Ise Yasuaki

Classifications

• H04N23/84 Primary

  for processing colour signals · CPC title

• H04N23/73

  by influencing the exposure time · CPC title

• H04N23/631

  Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters · CPC title

• H04N23/45

  for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images · CPC title

• H04N23/11

  for generating image signals from visible and infrared light wavelengths · CPC title