Moving object monitoring device, server device, and moving object monitoring system

Patent metadata

Field	Value
Publication number	US-10825158-B2
Application number	US-201816486353-A
Country	US
Kind code	B2
Filing date	Jan 17, 2018
Priority date	Feb 28, 2017
Publication date	Nov 3, 2020
Grant date	Nov 3, 2020

Abstract

Provided is a color camera that captures using environmental light, a monochrome camera that captures using infrared light, a signal processing unit that processes a signal of a color image output from the color camera and a signal of a monochrome image output from the monochrome camera, a storage unit that stores alignment information generated from a color image and a monochrome image captured in a state of sufficient environmental light, an alignment unit that performs an alignment based on alignment information to match positions of a subject shown in a color image and a monochrome image captured in a state of insufficient environmental light, and an image synthesis unit that acquires color information from the aligned color image and performs color conversion to colorize the monochrome image using the color information.

First claim

The invention claimed is:

1. A moving object monitoring device that synthesizes a color image and a monochrome image obtained by capturing a monitoring area where a moving object to be monitored appears and outputs the synthesized image, the moving object monitoring device comprising:
a color camera that captures the monitoring area using environmental light;
a monochrome camera that captures the monitoring area using infrared light;
a processor that processes a signal of a color image output from the color camera and a signal of a monochrome image output from the monochrome camera; and
a memory that stores alignment information generated based on the color image and the monochrome image captured in a state of sufficient environmental light,
wherein the moving object monitoring device is stationary,
wherein the processor further
performs an alignment, based on the alignment information, to match positions of images of a subject shown respectively in the color image and the monochrome image captured in a state of insufficient environmental light, and
acquires color information from the aligned color image and performs color conversion to colorize the monochrome image using the color information,

wherein the processor modifies capturing sensitivity of the color camera based on a detected signal level, the capturing sensitivity being increased by adding of signal values of adjacent pixels,
wherein, when the capturing sensitivity is increased, correspondingly reducing resolution by an inverse amount,
wherein the processor selects one of a first processing mode, a second processing mode, and a third processing mode based on the detected signal level of the color image,
wherein, when the detected signal level is greater than or equal to both a first threshold and a second threshold, the first processing mode is selected for signal processing,
wherein, when the detected signal level is greater or equal to the first threshold and less than the second threshold, the second processing mode is selected for signal processing,
wherein, when the detected signal level is less than the first threshold, the third processing mode is selected for signal processing,
wherein the first processing mode, the second processing mode, and the third processing mode have different levels of resolution conversion, and
wherein the first processing mode, the second processing mode, and the third processing mode have different levels of averaging reduction.

2. The moving object monitoring device of claim 1 ,
wherein the memory stores a plurality of pieces of the alignment information for each type of the moving object having a different shape, and
wherein the processor detects the moving object from the monochrome image, determines the type of the moving object, and performs the alignment based on the alignment information according to the type of the moving object.

3. The moving object monitoring device of claim 1 ,
wherein the processor generates the alignment information, and
wherein the memory stores the alignment information generated by the processor as calibration information together with information on processing conditions of the processor.

4. A server device that receives a color image and a monochrome image obtained by capturing, at a moving object monitoring device, a monitoring area where a moving object to be monitored appears, from the moving object monitoring device through a network, the server device comprising:
a communication device that receives a color image captured at the moving object monitoring device by a color camera that captures the monitoring area using environmental light and a monochrome image captured at the moving object monitoring device by a monochrome camera that captures the monitoring area using infrared light, wherein the moving object monitoring device is stationary;
a memory that stores alignment information generated based on the color image and the monochrome image captured in a state of sufficient environmental light; and
a processor that
performs an alignment, based on the alignment information, to match positions of images of a subject shown respectively in the color image and the monochrome image captured in a state of insufficient environmental light; and
acquires color information from the aligned color image and performs color conversion to colorize the monochrome image using the color information,

wherein capturing sensitivity of the color camera is modified based on a detected signal level, the capturing sensitivity being increased by adding of signal values of adjacent pixels,
wherein, when the capturing sensitivity is increased, resolution is correspondingly reduced by an inverse amount,
wherein the processor selects one of a first processing mode, a second processing mode, and a third processing mode based on the detected signal level of the color image,
wherein, when the detected signal level is greater than or equal to both a first threshold and a second threshold, the first processing mode is selected for signal processing,
wherein, when the detected signal level is greater or equal to the first threshold and less than the second threshold, the second processing mode is selected for signal processing,
wherein, when the detected signal level is less than the first threshold, the third processing mode is selected for signal processing,
wherein the first processing mode, the second processing mode, and the third processing mode have different levels of resolution conversion, and
wherein the first processing mode, the second processing mode, and the third processing mode have different levels of averaging reduction.

5. A moving object monitoring system that transmits a color image and a monochrome image obtained by capturing, at a moving object monitoring device, a monitoring area where a moving object to be monitored appears, from the moving object monitoring device to a server device through a network,
wherein the moving object monitoring device is stationary and includes
a color camera that captures the monitoring area using environmental light,
a monochrome camera that captures the monitoring area using infrared light,
a processor that processes a signal of a color image output from the color camera and a signal of a monochrome image output from the monochrome camera,
a memory that stores alignment information generated based on the color image and the monochrome image captured in a state of sufficient environmental light,
wherein the processor further
performs an alignment, based on the alignment information, to match positions of images of a subject shown respectively in the color image and the monochrome image captured in a state of insufficient environmental light, and
acquires color information from the aligned color image and performs color conversion to colorize the monochrome image using the color information, and

a communication device that transmits a synthesized image acquired by the processor to the server device,
wherein the processor modifies capturing sensitivity of the color camera based on a detected signal level, the capturing sensitivity being increased by adding of signal values of adjacent pixels,
wherein, when the capturing sensitivity is increased, resolution is correspondingly reduced by an inverse amount,
wherein the processor selects one of a first processing mode, a second processing mode, and a third processing mode based on the detected signal level of the color image,
wherein, when the detected signal level is greater than or equal to both a first threshold and a second threshold, the first processing mode is selected for signal processing,
wherein, when the detected signal level is greater or equal to the first threshold and less than the second threshold, the second processing mode is selected for

Assignees

• Panasonic Ip Man Co Ltd

Inventors

• Tanaka Shogo
• Tabei Kenji

Classifications

• H04N7/181 Primary

  for receiving images from a plurality of remote sources · CPC title

• G06V20/52

  Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69) · CPC title

• G06V10/75

  Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries · CPC title

• G06V10/56

  relating to colour · CPC title

• G06T5/50 Primary

  using two or more images, e.g. averaging or subtraction · CPC title