Image processing system, image processing device, image processing method, and computer-readable medium

Patent metadata

Field	Value
Publication number	US-11748977-B2
Application number	US-201917438632-A
Country	US
Kind code	B2
Filing date	Mar 22, 2019
Priority date	Mar 22, 2019
Publication date	Sep 5, 2023
Grant date	Sep 5, 2023

Abstract

A system includes: a sequential image string input unit configured to input a sequential image string having sequentiality; a reference image selection unit configured to select one or more images from the sequential image string as reference images; a variation calculation unit configured to select an adjacent reference image adjacent to the reference image from the sequential image string and calculate a variation between the reference image and the adjacent reference image; an image information regression unit configured to calculate class confidence by regression processing with the reference image as an input; a difference image information regression unit configured to calculate class confidence by regression processing with the variation as an input; a confidence integration unit configured to integrate class confidence calculated by the image information regression unit and class confidence calculated by the difference image information regression unit; and an output unit configured to output the integrated class confidence.

First claim

What is claimed is:

1. An image processing system comprising:
a sequential image string input unit configured to input a sequential image string having sequentiality;
a reference image selection unit configured to select one or more images from the sequential image string as reference images;
a first variation calculation unit configured to select an adjacent reference image adjacent to the reference image from the sequential image string and calculate a first variation being a variation between the reference image and the adjacent reference image;
an image information regression unit configured to calculate class confidence by regression processing with the reference image as an input;
a difference image information regression unit configured to calculate class confidence by regression processing with the first variation as an input;
a confidence integration unit configured to integrate class confidence calculated by the image information regression unit and class confidence calculated by the difference image information regression unit; and
an output unit configured to output the integrated class confidence.

2. The processing system according to claim 1 , wherein
the image information regression unit calculates class confidence by regression processing using an image information regression function with the reference image as an input, and
the difference image information regression unit calculates class confidence by regression processing using a difference image information regression function with the first variation as an input.

3. The processing system according to claim 2 , further comprising:
a learning sequential image string input unit configured to input a learning sequential image string having sequentiality;
a teacher image information input unit configured to input image information to be teacher data as teacher image information;
a learning reference image selection unit configured to select one or more images from the learning sequential image string as learning reference images;
a second variation calculation unit configured to select an adjacent learning reference image adjacent to the learning reference image from the learning sequential image string and calculate a second variation being a variation between the learning reference image and the adjacent learning reference image;
an image information regression function estimation unit configured to estimate the image information regression function from the teacher image information and the learning reference image; and
a difference image information regression function estimation unit configured to estimate the difference image information regression function from the teacher image information and the second variation, wherein
the image information regression unit uses an image information regression function estimated by the image information regression function estimation unit as the image information regression function, and
the difference image information regression unit uses a difference image information regression function estimated by the difference image information regression function estimation unit as the difference image information regression function.

4. The processing system according to claim 1 , wherein
the reference image selection unit selects an image in a specific frame from the sequential image string as the reference image, and
the first variation calculation unit
selects an image in a frame subsequent to the reference image from the sequential image string as the adjacent reference image and calculates a variation between the reference image and the adjacent reference image, and,
from then on, repeats an operation of successively selecting an image in a frame after a current adjacent reference image from the sequential image string as the adjacent reference image and calculating a variation between the selected adjacent reference image and the reference image.

5. The processing system according to claim 1 , wherein
the reference image selection unit selects an image in a specific frame from the sequential image string as the reference image, and
the first variation calculation unit
selects an image in a frame subsequent to the reference image from the sequential image string as the adjacent reference image and calculates a variation between the reference image and the adjacent reference image, and,
from then on, repeats an operation of successively selecting an image in a frame after a current adjacent reference image from the sequential image string as the adjacent reference image and calculating a variation between the selected adjacent reference image and the adjacent reference image in an immediately preceding frame.

6. An image processing device comprising:
a sequential image string input unit configured to input a sequential image string having sequentiality;
a reference image selection unit configured to select one or more images from the sequential image string as reference images;
a first variation calculation unit configured to select an adjacent reference image adjacent to the reference image from the sequential image string and calculate a first variation being a variation between the reference image and the adjacent reference image;
an image information regression unit configured to calculate class confidence by regression processing with the reference image as an input;
a difference image information regression unit configured to calculate class confidence by regression processing with the first variation as an input;
a confidence integration unit configured to integrate class confidence calculated by the image information regression unit and class confidence calculated by the difference image information regression unit; and
an output unit configured to output the integrated class confidence.

7. An image processing method by an image processing device, the method comprising:
a step of inputting a sequential image string having sequentiality;
a step of selecting one or more images from the sequential image string as reference images;
a step of selecting an adjacent reference image adjacent to the reference image from the sequential image string and calculating a first variation being a variation between the reference image and the adjacent reference image;
a first regression step of calculating class confidence by regression processing with the reference image as an input;
a second regression step of calculating class confidence by regression processing with the first variation as an input;
a step of integrating class confidence calculated by the first regression step and class confidence calculated by the second regression step; and
a step of outputting the integrated class confidence.

Assignees

• Nec Corp

Inventors

• Namiki Shigeaki
• Shibata Takashi
• Yachida Shoji

Classifications

• G06V10/766 Primary

  using regression, e.g. by projecting features on hyperplanes · CPC title

• G06T7/20

  Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51) · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title

• G06V10/82 Primary

  using neural networks · CPC title

• G06F18/27

  Regression, e.g. linear or logistic regression · CPC title