Image processing device, method for controlling the same, program, and storage medium

Patent metadata

Field	Value
Publication number	US-2018260961-A1
Application number	US-201815909708-A
Country	US
Kind code	A1
Filing date	Mar 1, 2018
Priority date	Mar 9, 2017
Publication date	Sep 13, 2018
Grant date	—

Abstract

A circuitry of an image processing device divides a first image into a plurality of regions, extracts a feature point from each of the regions, tracks the feature point among a plurality of images to detect a motion vector, estimates a notable target of the first image, calculates the priority level of setting of a tracking feature point for each of the regions for tracking motion of the notable target, and sets the tracking feature point to any of the regions based on the priority level.

First claim

What is claimed is:

1 . An image processing device comprising:
circuitry configured to divide a first image into a plurality of regions, extract a feature point from each of the regions, track the feature point among a plurality of images to detect a motion vector, estimate a notable target of the first image, calculate a priority level of setting of a tracking feature point for each of the regions for tracking motion of the notable target, and set the tracking feature point to any of the regions based on the priority level.

2 . The image processing device according to claim 1 , wherein
the circuitry estimates the notable target based on at least any of image capturing mode information, main object information, shutter speed information, focal length information, depth information, inertial sensor information, and user instruction information.

3 . The image processing device according to claim 1 , wherein
the circuitry calculates the priority level by a first calculation method for calculating a higher priority level of setting of the tracking feature point for a predetermined region, of the plurality of regions, as the number of feature points in the predetermined region and a region, of the plurality of regions, around the predetermined region decreases.

4 . The image processing device according to claim 3 , wherein
the circuitry calculates the priority level by the first calculation method in a case where a background is estimated as the notable target.

5 . The image processing device according to claim 1 , wherein
the circuitry calculates the priority level by a second calculation method for calculating a higher priority level of setting of the tracking feature point for a predetermined region, of the plurality of regions, as a distance between a representative position of the predetermined region and a representative position of a main object region, of the plurality of regions, decreases.

6 . The image processing device according to claim 5 , wherein
the circuitry calculates the priority level by the second calculation method in a case where an object is estimated as the notable target.

7 . The image processing device according to claim 2 , wherein
the circuitry calculates a background level and an object level for the notable target upon image capturing based on at least one of the image capturing mode information, the main object information, the shutter speed information, the focal length information, the depth information, the inertial sensor information, and the user instruction information.

8 . The image processing device according to claim 7 , wherein
the circuitry performs weight summing for multiple background levels and multiple object levels, thereby calculating a final background level and a final object level.

9 . The image processing device according to claim 8 , wherein
the circuitry compares the background level and the object level with each other to estimate a background as the notable target when the background level is greater and to estimate an object as the notable target when the object level is greater.

10 . The image processing device according to claim 1 , wherein
the circuitry calculates reliability of tracking of the feature point, thereby calculating a higher priority level of setting of the tracking feature point for a region, of the plurality of regions, having more feature points with high reliability.

11 . The image processing device according to claim 1 , wherein
the circuitry calculates reliability of tracking of the feature point based on a result of calculation of a correlation evaluation value obtained upon detection of the motion vector, and sets the tracking feature point to any of the regions based on the reliability and the priority level.

12 . The image processing device according to claim 11 , wherein
the circuitry performs weight summing for multiple reliabilities of tracking of the feature point, thereby calculating final reliability of tracking of the feature point.

13 . The image processing device according to claim 1 , wherein
the circuitry calculates high reliability of tracking of the feature point as an amount of change in a feature amount of the feature point before and after tracking of the feature point decreases, and sets the tracking feature point to any of the regions based on the reliability and the priority level.

14 . The image processing device according to claim 1 , wherein
the circuitry calculates reliability of tracking of the feature point, thereby setting, in descending order of the priority level of the each of the regions, a feature point newly extracted from a region, of the plurality of regions, with higher reliability as the tracking feature point instead of a feature point with reliability lower than a predetermined threshold.

15 . An image processing method comprising:
dividing a first image into a plurality of regions; extracting a feature point from each of the regions; tracking the feature point among a plurality of images to detect a motion vector; estimating a notable target of the first image; calculating a priority level of setting of a tracking feature point for each of the regions for tracking motion of the notable target; and setting the tracking feature point to any of the regions based on the priority level.

16 . A computer-readable non-volatile storage medium storing a program for causing a computer to execute each step of an image processing method, wherein the image processing method including
a region division step of dividing a first image into a plurality of regions; a feature point extraction step of extracting a feature point from each of the regions; a vector detection step of tracking the feature point among a plurality of images to detect a motion vector; a notable target estimation step of estimating a notable target of the first image; a priority level calculation step of calculating a priority level of setting of a tracking feature point for each of the regions for tracking motion of the notable target by means of output at the notable target estimation step; and a feature point compensation step of setting the tracking feature point to any of the regions based on the priority level.

Assignees

• Canon Kk

Inventors

• Narita Yu

Classifications

• G06T7/246 Primary

  using feature-based methods, e.g. the tracking of corners or segments · CPC title

• G06T7/20 Primary

  Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51) · CPC title

• H04N23/60

  Control of cameras or camera modules · CPC title

• G06V10/462

  Salient features, e.g. scale invariant feature transforms [SIFT] · CPC title

• G06V10/759

  Region-based matching · CPC title