Image processing apparatus for determining an object area where part of the object is occluded

Patent metadata

Field	Value
Publication number	US-12217528-B2
Application number	US-202217695622-A
Country	US
Kind code	B2
Filing date	Mar 15, 2022
Priority date	Sep 20, 2019
Publication date	Feb 4, 2025
Grant date	Feb 4, 2025

Abstract

An image processing apparatus includes a first detection unit configured to detect, from an image in which an object including a plurality of parts is captured, first feature points corresponding to the parts of the object, an acquisition unit configured to acquire a reliability indicating a likelihood that a position indicated by a feature point is a part corresponding to the feature point for each of the first feature points detected by the first detection unit, a second detection unit configured to detect a second feature point based on some of the first feature points for a part corresponding to a first feature point with the low reliability, and a determination unit configured to determine an area including the object based on some of the first feature points and the second feature point.

First claim

The invention claimed is:

1. An image processing apparatus comprising:
at least one memory storing instructions; and
at least one processor that, upon execution of the stored instructions, are configured to:
detect, from an image in which an object including a plurality of parts is captured, first feature points corresponding to the parts of the object;
acquire a reliability indicating a likelihood that a position indicated by a feature point is a part corresponding to the feature point for each of the first feature points detected;
detect a second feature point based on some of the first feature points for a part corresponding to a first feature point with the low reliability; and
determine an area including the object based on some of the first feature points and the second feature point.

2. The image processing apparatus according to claim 1 , wherein the second feature point detected is obtained by correcting a position of a part indicated by the first feature point whose reliability is lower than a predetermined value among the first feature points based on the first feature point indicating a position of a part close to the part indicated by the first feature point whose reliability is lower than the predetermined value among the first feature points.

3. The image processing apparatus according to claim 1 , wherein the second feature point detected is obtained by correcting a position of a part indicated by the first feature point whose reliability is lower than a predetermined value among the first feature points based on the first feature point whose reliability is higher than the predetermined value among the first feature points.

4. The image processing apparatus according to claim 1 , wherein the second feature point obtained by correcting a position of a part indicated by the first feature point with the low reliability among the first feature points based on a positional relationship among the plurality of parts included in the object.

5. The image processing apparatus according to claim 4 ,
wherein positions of a head, a neck, a waist, and an ankle of a person is detected as the first feature points, and
wherein, in a case where the reliability of the position of the ankle is low, the position of the ankle is detected as the second feature point based on a positional relationship between one of the head and the waist of the person and the ankle of the person.

6. The image processing apparatus according to claim 1 , wherein the second feature point is detected for the part indicated by the first feature point whose reliability is lower than the predetermined value among the first feature points detected, based on the first feature point indicating the part and whose reliability is higher than a predetermined value in a previous image.

7. The image processing apparatus according to claim 6 ,
wherein positions of a head, a neck, a waist, and an ankle of a person are detected as the first feature points from the image,
wherein the reliability of the position of the ankle of the person is acquired in a previous image captured before the image is captured, and
wherein, in a case where the reliability of the position of the ankle of the person in the previous image is higher than the predetermined value, the position of the ankle of the person in the previous image is detected as the second feature point in the image.

8. The image processing apparatus according to claim 1 ,
wherein the object is a person, and
wherein the first feature points are detected by inputting the image to a trained model obtained by learning of feature points corresponding to the parts of the person as the first feature points of the person.

9. The image processing apparatus according to claim 1 , wherein the at least one processor is further configured to:
extract a feature amount for recognizing the object based on a partial image obtained by clipping the area determined from the image; and
recognize whether the object captured in the image is identical to a specific object being preliminarily registered, by comparing the extracted feature amount with a feature amount of the specific object.

10. The image processing apparatus according to claim 9 ,
wherein the object is a human body, and
wherein a person captured in the image is identified from among preliminarily registered persons by comparing the extracted feature amount with feature amounts of the preliminarily registered persons.

11. The image processing apparatus according to claim 10 , wherein the feature amount is extracted of a partial image obtained by clipping the area determined from the image, based on a trained model for outputting a feature amount indicating each part of the object from an input image.

12. The image processing apparatus according to claim 10 , wherein a feature amount is extracted from a partial area corresponding to the part of which the reliability of the first feature point is higher in a partial image obtained by clipping the area determined from the image.

13. The image processing apparatus according to claim 1 , wherein the at least one processor is further configured to integrate the feature amount extracted from the image for each of the parts based on the reliability.

14. The image processing apparatus according to claim 1 , wherein, in a case where a feature point is located outside a predetermined area, the reliability of the feature point is acquired in such a manner that the reliability is low.

15. The image processing apparatus according to claim 1 , wherein an area including the object is determined based on a central axis of the object estimated based on the detected first feature points.

16. The image processing apparatus according to claim 1 , wherein a rectangle including some of the first feature points is determined as the area including the object.

17. The image processing apparatus according to claim 1 , wherein the at least one processor is further configured to output the first feature points detected and the second feature point detected in a distinguishable manner.

18. An image processing apparatus comprising:
at least one memory storing instructions; and
at least one processor that, upon execution of the stored instructions, are configured to:
acquire a feature point indicating a position in an image of each of a plurality of parts of an object detected from the image, and a reliability indicating a likelihood that a part correspond to the feature point;
extract a first image feature from the image for each of the parts indicated by the feature points;
output a second image feature indicating a feature amount specific to the object based on the reliability and the first image feature; and
identify the object detected from the image and a preliminarily registered predetermined object based on a feature amount indicating the preliminarily registered predetermined object and the second feature amount.

19. An image processing method comprising:
detecting, as a first detection, from an image in which an object including a plurality of parts is captured, first feature points corresponding to the parts of the object;
acquiring a reliability indicating a likelihood that a position indicated by a feature point is a part corresponding to the feature point for each of the first feature points detected in the first detection;
detecting, as a second detection, a second feature point based on some of the first feature points for a part corresponding to the first feature point with the low reliability; and
determining an area inc

Assignees

• Canon Kk

Inventors

• Uchiyama Hiroyuki
• Tojo Hiroshi
• Yamamoto Shinji

Classifications

• G06T2207/30196

  Human being; Person · CPC title

• G06T2207/20081

  Training; Learning · CPC title

• G06T2207/20076

  Probabilistic image processing · CPC title

• G06V10/98

  Detection or correction of errors, e.g. by rescanning the pattern or by human intervention; Evaluation of the quality of the acquired patterns · CPC title

• G06V10/751

  Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching · CPC title