Image processing apparatus and method for recognizing state of subject

Patent metadata

Field	Value
Publication number	US-12424015-B2
Application number	US-202117505416-A
Country	US
Kind code	B2
Filing date	Oct 19, 2021
Priority date	Apr 22, 2019
Publication date	Sep 23, 2025
Grant date	Sep 23, 2025

Abstract

An image processing apparatus includes detecting means for detecting a plurality of predetermined parts of a subject from an image, estimating means for estimating a cause of failed detection if any of the plurality of predetermined parts is not detected in a result of the detection made by the detecting means, and determining means for determining a state of the subject based on a part detected by the detecting means and the cause estimated by the estimating means.

First claim

The invention claimed is:

1. An image processing apparatus comprising: a memory storing instructions; and at least one processor configured to execute the instructions to; detect a plurality of predetermined parts of a subject from an image; estimate a cause of failed detection if any of the plurality of predetermined parts is not detected in a result of the detection made; determine a state of the subject based on a part detected and the cause estimated; extract a feature quantity of the subject based on the part detected; classify the feature quantity into one of a plurality of categories based on the state determined; and recognize a characteristic of the subject based on a result of the classification performed.

2. The image processing apparatus according to claim 1 , the cause of the failed detection is estimated based on a position of the subject in the image.

3. The image processing apparatus according to claim 2 , wherein the cause of the failed detection is estimated based on a distance between the position of the subject in the image and an end portion of the image.

4. The image processing apparatus according to claim 1 , wherein the cause of the failed detection is estimated based on a position of the subject and a position of an object not being the subject.

5. The image processing apparatus according to claim 1 , wherein the state of the subject is determined based on a position of the detected part and the estimated cause.

6. The image processing apparatus according to claim 5 , wherein the position of the part is a position relative to a reference position of the subject.

7. The image processing apparatus according to claim 1 , wherein whether the state of the subject is normal or abnormal is determined.

8. The image processing apparatus according to claim 1 , wherein a pose of the subject is determined.

9. The image processing apparatus according to claim 8 , wherein whether the pose of the subject is abnormal is determined.

10. The image processing apparatus according to claim 1 , wherein the subject is a human, and the predetermined parts are joints or regions of the human.

11. The image processing apparatus according to claim 1 , wherein a method of recognition is changed based on a result of the classification performed.

12. The image processing apparatus according to claim 11 , wherein a feature quantity classified into some of the categories is excluded from a target of recognition.

13. The image processing apparatus according to claim 11 , wherein the method of recognition is changed by changing a threshold used in the recognition.

14. The image processing apparatus according to claim 11 , wherein the method of recognition is changed by changing a statistical model used in the recognition.

15. The image processing apparatus according to claim 1 , wherein the feature quantity is classified for each of a plurality of images including the subject; and the characteristic of the subject is recognized based on a result of the classification performed for the plurality of images.

16. The image processing apparatus according to claim 1 , wherein the feature quantity of the subject is classified using a plurality of chronological images.

17. The image processing apparatus according to claim 1 , wherein the at least one processor is further configured to execute the instructions to track the subject using the plurality of chronological images, wherein the feature quantity of the subject is classified based on a result of the tracking.

18. An image processing method comprising:
a detecting step of detecting a plurality of predetermined parts of a subject from an image;
an estimating step of estimating a cause of failed detection if any of the plurality of predetermined parts is not detected in a result of the detection made in the detecting step;
a determining step of determining a state of the subject based on a part detected in the detecting step and the estimated cause;
extracting step of extracting a feature quantity of the subject based on the part detected by the detecting step;
classifying step of classifying the feature quantity into one of a plurality of categories based on the state determined by the determining step; and
characteristic recognizing step of recognizing a characteristic of the subject based on a result of the classification performed by the classifying step.

19. A non-transitory computer-readable medium storing a program causing a computer to execute an image processing method, the image processing method comprising: a detecting step of detecting a plurality of predetermined parts of a subject from an image; an estimating step of estimating a cause of failed detection if any of the plurality of predetermined parts is not detected in a result of the detection made in the detecting step; a determining step of determining a state of the subject based on a part detected in the detecting step and the estimated cause; an extracting step of extracting a feature quantity of the subject based on the part detected by the detecting step; a classifying step of classifying the feature quantity into one of a plurality of categories based on the state determined by the determining step; and a characteristic recognizing step of recognizing a characteristic of the subject based on a result of the classification performed by the classifying step.

Assignees

• Canon Kk

Inventors

• Sato Shunsuke
• Takeuchi Koichi

Classifications

• G06N7/01

  Probabilistic graphical models, e.g. probabilistic networks · CPC title

• G06F18/2431

  Multiple classes · CPC title

• G06F18/2415

  based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate · CPC title

• G06V10/98

  Detection or correction of errors, e.g. by rescanning the pattern or by human intervention; Evaluation of the quality of the acquired patterns · CPC title

• G06V10/462

  Salient features, e.g. scale invariant feature transforms [SIFT] · CPC title