Subject recognizing method and apparatus

Patent metadata

Field	Value
Publication number	US-11423266-B2
Application number	US-202016903702-A
Country	US
Kind code	B2
Filing date	Jun 17, 2020
Priority date	Aug 14, 2017
Publication date	Aug 23, 2022
Grant date	Aug 23, 2022

Abstract

Disclosed is a subject recognizing apparatus and method. The method may include extracting feature points from a target image, respectively measuring movement information of each of a plurality of the extracted feature points, selectively grouping the extracted feature points into one or more groups based on the respectively measured movement information, determining a type of subject present in at least one group of the one or more groups based on at least a portion of the subject present in the at least one group, and recognizing a subject included in the target image based on the determined type of subject.

First claim

What is claimed is:

1. A processor-implemented subject recognizing method, the method comprising:
extracting feature points from a target image being received in addition to any one and any combination of a depth map and a disparity mapping associated with the target image;
respectively measuring movement information of each of a plurality of the extracted feature points, wherein the measured movement information is based on coordinate position movement of each of the plurality of the extracted feature points;
selectively grouping the extracted feature points into one or more groups based on the respectively measured movement information;
determining a type of subject present in at least one group of the one or more groups based on at least a portion of the subject present in the at least one group; and
recognizing a subject included in the target image based on the determined type of subject.

2. The method of claim 1 , wherein the respective measuring of the movement information includes measuring a temporal movement or a spatial movement of each of the plurality of the extracted feature points.

3. The method of claim 1 , wherein the recognizing of the subject included in the target image is based on a non-occluded portion of the subject, without consideration of an occluded portion of the subject not present in the target image, based on the determined type of subject.

4. The method of claim 1 , wherein the determining the type of the subject present in the at least one group of the one or more groups includes determining a feature point descriptor for a feature point, of the plurality of the extracted feature points, included in the one group and hierarchically spreading the feature point descriptor from a root node to a leaf node of an image vocabulary tree to determine a most similar class of plural classes of the image vocabulary tree, the most similar class being indicative of a corresponding type of subject present in the one group.

5. The method of claim 1 , wherein the determining of the type of the subject present in the at least one group of the one or more groups includes determining respective types of subjects in each of two or more groups, including comparing each of the two or more groups to classes in which source images of objects including the subject are preclassified, and
wherein the recognizing of the subject includes determining whether any of the two or more groups include an identical type among the determined respective types of subjects, and combining select groups in which the identical type is determined based on corresponding measured movement information of the respectively measured movement information.

6. The method of claim 5 , wherein the classes are preclassified in an image vocabulary database structure in a hierarchical classification of the source images of the objects in a feature space determined based on elements of feature point descriptors indicating features of pixels associated with feature points in the source images.

7. The method of claim 6 , further comprising performing the preclassifying by performing the hierarchical classifying of the source images, with the image vocabulary database structure being an image vocabulary tree.

8. The method of claim 5 , further comprising:
extracting, from the one or more groups, a group in which a corresponding type of subject is indeterminable.

9. The method of claim 5 , wherein the combining comprises selectively combining the select groups, the selective combining being dependent on a determination, based on the corresponding measured movement information, of which of a single subject and different subjects the select groups correspond to.

10. The method of claim 9 , wherein the combining further comprises:
calculating the corresponding movement information by calculating movement information of each of the select groups by respectively combining pieces of movement information of feature points for each of the select groups; and
the determination of which of the single subject and the different subjects the select groups correspond to being based on results of comparing the respectively combined pieces of the movement information.

11. The method of claim 1 , wherein the respective measuring of the movement information of each of the plurality of the extracted feature points comprises determining temporal movement information indicating a respective temporal movement of each of the plurality of extracted feature points based on a reference image captured before or after a point in time at which the target image is captured.

12. The method of claim 1 , wherein the respective measuring of the movement information of each of the plurality of the extracted feature points comprises:
determining spatial movement information indicating a respective spatial movement of each of the plurality of the extracted feature points based on depth information indicating a determined depth for each of the plurality of the extracted feature points from a camera based on comparisons of the target image and a parallax image corresponding to the target image and captured at a same time as the target image by the camera.

13. The method of claim 12 , wherein the camera is a stereo camera including at least two image sensors separated by a preset distance, at least one of the two image sensors capturing the target image and the two image sensors being used to generate the parallax image.

14. A non-transitory computer readable medium storing instructions that, when executed by a processor, cause the processor to implement the method of claim 1 .

15. A processor-implemented subject recognizing method, the method comprising:
extracting feature points from a target image being received in addition to any one and any combination of a depth map and a disparity mapping associated with the target image;
determining coordinate position movement information of the extracted feature points;
grouping the extracted feature points to generate plural groups of the extracted feature points, based on the determined coordinate position movement information of the extracted feature points;
determining respective types of subjects included in each of the plural groups, including determining whether two or more groups of the plural groups that have determined identical types of subject include at least respective portions of a same subject; and
recognizing a subject included in the target image based on the determined respective types of subjects.

16. The method of claim 15 , wherein the generating of the plural groups comprises generating the plural groups based on respectively determined similarities between the extracted feature points, the respective determination of the similarities being determined by comparing respective coordinates of the extracted feature points and the determined coordinate position movement information of the extracted feature points, and
wherein the determining of the coordinate position movement information of the extracted feature points comprises comparing the target image to a reference image captured before the target image is captured.

17. The method of claim 15 , wherein the determining of the respective types of subjects comprises determining a type of subject present in each of the plural groups by comparing each of the plural groups to classes in which source images of objects are preclassified in hierarchical classification of the source images of the objects in a feature space determined based on elements of feature point descriptors indicating features of pixels associated with feature

Assignees

• Samsung Electronics Co Ltd

Inventors

• Lee Huijin
• Ahn Minsu
• Hong Sunghoon

Classifications

• G06V10/7625

  Hierarchical techniques, i.e. dividing or merging patterns to obtain a tree-like representation; Dendograms · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title

• G06F18/24323

  Tree-organised classifiers · CPC title

• G06V20/10 Primary

  Terrestrial scenes (scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59) · CPC title

• G06F18/231

  Hierarchical techniques, i.e. dividing or merging pattern sets so as to obtain a dendrogram · CPC title