Method and system for human motion recognition

Patent metadata

Field	Value
Publication number	US-2016148391-A1
Application number	US-201414898346-A
Country	US
Kind code	A1
Filing date	Jun 12, 2014
Priority date	Jun 12, 2013
Publication date	May 26, 2016
Grant date	—

Abstract

A system and method for human motion recognition are provided. The system includes a video sequence decomposer, a feature extractor, and a motion recognition module. The video sequence decomposer decomposes a video sequence into a plurality of atomic actions. The feature extractor extracts features from each of the plurality of atomic actions, the features including at least a motion feature and a shape feature. And the motion recognition module performs motion recognition for each of the plurality of atomic actions in response to the features.

First claim

1 . A method for human motion recognition comprising:
decomposing a video sequence into a plurality of atomic actions;
extracting features from each of the plurality of atomic actions, the features comprising at least a motion feature and a shape feature; and
performing motion recognition for each of the plurality of atomic actions in response to the features.

2 . The method in accordance with claim 1 wherein the step of performing motion recognition for each of the plurality of atomic actions comprises performing motion recognition for each of the plurality of atomic actions by convolving histograms of the features of each of the plurality of atomic actions.

3 . The method in accordance with claim 2 wherein the step of extracting features from each of the plurality of atomic actions comprises extracting a set of shape vectors depicting shape flow from each of the plurality of atomic actions.

4 . The method in accordance with claim 3 wherein convolving histograms of the features of each of the plurality of atomic actions comprises deriving a shape descriptor by determining a histogram-of-oriented gradient of the set of shape vectors for each of the plurality of atomic actions.

5 . The method in accordance with claim 2 wherein the step of extracting features from each of the plurality of atomic actions comprises extracting a set of motion vectors depicting motion flow from each of the plurality of atomic actions.

6 . The method in accordance with claim 5 wherein convolving histograms of the features of each of the plurality of atomic actions comprises deriving a motion descriptor by determining a histogram-of-oriented optical flow of the set of motion vectors for each of the plurality of atomic actions.

7 . The method in accordance with claim 2 wherein the features further comprise a pose feature, and wherein the step of extracting features from each of the plurality of atomic actions comprises extracting a set of pose vectors from each of the plurality of atomic actions, and wherein convolving histograms of the features of each of the plurality of atomic actions comprises deriving a pose descriptor by determining a histogram-of-oriented gradient of the set of shape vectors for each of the plurality of atomic actions.

8 . The method in accordance with claim 1 wherein the features further comprise a spatial feature, and wherein the step of extracting features from each of the plurality of atomic actions comprises deriving each of a set of shape, motion or pose vectors for each of two or more regions of a bounding box within each of the plurality of atomic actions, the bounding box in each of the plurality of atomic actions configured to include all of a subject pictured in the one of the plurality of atomic actions, and wherein the step of performing motion recognition for each of the plurality of atomic actions comprises convolving histograms of each of the shape, motion or pose descriptors to generate a resultant histogram.

9 . The method in accordance with claim 2 wherein convolving histograms of the features of each of the plurality of atomic actions comprises normalization of the histograms of each of the plurality of atomic actions to sum up to unity.

10 . The method in accordance with claim 1 further comprising K-means clustering of all of the atomic actions to generate a distance weighted bag-of-automatic-actions model of the video sequence.

11 . A system for human motion recognition comprising:
a video sequence decomposer for decomposing a video sequence into a plurality of atomic actions;
a feature extractor for extracting features from each of the plurality of atomic actions, the features comprising at least a motion feature and a shape feature; and
a motion recognition module for performing motion recognition for each of the plurality of atomic actions in response to the features.

12 . The system in accordance with claim 11 wherein the motion recognition module performs motion recognition for each of the plurality of atomic actions by convolving histograms of the features of each of the plurality of atomic actions.

13 . The system in accordance with claim 12 wherein the feature extractor extracts a set of shape vectors depicting shape flow from each of the plurality of atomic actions.

14 . The system in accordance with claim 13 wherein the motion recognition module convolves histograms of the shape features of each of the plurality of atomic actions by deriving a shape descriptor by determining a histogram-of-oriented gradient of the set of shape vectors for each of the plurality of atomic actions.

15 . The system in accordance with claim 12 wherein the feature extractor extracts a set of motion vectors depicting motion flow from each of the plurality of atomic actions.

16 . The system in accordance with claim 15 wherein the motion recognition module convolves histograms of the motion features of each of the plurality of atomic actions by deriving a motion descriptor by determining a histogram-of-oriented optical flow of the set of motion vectors for each of the plurality of atomic actions.

17 . The system in accordance with claim 12 wherein the features further comprise a pose feature, and wherein the feature extractor further extracts a set of pose vectors from each of the plurality of atomic actions, and wherein the motion recognition module convolves histograms of the pose features of each of the plurality of atomic actions by deriving a pose descriptor by determining a histogram-of-oriented gradient of the set of pose vectors for each of the plurality of atomic actions.

18 . The system in accordance with claim 12 wherein the features further comprise a spatial feature, and wherein the feature extractor derives each of a set of shape, motion or pose vectors for each of two or more regions of a bounding box within each of the plurality of atomic actions, the bounding box in each of the plurality of atomic actions configured to include all of a subject pictured in the one of the plurality of atomic actions, and wherein the motion recognition module convolves histograms of each of the shape, motion or pose descriptors to generate a resultant histogram.

19 . The system in accordance with claim 12 wherein the motion recognition module normalizes the histograms of each of the plurality of atomic actions to sum up to unity.

20 . The system in accordance with claim 11 wherein the motion recognition module further comprises a bag-of-words model module for K-means clustering of all of the atomic actions to generate a distance weighted bag-of-automatic-actions model of the video sequence.

Assignees

• Agency Science Tech & Res

Inventors

• Chua Teck Wee
• Pham Nam Trung
• Leman Karianto
• Chang Richard Philippe
• Wang Yue

Classifications

• G06F18/23213

  with fixed number of clusters, e.g. K-means clustering · CPC title

• G06V40/23 Primary

  Recognition of whole body movements, e.g. for sport training · CPC title

• G06K9/6223

  Physics · mapped topic

• G06K9/4642

  Physics · mapped topic

• G06T2207/30196

  Human being; Person · CPC title