Motion recognition device and motion recognition method

Patent metadata

Field	Value
Publication number	US-11176359-B2
Application number	US-201916362701-A
Country	US
Kind code	B2
Filing date	Mar 25, 2019
Priority date	Oct 11, 2016
Publication date	Nov 16, 2021
Grant date	Nov 16, 2021

Abstract

A motion recognition device includes a memory, and a processor coupled to the memory and configured to classify a plurality of frames including positional information of a feature point that corresponds to a predetermined part or a joint part of a body of a subject into a plurality of groups in time series by segmenting the plurality of frames in time series based on a position of the predetermined part of the body of the subject, identify a type of a basic motion that corresponds to the group, based on movement of the feature point included in a consecutive frame, for each group, and evaluate a skill and a difficulty level of a motion performed by the subject based on an order of each type of the basic motion that corresponds to a group which is consecutive in time series.

First claim

What is claimed is:

1. A motion recognition device comprising:
a memory; and
a processor coupled to the memory and configured to:
classify a plurality of frames including positional information of a feature point that corresponds to a predetermined part or a joint part of a body of a subject into a plurality of groups in time series by segmenting the plurality of frames in time series based on a position of the predetermined part of the body of the subject;
identify, for a respective group, a type of a basic motion that corresponds to the respective group, based on movement of the feature point included in a consecutive frame; and
evaluate, for the respective group, a skill and a difficulty level of the movement indicated by the respective group, by using a criteria identified by both of a first basic motion type and a second basic motion type, the first basic motion type being the type of the basic motion identified for the respective group, the second basic motion type being the type of the basic motion identified for a group one before the respective group,
wherein the evaluating is configured to evaluate, for each group, a score of the respective group by comparing features of the feature points included in the frame of the respective group with evaluation criteria, the evaluation criterion being information indicating a respective score corresponding to each feature indicated by feature points to be included in frames.

2. The motion recognition device according to claim 1 , the processor further configured to:
in response to performing of the evaluating, upgrade a value corresponding to the evaluated difficulty level of the motion performed by the subject based on the order of the basic motion that corresponds to the group which is consecutive in time series and a rotation angle of the body of the subject.

3. The motion recognition device according to claim 1 , the processor further configured to:
repeatedly execute a process of determining a frame that serves as a segment point based on a direction of a normal vector of a plane that passes through both shoulders and a back of the subject and the positions of both hands of the subject, and classifies the plurality of frames sandwiched between the frames that serve as the segment points into one group.

4. The motion recognition device according to claim 1 , the processor further configured to:
repeatedly execute a process of determining a frame that serves as a segment point based on a direction of a normal vector of a plane that passes through both hip joints and a back of the subject and the positions of both hands of the subject, and classifies the plurality of frames sandwiched between the frames that serve as the segment points into one group.

5. A motion recognition method of which a process is executed by a computer, the process comprising:
classifying a plurality of frames including positional information of a feature point that corresponds to a predetermined part or a joint part of a body of a subject into a plurality of groups in time series by segmenting the plurality of frames in time series based on a position of the predetermined part of the body of the subject,
identifying, for a respective group, a type of a basic motion that corresponds to the respective group, based on movement of the feature point included in a consecutive frame; and
evaluating, for the respective group, a skill and a difficulty level of the movement indicated by the respective group, by using a criteria identified by both of a first basic motion type and a second basic motion type, the first basic motion type being the type of the basic motion identified for the respective group, the second basic motion type being the type of the basic motion identified for a group one before the respective group,
wherein the evaluating is configured to evaluate, for each group, a score of the respective group by comparing features of the feature points included in the frame of the respective group with evaluation criteria, the evaluation criterion being information indicating a respective score corresponding to each feature indicated by feature points to be included in frames.

6. The motion recognition method according to claim 5 , wherein
in response to performing of the evaluating, upgrading a value corresponding to the evaluated difficulty level of the motion performed by the subject based on the order of the basic motion that corresponds to the group which is consecutive in time series and a rotation angle of the body of the subject.

7. The motion recognition method according to claim 5 , wherein
in the process of classifying, a process of determining a frame that serves as a segment point is repeatedly executed based on a direction of a normal vector of a plane that passes through both shoulders and a back of the subject and the positions of both hands of the subject, and the plurality of frames sandwiched between the frames that serve as the segment points are classified into one group.

8. The motion recognition method according to claim 5 , wherein
in the process of classifying, a process of determining a frame that serves as a segment point is repeatedly executed based on a direction of a normal vector of a plane that passes through both hip joints and a back of the subject and the positions of both hands of the subject, and the plurality of frames sandwiched between the frames that serve as the segment points are classified into one group.

9. A non-transitory computer-readable storage medium storing a motion recognition program that causes a computer to execute a process, the process comprising:
classifying a plurality of frames including positional information of a feature point that corresponds to a predetermined part or a joint part of a body of a subject into a plurality of groups in time series by segmenting the plurality of frames in time series based on a position of the predetermined part of the body of the subject;
identifying, for a respective group, a type of a basic motion that corresponds to the respective group, based on movement of the feature point included in a consecutive frame; and
evaluating, for the respective group, a skill and a difficulty level of the movement indicated by the respective group, by using a criteria identified by both of a first basic motion type and a second basic motion type, the first basic motion type being the type of the basic motion identified for the respective group, the second basic motion type being the type of the basic motion identified for a group one before the respective group,
wherein the evaluating is configured to evaluate, for each group, a score of the respective group by comparing features of the feature points included in the frame of the respective group with evaluation criteria, the evaluation criterion being information indicating a respective score corresponding to each feature indicated by feature points to be included in frames.

10. The non-transitory computer-readable storage medium according to claim 9 , the process further comprising:
in response to performing of the evaluating, upgrading a value corresponding to the evaluated difficulty level of the motion performed by the subject based on the order of the basic motion that corresponds to the group which is consecutive in time series and a rotation angle of the body of the subject.

11. The non-transitory computer-readable storage medium according to claim 9 , wherein
in the process of classifying, a process of determining a frame that serves as a segment point is repeatedly executed based on a direction of a normal vector of a plane that passes through both shoulders and a back of the subject and the positions of both hands

Assignees

• Fujitsu Ltd

Inventors

• Yabuki Akihiko
• Nakamura Yoshihiko
• Takano Wataru

Classifications

• A61B5/1128 Primary

  using image analysis (A61B5/1127 takes precedence) · CPC title

• G06V40/23

  Recognition of whole body movements, e.g. for sport training · CPC title

• G06F18/24

  Classification techniques · CPC title

• G06V20/47

  Detecting features for summarising video content · CPC title

• G06V20/49

  Segmenting video sequences, i.e. computational techniques such as parsing or cutting the sequence, low-level clustering or determining units such as shots or scenes · CPC title