Motion visualization system and the motion visualization method

Patent metadata

Field	Value
Publication number	US-12456332-B2
Application number	US-202118031263-A
Country	US
Kind code	B2
Filing date	Oct 4, 2021
Priority date	Oct 15, 2020
Publication date	Oct 28, 2025
Grant date	Oct 28, 2025

Abstract

A motion visualization system 1 includes a skeleton recognition program 2 obtaining joint coordinates of a target person, a pitch extraction program 6 extracting a walk cycle of the target person based on the joint coordinates, and a storage unit 106 storing a skeleton normalization program 3 transforming coordinates of the joint coordinates in a reference coordinate system where a direction in which a predetermined portion of the target person moves is set as an axis of a traveling direction in one walk cycle extracted and transforming the joint coordinates based on information on the target person.

First claim

The invention claimed is:

1. A motion visualization system comprising:
a camera having a z-axis, which is a direction along an optical axis of a lens of the camera;
a controller coupled to the camera; and
a memory storing instructions that when executed by the controller, configures the controller to:
obtain skeleton coordinate information of a target person for each of a plurality of frames captured by the camera,
extract a walk cycle in a traveling direction of the target person based on the skeleton coordinate information,
determine information a predetermined portion of the target person based on the skeleton coordinate information of one of the plurality frames,
transform the skeleton coordinate information in a reference coordinate system where a direction in which the predetermined portion of the target person moves is set as an axis of the traveling direction in one walk cycle extracted by the cycle extraction unit, the traveling direction being different than the z-axis of the camera, and
transform the skeleton coordinate information based on the information of the predetermined portion of the target person, as reference data, so that the z-axis and the traveling direction overlap with one another.

2. The motion visualization system according to claim 1 ,
wherein the information of the target person is a height, a stride length, or a length of the predetermined portion of a body of the target person.

3. The motion visualization system according to claim 2 , further comprising:
a display coupled to the processor,
wherein the processor is configured to:
calculate features of a walk of the target person based on the transformed skeleton coordinate information, and
display the features of the walk of the target person on the display.

4. The motion visualization system according to claim 3 ,
wherein the processor is configured to display a movement trajectory of the predetermined portion of the target person accompanying the walk.

5. The motion visualization system according to claim 4 ,
wherein the processor is configured to display changes in rotation of the predetermined portion of the target person in the walk cycle and a wobble of the target person in the one walk cycle.

6. A motion visualization method comprising:
obtaining, by a camera having a z-axis, which is a direction along an optical axis of a lens of the camera, a plurality of frames of a target person walking;
a skeleton recognition process of obtaining changes in skeleton coordinates of the target person for each of the plurality of frames;
a cycle extraction process of extracting a walk cycle in a travelling direction of the target person based on the change in the skeleton coordinates;
determining information a predetermined portion of the target person based on the skeleton coordinate information of one of the plurality frames;
a coordinate transformation process of transforming the skeleton coordinates in a reference coordinate system in which a direction in which the predetermined portion of the target person moves is set as an axis of the traveling direction in one walk cycle extracted in the cycle extraction process, the traveling direction being different than the z-axis of the camera; and
a skeleton normalization process of transforming the skeleton coordinates based on the information of the predetermined portion of the target person, as reference data, so that the z-axis and the traveling direction overlap with one another.

7. The motion visualization method according to claim 6 ,
wherein the information of the target person is a height, a stride length or a length of a predetermined portion of the target person.

8. The motion visualization method according to claim 7 , further comprising:
a walk feature calculation process of calculating features of a walk of the target person based on the transformed skeleton coordinates obtained in the coordinate transformation process and the skeleton normalization process; and
a display process of displaying the features of the walk of the target person.

9. The motion visualization method according to claim 8 ,
wherein, in the display process, a movement trajectory of the predetermined portion of the target person accompanying the walk is displayed.

10. The motion visualization method according to claim 9 ,
wherein, in the display process, changes in rotation of the predetermined portion of the target person in the walk cycle and wobble of the target person in the walk cycle are displayed.

Assignees

• Hitachi High Tech Corp

Inventors

• Tanaka Takeshi
• Fukui Daisuke
• Nakagawa Hiromitsu
• Toyomura Takashi

Classifications

• G06T3/60

  Rotation of whole images or parts thereof · CPC title

• A61B5/112

  Gait analysis · CPC title

• G06V40/25 Primary

  Recognition of walking or running movements, e.g. gait recognition · CPC title

• A61B5/1122

  of movement trajectories · CPC title

• A61B5/1128

  using image analysis (A61B5/1127 takes precedence) · CPC title