Method for human motion analysis, apparatus for human motion analysis, device and storage medium

Patent metadata

Field	Value
Publication number	US-10970528-B2
Application number	US-201916459890-A
Country	US
Kind code	B2
Filing date	Jul 2, 2019
Priority date	Jul 3, 2018
Publication date	Apr 6, 2021
Grant date	Apr 6, 2021

Abstract

A method for human motion analysis, an apparatus for human motion analysis, a device, and a storage medium. The method includes: acquiring image information captured by a number of photographing devices, where at least one of the number of photographing devices is disposed above a shelf; performing human tracking according to the image information captured by the plurality of photographing devices, and determining position information in space of at least one human body and identification information of the at least one human body; acquiring, according to the position information in space of a target human body of the at least one human body, a target image captured by the photographing device above a shelf corresponding to the position information; and recognizing an action of the target human body according to the target image and detection data of a non-visual sensor corresponding to the position information.

First claim

What is claimed is:

1. A method for human motion analysis, comprising:
acquiring image information captured by a plurality of photographing devices, wherein at least one of the plurality of photographing devices is disposed above a shelf;
performing human tracking according to the image information captured by the plurality of photographing devices, and determining position information in space of at least one human body and identification information of the at least one human body;
acquiring, according to the position information in space of a target human body of the at least one human body, a target image captured by the photographing device above a shelf corresponding to the position information;
acquiring a key point of the target human body in the target image, wherein the key point of the target human body comprises a key point of a hand of the target human body; and
recognizing an action of the target human body according to the key point of the hand of the target human body and detection data of a non-visual sensor corresponding to the position information;
wherein the non-visual sensor comprises a light curtain sensor which is disposed on a beam of the shelf facing a customer and used to detect an occurrence time of the action and an occurrence position of the action, and an infrared sensor that is disposed on a layer plate of the shelf; and
the recognizing an action of the target human body according to the key point of the hand of the target human body and detection data of a non-visual sensor corresponding to the position information comprises:
acquiring, according to the occurrence position of the action detected by the light curtain sensor, an intensity change of infrared radiation of a commodity detected by the infrared sensor corresponding to the occurrence position of the action at the occurrence time of the action detected by the light curtain sensor; and
recognizing, according to the intensity change of infrared radiation of the commodity detected by the infrared sensor corresponding to the occurrence position of the action and the key point of the hand of the target human body, the action of the target human body.

2. The method according to claim 1 , wherein the recognizing an action of the target human body comprises:
recognizing an action of picking up a commodity of the target human body.

3. An apparatus for human motion analysis, comprising: a processor and a non-transitory computer-readable medium for storing program codes, which, when executed by the processor, cause the processor to:
acquire image information captured by a plurality of photographing devices, wherein at least one of the plurality of photographing devices is disposed above a shelf;
perform human tracking according to the image information captured by the plurality of photographing devices, and determine position information in space of at least one human body and identification information of the at least one human body;
acquire, according to the position information in space of a target human body of the at least one human body, a target image captured by the photographing device above a shelf corresponding to the position information;
acquire a key point of the target human body in the target image, wherein the key point of the target human body comprises a key point of a hand of the target human body; and
recognize an action of the target human body according to the key point of the hand of the target human body and detection data of a non-visual sensor corresponding to the position information;
wherein the non-visual sensor comprises a light curtain sensor which is disposed on a beam of the shelf facing a customer and used to detect an occurrence time of the action and an occurrence position of the action, and an infrared sensor that is disposed on a layer plate of the shelf; and
the program codes further cause the processor to:
acquire, according to the occurrence position of the action detected by the light curtain sensor, an intensity change of infrared radiation of a commodity detected by the infrared sensor corresponding to the occurrence position of the action at the occurrence time of the action detected by the light curtain sensor; and
recognize, according to the intensity change of infrared radiation of the commodity detected by the infrared sensor corresponding to the occurrence position of the action and the key point of the hand of the target human body, the action of the target human body.

4. The apparatus according to claim 3 , wherein the program codes further cause the processor to:
recognize an action of picking up a commodity of the target human body.

5. The method of claim 1 , wherein the human motion analysis is performed by a device comprising:
a memory;
a processor; and
a computer program;
wherein the computer program is stored in the memory and is configured to be executed by the processor.

6. A non-transitory computer readable storage medium, having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the method according to claim 1 .

7. The method according to claim 1 , wherein the non-vision sensor further comprises a gravity sensor that is disposed on the layer plate of the shelf;
the method further comprises:
acquiring, according to the occurrence position of the action detected by the light curtain sensor, a gravity change detected by the gravity sensor corresponding to the occurrence position of the action at the occurrence time of the action detected by the light curtain sensor; and
the recognizing, according to the intensity change of infrared radiation of the commodity detected by the infrared sensor corresponding to the occurrence position of the action and the key point of the hand of the target human body, the action of the target human body comprises:
recognizing, according to the intensity change of infrared radiation of the commodity detected by the infrared sensor corresponding to the occurrence position of the action, the gravity change detected by the gravity sensor corresponding to the occurrence position of the action, and the key point of the hand of the target human body, the action of the target human body.

8. The method according to claim 1 , wherein the recognizing an action of the target human body comprises:
recognizing an action of putting down a commodity of the target human body.

9. The apparatus according to claim 3 , wherein the non-vision sensor further comprises a gravity sensor that is disposed on the layer plate of the shelf;
the program codes further cause the processor to:
acquire, according to the occurrence position of the action detected by the light curtain sensor, a gravity change detected by the gravity sensor corresponding to the occurrence position of the action at the occurrence time of the action detected by the light curtain sensor; and
recognize, according to the intensity change of infrared radiation of the commodity detected by the infrared sensor corresponding to the occurrence position of the action, the gravity change detected by the gravity sensor corresponding to the occurrence position of the action, and the key point of the hand of the target human body, the action of the target human body.

10. The apparatus according to claim 3 , wherein the program codes further cause the processor to:
recognize an action of putting down the commodity of the target human body.

Assignees

• Baidu online network technology beijing co ltd

Inventors

• Wang Jian
• Li Xubin
• Kang Le
• Liu Zeyu
• Chi Zhizhen
• Zhang Chengyue
• Liu Xiao
• Sun Hao
• Wen Shilei
• Bao Yingze
• Chen Mingyu
• Ding Errui

Classifications

• G06V10/145

  Illumination specially adapted for pattern recognition, e.g. using gratings · CPC title

• G06V40/28

  Recognition of hand or arm movements, e.g. recognition of deaf sign language (static hand signs G06V40/113) · CPC title

• G06T7/292 Primary

  Multi-camera tracking · CPC title

• G06V40/20 Primary

  Movements or behaviour, e.g. gesture recognition (recognition of facial expressions G06V40/16) · CPC title

• G06Q30/06

  Buying, selling or leasing transactions · CPC title