Recognition method and apparatus and mobile platform

Patent metadata

Field	Value
Publication number	US-11250248-B2
Application number	US-201916553680-A
Country	US
Kind code	B2
Filing date	Aug 28, 2019
Priority date	Feb 28, 2017
Publication date	Feb 15, 2022
Grant date	Feb 15, 2022

Abstract

The present disclosure provides a gesture recognition method. The method includes the following steps: acquiring a depth image of a user; determining a point set of a two-dimensional image indicating a palm based on a depth information of the depth image; and, determining a gesture based on the point set.

First claim

What is claimed is:

1. A gesture recognition method, comprising:
acquiring a plurality of frames of depth image of a user;
determining a point set of a two-dimensional image indicating a palm corresponding to each frame of depth image of the plurality of frames of depth image based on depth information of each frame of depth image, including:
clustering point sets of the two-dimensional image based on the depth information; and
determining the point set indicating the palm based on the clustered point sets and a distance threshold;
determining a point cloud, associated with a three-dimensional space, indicating the palm corresponding to each frame of depth image based on a coordinate correspondence between the point set and the point cloud and the point set indicating the palm; and
determining a gesture based on at least one of the point set or the point cloud, including:
obtaining trajectory information of a movement of the palm in a period of time, the trajectory information containing a sequence of movement directions of the palm during the movement of the palm, the sequence of movement directions is generated based on the at least one of the point set or the point cloud;
determining a ratio of each movement direction in the sequence of movement directions to obtain a sequence of ratios; and determining the gesture based on the sequence of ratios.

2. The method of claim 1 , wherein determining the point set of the two-dimensional image indicating the palm based on the depth information further includes:
determining a point indicating the palm corresponding to each frame of depth image of the plurality of frames of depth image on the two-dimensional image based on the depth information of each frame of depth image;
determining a point set connected with the point indicating the palm based on a depth range; and
determining the point set of the two-dimensional image indicating the palm from the connected point set.

3. The method of claim 2 , wherein determining the point set of the two-dimensional image indicating the palm from the connected point set includes:
deleting a point set indicating an arm from the connected point set; and
determining the point set indicating the palm based on a remaining point set.

4. The method of claim 3 , wherein deleting the point set indicating the arm from the connected point set and determining the point set indicating the palm based on the remaining point set includes:
obtaining a minimum circumscribed rectangle of the connected point set;
determining a plurality of distances from points in the connected point set to a designated side of the circumscribed rectangle;
determining points not meeting a distance requirement to be points indicating the arm; and
deleting the points indicating the arm.

5. The method of claim 4 , wherein the distance requirement is determined based on a side length of the minimum circumscribed rectangle.

6. The method of claim 1 , wherein determining the gesture based on the at least one of the point set or the point cloud further includes:
determining a distribution characteristic of the point set; and
determining the gesture further based on the distribution characteristic.

7. The method of claim 6 , wherein determining the distribution characteristic of the point set includes:
determining a distribution area of the point set of the two-dimensional image indicating the palm; and
determining the distribution characteristic of the point set based on the distribution area.

8. The method of claim 7 , wherein determining the distribution characteristic of the point set based on the distribution area includes:
using a polygonal area to cover the distribution area;
determining a plurality of non-overlapping areas between the polygonal area and the distribution area; and
determining the distribution characteristic of the point set based on the non-overlapping areas.

9. The method of claim 8 , wherein using the polygonal area to cover the distribution area includes:
using a convex polygonal area with a least number of sides to cover the distribution area.

10. The method of claim 8 , wherein determining the distribution characteristic of the point set based on the non-overlapping areas includes:
determining a plurality of farthest distances from the points in the non-overlapping areas to a corresponding side of the polygon; and
determining the farthest distances to be the distribution characteristic of the point set.

11. The method of claim 10 , wherein determining the distribution characteristic of the point set based on the non-overlapping areas further includes:
determining the gesture to be a first in response to the farthest distances corresponding to each of the non-overlapping areas being less than or equal to the distance threshold.

12. The method of claim 10 , wherein determining the distribution characteristic of the point set based on the non-overlapping areas further includes:
determining the gesture to be a stretched palm in response to one or more of the farthest distances corresponding to the non-overlapping areas being greater than or equal to the distance threshold.

13. The method of claim 2 , wherein determining the gesture based on the at least one of the point set or the point cloud further includes:
determining location information of the palm based on the point cloud indicating the palm to determine a sequence of location information indicating the palm.

14. The method of claim 13 , wherein determining the gesture of the palm further includes:
determining the movement directions of the palm based on the sequence of location information.

15. The method of claim 14 , wherein determining the gesture based on the sequence of ratios includes:
determining the gesture based on a combination of the ratios in the sequence of ratios.

16. The method of claim 15 , wherein determining the gesture based on the combination of the ratios includes:
inputting the combination of the ratio into a computing model; and
determining the gesture based on the computing model.

17. The method of claim 14 , wherein determining the movement directions of the palm includes:
determining a plurality of velocity directions corresponding to the location information based on the sequence of location information indicating the palm; and
determining the movement directions of the palm based on the velocity directions.

18. The method of claim 17 , wherein determining the movement directions of the palm based on the velocity directions includes:
determining the velocity directions corresponding to the sequence of location information;
determining a plurality of angles between the velocity directions and each of a plurality of predetermined directions; and
determining the movement directions based on the angles.

19. The method of claim 18 , wherein determining the movement directions based on the angles includes:
determining a first direction having a smallest angle with the velocity directions from the predetermined directions; and
determining the first direction to be the movement direction corresponding to the velocity direction.

20. A gesture recognition device, comprising:
a TOF camera; and
a processor configured to:
determine a point set of a two-dimensional image indicating a palm corresponding to each frame of depth image of a plurality of frames of depth image of a user, acquired by the TOF camera, based on depth information of each frame of depth image, including:
clustering point sets of the two

Assignees

• Sz Dji Technology Co Ltd

Inventors

• Tang Ketan
• Zhou You
• Zhou Guyue
• Guo Zhuo

Classifications

• G05D1/2435

  Extracting 3D information · CPC title

• G05D2111/64

  taken simultaneously from spaced apart sensors, e.g. stereovision · CPC title

• G05D2111/10

  Optical signals · CPC title

• G05D2109/254

  Flying platforms, e.g. multicopters · CPC title

• G05D1/2285

  using voice or gesture commands · CPC title