Generating synthetic models or virtual objects for training a deep learning network

Patent metadata

Field	Value
Publication number	US-10430692-B1
Application number	US-201916250719-A
Country	US
Kind code	B1
Filing date	Jan 17, 2019
Priority date	Jan 17, 2019
Publication date	Oct 1, 2019
Grant date	Oct 1, 2019

Abstract

In some implementations, a training platform may receive data for generating synthetic models of a body part, such as a hand. The data may include information relating to a plurality of potential poses of the hand. The training platform may generate a set of synthetic models of the hand based on the information, where each synthetic model, in the set of synthetic models, representing a respective pose of the plurality of potential poses. The training platform may derive an additional set of synthetic models based on the set of synthetic models by performing one or more processing operations with respect to at least one synthetic model in the set of synthetic models, and causing the set of synthetic models and the additional set of synthetic models to be provided to a deep learning network to train the deep learning network to perform image segmentation, object recognition, or motion recognition.

First claim

What is claimed is:

1. A method, comprising:
receiving, by a training platform, data for generating synthetic models of a body part,
the body part including a hand, and
the data including information relating to a plurality of potential poses of the hand;

generating, by the training platform, a set of synthetic models of the hand based on the information,
a synthetic model, in the set of synthetic models, representing a pose of the plurality of potential poses,
wherein generating the set of synthetic models includes:
generating the synthetic model such that no portions of two of a first component of the synthetic model, a second component of the synthetic model, a third component of the synthetic model, a fourth component of the synthetic model, or a fifth component of the synthetic model occupy a common voxel,

wherein the first component corresponds to a first finger of the hand,
wherein the second component corresponds to a second finger of the hand,
wherein the third component, corresponding to a third finger of the hand,
wherein the fourth component, corresponding to a fourth finger of the hand, and
wherein the fifth component, corresponding to a fifth finger of the hand;

deriving, by the training platform and after generating the set of synthetic models, an additional set of synthetic models based on the set of synthetic models,
the deriving including:
performing one or more processing operations with respect to at least one synthetic model in the set of synthetic models; and

causing, by the training platform, the set of synthetic models and the additional set of synthetic models to be provided to a deep learning network to train the deep learning network to perform image segmentation, object recognition, or motion recognition.

2. The method of claim 1 , wherein generating the set of synthetic models includes:
randomly generating the set of synthetic models based on the information.

3. The method of claim 1 , wherein the positions, that any two components of the first component, the second component, the third component, the fourth component, the fifth component, or a sixth component, corresponding to a palm of the hand, are permitted to assume, are different.

4. The method of claim 1 , wherein the information identifies ranges of permissible angular positions relating to one or more joints associated with the first component, the second component, the third component, the fourth component, the fifth component, or a sixth component corresponding to a palm of the hand.

5. The method of claim 1 , wherein the information identifies permissible relative positions between any two components of the first component, the second component, the third component, the fourth component, the fifth component, or a sixth component corresponding to a palm of the hand.

6. The method of claim 1 , wherein the data further includes additional information that identifies a range of permissible sizes of the first component.

7. The method of claim 1 , wherein the information identifies at least one of:
impermissible relative position between any two components of the first component, the second component, the third component, the fourth component, or the fifth component, or
ranges of impermissible size of the first component, the second component, the third component, the fourth component, or the fifth component.

8. A device, comprising:
one or more memories; and
one or more processors, communicatively coupled to the one or more memories, configured to:
receive data for generating synthetic models of a hand,
the data including information relating to a plurality of potential poses of the hand;

generate a set of synthetic models of the hand based on the information,
a synthetic model, in the set of synthetic models, representing a pose of the plurality of potential poses,
wherein the one or more processors, when generating the set of synthetic models, are configured to:
generate the synthetic model such that no portions of two of a first component of the synthetic model, a second component of the synthetic model, a third component of the synthetic model, a fourth component of the synthetic model, or a fifth component of the synthetic model occupy a common voxel;

wherein the first component corresponds to a first finger of the hand,
wherein the second component corresponds to a second finger of the hand,
wherein the third component, corresponding to a third finger of the hand,
wherein the fourth component, corresponding to a fourth finger of the hand, and
wherein the fifth component, corresponding to a fifth finger of the hand;

derive, after generating the set of synthetic models, an additional set of synthetic models based on the set of synthetic models,
wherein the one or more processors, when deriving the additional set of synthetic models, are configured to:
determine one or more adjustments to one or more synthetic models in the set of synthetic models, and

generate the additional set of synthetic models based on the one or more adjustments; and
provide the set of synthetic models and the additional set of synthetic models to a deep learning network to train the deep learning network to perform image segmentation, object recognition, or motion recognition.

9. The device of claim 8 , wherein each synthetic model, in the set of synthetic models, is a three-dimensional model.

10. The device of claim 8 , wherein the information identifies an additional component corresponding to a wrist of the hand.

11. The device of claim 8 , wherein the information identifies restrictions on angles relating to one or more joints associated with the hand.

12. The device of claim 8 , wherein the plurality of potential poses is based on natural limits associated with an anatomy of a human hand.

13. The device of claim 8 , wherein the hand includes a human hand.

14. The device of claim 8 , wherein the information identifies at least one of:
impermissible relative position between any two components of the first component, the second component, the third component, the fourth component, or the fifth component, or
ranges of impermissible size of the first component, the second component, the third component, the fourth component, or the fifth component.

15. A non-transitory computer-readable medium comprising instructions, the instructions comprising:
one or more instructions that, when executed by one or more processors of a device, cause the one or more processors to:
receive data for generating synthetic models of a body part,
the data including information relating to a plurality of potential poses of the body part;

derive a first set of synthetic models of the body part based on the information,
a synthetic model, in the first set of synthetic models, representing a pose of the plurality of potential poses of the body part,
wherein the one or more instructions, that cause the one or more processors to derive the first set of synthetic models, cause the one or more processors to:
derive the first set of synthetic models such that no portions of two of a first component of the synthetic model, a second component of the synthetic model, a third component of the synthetic model, a fourth component of the synthetic model, or a fifth component of the synthetic model occupy a common voxel,

wherein the first component corresponds to a first finger of the body part,
wherein the second component corresponds to a second finger of the body part,
wherein the third component, corresponding to a third finger of the body part,
wherein the fourth component, corresponding to a fourth finge

Assignees

• Capital One Services Llc

Inventors

• Farivar Reza
• Taylor Kenneth
• Walters Austin
• Ford Iii Joseph
• Adhikari Rittika

Classifications

• G06V40/107

  Static hand or arm · CPC title

• G06V10/774

  Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title

• G06V10/82

  using neural networks · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title

• G06F18/217

  Validation; Performance evaluation; Active pattern learning techniques · CPC title