Generating synthetic models or virtual objects for training a deep learning network

Patent metadata

Field	Value
Publication number	US-12307366-B2
Application number	US-202318354932-A
Country	US
Kind code	B2
Filing date	Jul 19, 2023
Priority date	Jan 17, 2019
Publication date	May 20, 2025
Grant date	May 20, 2025

Abstract

In some implementations, a training platform may receive data for generating synthetic models of a body part, such as a hand. The data may include information relating to a plurality of potential poses of the hand. The training platform may generate a set of synthetic models of the hand based on the information, where each synthetic model, in the set of synthetic models, representing a respective pose of the plurality of potential poses. The training platform may derive an additional set of synthetic models based on the set of synthetic models by performing one or more processing operations with respect to at least one synthetic model in the set of synthetic models, and causing the set of synthetic models and the additional set of synthetic models to be provided to a deep learning network to train the deep learning network to perform image segmentation, object recognition, or motion recognition.

First claim

What is claimed is:

1. A method, comprising:
generating, by one or more devices, one or more synthetic models of an object based on data that identifies a permissible relative distance between a first component corresponding to a first part of the object and a second component corresponding to a second part of the object,
wherein the object is associated with a body part, and
wherein the one or more synthetic models are generated such that no portion of the first component and the second component overlap one another;

deriving, by the one or more devices, additional synthetic models based on the one or more synthetic models by one or more of:
modifying a background associated with a copy of a synthetic model, of the one or more synthetic models, of the body part,
modifying one or more parameters relating to one or more components of the copy of the synthetic model, or
adjusting a field of view of a virtual camera associated with the synthetic model; and

providing, by one or more devices, the additional synthetic models to a deep learning network to train the deep learning network to perform image segmentation, object recognition, or motion recognition.

2. The method of claim 1 , wherein the synthetic model includes one or more of:
a three-dimensional (3D) model of the body part,
a two-dimensional (2D) model of the body part,
a virtual object that includes an image of the body part, or
a data structure that stores parameters used to generate the virtual object.

3. The method of claim 1 , wherein deriving the additional synthetic models comprises:
deriving the additional synthetic models by modifying the background associated with the copy of the synthetic model.

4. The method of claim 3 , wherein deriving the additional synthetic models by modifying the background associated with the copy of the synthetic model comprises:
deriving the additional synthetic models by changing one or more of:
a color of the background associated with the copy of the synthetic model, or
a pattern of the background associated with the copy of the synthetic model, or adding noise to the background associated with the copy of the synthetic model.

5. The method of claim 3 , wherein deriving the additional synthetic models by modifying the background associated with the copy of the synthetic model comprises:
deriving the additional synthetic models by modifying the background associated with the copy of the synthetic model based on facilitating the deep learning network to be trained on a portion of an image, of the synthetic model, that includes the body part.

6. The method of claim 1 ,
wherein deriving the additional synthetic models comprises:
deriving the additional synthetic models by modifying the one or more parameters relating to the one or more components of the copy of the synthetic model, and

wherein the one or more parameters include one or more of positions, angles, color, or texture of the one or more components of the copy of the synthetic model.

7. The method of claim 1 ,
wherein deriving the additional synthetic models comprises:
deriving the additional synthetic models by adjusting the field of view of the virtual camera associated with the synthetic model, and

wherein the additional synthetic models include one or more additional synthetic models that exhibit alternative views of the synthetic model from different perspectives.

8. A system, comprising:
one or more memories; and
one or more processors, coupled to the one or more memories, configured to:
generate one or more synthetic models of an object based on data that identifies a permissible relative distance between a first component corresponding to a first part of the object and a second component corresponding to a second part of the object,
wherein the one or more synthetic models are generated such that no portion of the first component and the second component overlap one another;
derive additional synthetic models based on one or more synthetic models by one or more of:
modifying a background associated with a copy of a synthetic model of the one or more synthetic models,
modifying one or more parameters relating to one or more components of the copy of the synthetic model, or
adjusting a field of view of a virtual camera associated with the synthetic model; and

provide the additional synthetic models to a deep learning network to train the deep learning network to perform image segmentation, object recognition, or motion recognition.

9. The system of claim 8 , wherein the synthetic model includes one or more of:
a three-dimensional (3D) model of a body part,
a two-dimensional (2D) model of the body part,
a virtual object that includes an image of the body part, or
a data structure that stores parameters used to generate the virtual object.

10. The system of claim 8 , wherein the one or more processors, to derive the additional synthetic models, are configured to:
derive the additional synthetic models by modifying the background associated with the copy of the synthetic model.

11. The system of claim 10 , wherein the one or more processors, to derive the additional synthetic models by modifying the background associated with the copy of the synthetic model, are configured to:
derive the additional synthetic models by changing one or more of:
a color of the background associated with the copy of the synthetic model, or
a pattern of the background associated with the copy of the synthetic model, or remove noise from the background associated with the copy of the synthetic model.

12. The system of claim 10 , wherein the one or more processors, to derive the additional synthetic models by modifying the background associated with the copy of the synthetic model, are configured to:
derive the additional synthetic models by modifying the background associated with the copy of the synthetic model based on facilitating the deep learning network to be trained on a portion of an image, of the synthetic model, that includes a body part.

13. The system of claim 8 ,
wherein the one or more processors, to derive the additional synthetic models, are configured to:
derive the additional synthetic models by modifying the one or more parameters relating to the one or more components of the copy of the synthetic model, and

wherein the one or more parameters include one or more of positions, angles, color, or texture of the one or more components of the copy of the synthetic model.

14. The system of claim 8 ,
wherein the one or more processors, to derive the additional synthetic models, are configured to:
derive the additional synthetic models by adjusting the field of view of the virtual camera associated with the synthetic model, and

wherein the additional synthetic models include one or more additional synthetic models that exhibit alternative views of the synthetic model from different perspectives.

15. The system of claim 8 , wherein the additional synthetic models include an array of synthetic models that represent a plurality of different poses of a body part in a plurality of different fields of view.

16. The system of claim 8 , wherein no portion of the first component and the second component overlap one another based on generating the one or more synthetic models such that no portion of the first component and the second component occupy a common voxel in a three dimensional space.

17. A non-transitory computer-readable medium storing a set of instructions, the set of instructions comprising:
one or more instructions that, when executed by one o

Assignees

• Capital One Services Llc

Inventors

• Farivar Reza
• Taylor Kenneth
• Walters Austin
• Ford Iii Joseph
• Adhikari Rittika

Classifications

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06V40/107

  Static hand or arm · CPC title

• G06V10/82

  using neural networks · CPC title

• G06V10/774

  Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title