Part recognition and damage characterization using deep learning

Patent metadata

Field	Value
Publication number	US-10460431-B2
Application number	US-201815871526-A
Country	US
Kind code	B2
Filing date	Jan 15, 2018
Priority date	Jan 15, 2018
Publication date	Oct 29, 2019
Grant date	Oct 29, 2019

Abstract

According to one embodiment, a method of identifying a part of a conveyance system is provided. The method comprising: capturing an image of a part of a conveyance system using a camera; classifying the part of the conveyance system using supervised learning; and displaying a classification of the part of the part on a mobile computing device.

First claim

What is claimed is:

1. A method of identifying a part of a conveyance system, the method comprising:
capturing real data of a conveyance system;
capturing synthetic data of the conveyance system;
performing domain adaption to bridge between the synthetic data and the real data;
determining a supervised deep learning model in response to the real data, the synthetic data, and the bridge adaption;
determining an unsupervised deep learning model in response to the real data, the synthetic data, and the bridge adaption;
capturing an image of a part of the conveyance system using a camera;
classifying the part of the conveyance system using the image and at least one of the supervised deep learning model and the unsupervised deep learning model;
displaying a classification of the part on a mobile computing device;
determining a reconstruction error in response to the classification of the part and the nearest neighbor; and
determining an amount of damage to the part in response to the reconstruction error using recurrent architectures and feature extraction modules.

2. The method of claim 1 , wherein:
supervised deep learning model further includes deep learning models.

3. The method of claim 1 , wherein classifying further includes:
determining a classification of the part in response to the image and the supervised deep learning model.

4. The method of claim 1 , wherein classifying further includes:
extracting a low dimension representation of the image using the unsupervised deep learning model;
comparing the low dimensional representation of the image to at least one of renders of the synthetic data and the real data; and
determining a nearest neighbor for a classification of the part.

5. The method of claim 1 , wherein:
the low dimension representation of the images are extracted utilizing unsupervised feature extraction.

6. The method of claim 1 , wherein:
the sensor is operably included within the mobile computing device.

7. The method of claim 1 , wherein:
the sensor is operably attached to the mobile computing device.

8. A computer program product tangibly embodied on a computer readable medium, the computer program product including instructions that, when executed by a processor, cause the processor to perform operations comprising:
capturing real data of a conveyance system;
capturing synthetic data of the conveyance system;
performing domain adaption to bridge between the synthetic data and the real data;
determining a supervised deep learning model in response to the real data, the synthetic data, and the bridge adaption;
determining an unsupervised deep learning model in response to the real data, the synthetic data, and the bridge adaption;
capturing an image of a part of the conveyance system using a sensor;
classifying the part of the conveyance system using the image and at least one of the supervised deep learning model and the unsupervised deep learning model;
displaying a classification of the part on a mobile computing device;
determining a reconstruction error in response to the classification of the part and the nearest neighbor; and
determining an amount of damage to the part in response to the reconstruction error using recurrent architectures and feature extraction modules.

9. The computer program product of claim 8 , wherein:
supervised deep learning model further includes deep learning model.

10. The computer program product of claim 8 , wherein classifying further includes:
determining a classification of the part in response to the image and the supervised deep learning model.

11. The computer program product of claim 8 , wherein classifying further includes:
extracting a low dimension representation of the image using the unsupervised deep learning model;
comparing the low dimensional representation of the image to at least one of renders of the synthetic data and the real data; and
determining a nearest neighbor for a classification of the part.

12. The computer program product of claim 8 , wherein:
the low dimension representation of the images are extracted utilizing unsupervised feature extraction.

13. The computer program product of claim 8 , wherein:
the sensor is operably included within the mobile computing device.

14. The computer program product of claim 8 , wherein:
the sensor is operably attached to the mobile computing device.

Assignees

• Otis Elevator Co

Inventors

• Bertuccelli Luca F
• Reddy Kishore K
• Lore Kin Gwn

Classifications

• G06V20/64

  Three-dimensional [3D] objects · CPC title

• G06V10/993

  Evaluation of the quality of the acquired pattern · CPC title

• G06V10/82

  using neural networks · CPC title

• G06V10/454

  Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN] · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title