Using photogrammetry to aid identification and assembly of product parts

Patent metadata

Field	Value
Publication number	US-2019033072-A1
Application number	US-201816151371-A
Country	US
Kind code	A1
Filing date	Oct 4, 2018
Priority date	Oct 2, 2015
Publication date	Jan 31, 2019
Grant date	—

Abstract

A user may be aided in modifying a product that is an assemblage of parts. This aid may involve a processor obtaining images of a target part captured by the user on a mobile device camera. The processor may compare, based on the captured images and a plurality of images of identified parts, the target part to the identified parts. Based on the comparison, the processor may determine an identity of the target part. This aid may also involve a processor obtaining images of a first configuration of a partial assembly of the product captured by a mobile device camera. The processor may compare, based on the captured images, the first configuration to a correct configuration of the partial assembly. Based on the comparison, the processor may determine that the first configuration does not match the correct configuration and may notify the user accordingly.

First claim

What is claimed is:

1 . A computer-implemented method for aiding a user in modifying a product, wherein the product is an assemblage of a plurality of parts, the method comprising:
obtaining, by a processor, a plurality of images of a target part of the product captured by the user on a camera; generating, by the processor and based on the plurality of images of the target part, a three-dimensional model of the target part; comparing, by the processor, the target part to a plurality of identified parts by overlaying a three-dimensional model of each identified part onto the three-dimensional model of the target part; and determining, by the processor and based on the comparing, an identity of the target part.

2 . The method of claim 1 , further comprising:
notifying the user of the identity of the target part.

3 . The method of claim 1 , further comprising:
obtaining an identifier of the product, wherein the product identifier is selected by the user; and selecting, prior to the comparing and by the processor, the three-dimensional models of the identified parts from a database based on the obtained product identifier.

4 . The method of claim 3 , wherein the database includes images of parts sorted by product identifiers of products in which the parts are included.

5 . The method of claim 1 , further comprising:
providing the user with the identity of the target part and a video showing how to install the target part within the assemblage.

6 . A computer program product for aiding a user in obtaining a suitable replacement for a broken part included an assembly of a product, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to cause the processor to perform a method comprising:
obtaining, by the processor, a plurality of images of the broken part captured by a camera; generating, by the processor and based on the plurality of images of the broken part, a three-dimensional model of the broken part; comparing the three-dimensional model of the broken part to the three-dimensional model of a potential replacement part by overlaying the models on top of each other; identifying, by the processor and based on the comparing, that functional portions of the broken part that are designed to be points of connection with other parts of the product assembly are the same as corresponding functional portions of the potential replacement part; determining, by the processor, based on the identification that the corresponding functional portions are the same that the potential replacement part is a suitable replacement for the broken part; and notifying, by the processor, the user that the potential replacement part is a suitable replacement for the broken part.

7 . A system for aiding a user in identifying an unidentified part of a plurality of parts to be included in an assembly of a product, the system comprising:
a memory; a processor in communication with the memory, the processor configured to perform a method comprising: obtaining a plurality of images of a target part of the product captured by the user on a camera; generating, based on the plurality of images of the target part, a three-dimensional model of the target part; comparing the target part to a plurality of identified parts by overlaying a three-dimensional model of each identified part onto the three-dimensional model of the target part; and determining, based on the comparing, an identity of the target part.

8 . The system of claim 7 , wherein the method further comprises:
notifying the user of the identity of the target part.

9 . The system of claim 7 , wherein the method further comprises:
obtaining an identifier of the product, wherein the product identifier is selected by the user; and selecting, prior to the comparing and by the processor, the three-dimensional models of the identified parts from a database based on the obtained product identifier.

10 . The system of claim 9 , wherein the database includes images of parts sorted by product identifiers of products in which the parts are included.

11 . The system of claim 7 , wherein the method further comprises:
providing the user with the identity of the target part and a video showing how to install the target part within the assemblage.

Assignees

• IBM

Inventors

• Bender Michael
• Childress Rhonda L
• Kumhyr David B
• Spisak Michael J

Classifications

• G06Q30/016

  After-sales · CPC title

• G06T17/00

  Three-dimensional [3D] modelling for computer graphics · CPC title

• G06F30/00

  Computer-aided design [CAD] · CPC title

• G01C11/06 Primary

  by comparison of two or more pictures of the same area · CPC title

• G06K2209/40

  Physics · mapped topic