Wire harness assembly

Patent metadata

Field	Value
Publication number	US-9589390-B2
Application number	US-201514711294-A
Country	US
Kind code	B2
Filing date	May 13, 2015
Priority date	May 13, 2015
Publication date	Mar 7, 2017
Grant date	Mar 7, 2017

Abstract

A method and apparatus for an augmented reality vision system. A first input selecting a branch is received. Task information associated with a wire harness containing the branch is retrieved based on the first input. A second input requesting information related to inserting a wire is received. A plug map of a plug of the branch of the wire harness is generated based on the second input and using the task information associated with the wire harness. The plug map is displayed on a display system. A location of a selected wire is indicated in the plug map.

First claim

What is claimed is:

1. An augmented reality vision system for a wire harness assembly environment, the augmented reality vision system comprising:
a display system having a viewing area configured to display an image, wherein the display system is head-mounted; and
a processor running a mobile application to provide data regarding the wire harness, in which the mobile application receives a first input selecting a branch, retrieves task information associated with the wire harness containing the branch based on the first input, receives a second input requesting information related to inserting a wire, generates a plug map of a plug of the branch of the wire harness based on the second input and using the task information associated with the wire harness, displays the plug map on the display system, and indicates a location of a selected wire in the plug map, wherein the mobile application is further configured to generate video data of a combination of a display of the plug map on the display system and live video feed of the wire harness, and wherein the mobile application is further configured to provide the video data to a second display system viewable by a second person other than a first person forming the wire harness.

2. The augmented reality vision system of claim 1 further comprising:
a computer system configured to send the task information regarding the wire harness to the display system to display on the display system.

3. The augmented reality vision system of claim 2 , wherein the camera system is configured to generate image data related to a number of tasks performed in the wire harness assembly environment.

4. The augmented reality vision system of claim 2 , wherein the camera system is configured to capture an image of an indicator of a shop order.

5. The augmented reality vision system of claim 1 , wherein the mobile application is further configured to generate a record of operations of tasks performed while forming the wire assembly.

6. The augmented reality vision system of claim 1 further comprising:
an input device associated with the display system, and is configured to receive the first input, wherein the input device is a touch pad.

7. The augmented reality vision system of claim 1 , wherein the mobile application uses additional input from a camera system to verify that the wire has been inserted, and wherein the mobile application additionally prevents display of a subsequent task to be performed with respect to the wire assembly until inserting the wire has been verified as being completed correctly.

8. The augmented reality vision system of claim 1 , wherein the mobile application further provides, simultaneously with a display of the plug map on the display system, a second display of instructions and checklists associated with forming the wire harness on the display system.

9. An augmented reality vision system for a wire harness manufacturing environment, the augmented reality vision system comprising:
a display system having a viewing area that is substantially transparent, wherein the viewing area is configured to display an image, wherein the display system takes the form of augmented reality glasses;
an input device associated with the display system, wherein the input device is configured to receive at least one of a first input or a second input;
a camera system associated with the display system; and
a processor associated with the display system and running a mobile application to provide data regarding the wire harness in which the mobile application receives the first input selecting a branch, retrieves task information associated with the wire harness containing the branch based on the first input, receives the second input requesting information related to inserting a wire, generates a plug map of a plug of the branch of the wire harness based on the second input and using the task information associated with the wire harness, displays the plug map on the display system, and indicates a location of a selected wire in the plug map, and wherein the mobile application is further configured to generate a record of operations of tasks performed while forming the wire assembly.

10. A method of displaying an image using an augmented reality vision system, the method comprising:
receiving a first input selecting a branch;
retrieving task information associated with a task regarding a wire harness containing the branch based on the first input;
receiving a second input requesting information related to inserting a wire;
generating a plug map of a plug of the branch of the wire harness based on the second input and using the task information associated with the wire harness;
displaying the plug map on a display system;
indicating a location of a selected wire in the plug map;
using the camera system to take a second image of the wire harness after the first task is performed;
comparing the first image to the second image to form a comparison;
verifying that the first task was completed correctly based on the comparison to form a verification;
responsive to the verification indicating successful completion of the first task, displaying an indication that the first task was completed successfully;
responsive to the verification indicating unsuccessful completion of the first task, displaying an alert that the first task was not completed successfully; and
preventing display of a second task to be performed with respect to the wire assembly until the first task is completed successfully.

11. The method of claim 10 , wherein the augmented reality vision system comprises augmented reality glasses, and wherein retrieving the task information is performed by retrieving the task information from a data storage that is part of the augmented reality vision system.

12. The method of claim 11 further comprising:
indicating the selected wire has been completed in response to receiving a third input; and
indicating a location of a next wire in the number of wires after indicating the selected wire has been completed.

13. The method of claim 10 , further comprising:
receiving, from the second person via the communication device, assistance feedback that is based on the video data, the assistance feedback communicating assistance in forming the wire harness by the first person.

14. The method of claim 10 further comprising:
displaying strip and crimp information related to the selected wire prior to displaying the plug map.

15. The method of claim 14 , wherein the strip and crimp information includes at least one of a wire type, a wire length, or a tool type.

16. The method of claim 10 further comprising:
creating a recommended order of installation of the wires into the plug of the branch of the wire harness based on a knowledge base.

17. The method of claim 10 , further comprising:
generating, by the augmented reality vision system, video data of a combination of a display of the plug map on the display system and live video feed of the wire harness; and
communicating, via a communication device that is part of the augmented reality vision system, the video data to a second display system viewable by a second person other than a first person forming the wire harness while using the augmented reality vision system.

18. The augmented reality vision system of claim 9 , wherein the record of operations include image data taken by the camera system representing what is seen on the display system.

19. The augmented reality vision system of claim 9 , wherein augmented reality syste

Assignees

• Boeing Co

Inventors

• Destories Jason Gerald
• Edman Frederick C
• Crocco James R

Classifications

• G02B2027/0178

  Eyeglass type (eyeglass details G02C) · CPC title

• G06F2113/16

  Cables, cable trees or wire harnesses · CPC title

• G06F3/167 Primary

  Audio in a user interface, e.g. using voice commands for navigating, audio feedback · CPC title

• G06F30/15

  Vehicle, aircraft or watercraft design · CPC title

• G06T19/006 Primary

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title