Inspection support system for erroneous attachment using image color conversion

Patent metadata

Field	Value
Publication number	US-11869219-B2
Application number	US-202017119478-A
Country	US
Kind code	B2
Filing date	Dec 11, 2020
Priority date	Jan 17, 2020
Publication date	Jan 9, 2024
Grant date	Jan 9, 2024

Abstract

An inspection support system for erroneous attachment includes an imaging device and an erroneous attachment detector. The imaging device is configured to capture an image that represents a state in which multiple components are attached to each other. The erroneous attachment detector is configured to convert a color space of the image into an HSV color space, create a comparative image on the basis of hue and saturation in the HSV color space, determine whether the multiple components are each a correct component and whether the multiple components are attached in a correct manner by comparing the comparative image with a correct image that is based on the hue and the saturation, and detect the erroneous attachment of at least one of the multiple components, in a case where the at least one of the multiple components is not the correct component and/or is not attached in the correct manner.

First claim

The invention claimed is:

1. An inspection support system for erroneous attachment, the inspection support system comprising:
a controller including a processor and a memory communicably connected with the processor; and
an imaging device communicably connected with the controller and configured to capture an image that represents a state in which multiple components are attached to each other, wherein
the controller is configured to
convert a color space of the image captured by the imaging device into an HSV color space,
create a comparative image on a basis of hue and saturation in the HSV color space,
determine whether the multiple components are each a correct component and whether the multiple components are attached to each other in a correct manner by comparing the comparative image with a correct image, the correct image being based on the hue and the saturation and representing a state in which the multiple components are attached to each other correctly, and
detect the erroneous attachment of at least one of the multiple components, in a case where the erroneous attachment detector determines that the at least one of the multiple components is not the correct component and/or that the at least one of the multiple components is not attached in the correct manner, and
wherein the comparative image is constituted by only hue (H) and saturation (S) in the HSV color space of the image and omits value (V) in the HSV color space of the image.

2. The inspection support system for the erroneous attachment according to claim 1 , wherein the determining whether the multiple components are attached to each other in the correct manner comprises at least one of: determining whether the multiple components are attached to each other at a correct position; determining whether the multiple components are attached to each other in a correct orientation; or determining whether the multiple components are attached to each other at a correct angle.

3. The inspection support system for the erroneous attachment according to claim 2 , wherein the determining whether the multiple components are attached to each other at the correct position comprises determining presence of mistaking one of the multiple components for another of the multiple components subjected to attachment.

4. The inspection support system for the erroneous attachment according to claim 1 , wherein the erroneous attachment of the at least one of the multiple components comprises absence of attachment of one of the multiple components to another of the multiple components.

5. The inspection support system for the erroneous attachment according to claim 2 , wherein the erroneous attachment of the at least one of the multiple components comprises absence of attachment of one of the multiple components to another of the multiple components.

6. The inspection support system for the erroneous attachment according to claim 3 , wherein the erroneous attachment of the at least one of the multiple components comprises absence of attachment of one of the multiple components to another of the multiple components.

7. The inspection support system for the erroneous attachment according to claim 1 , further comprising a notifier comprising a display screen and configured to notify a user of the erroneous attachment of the at least one of the multiple components detected by the erroneous attachment detector.

8. The inspection support system for the erroneous attachment according to claim 2 , further comprising a notifier comprising a display screen and configured to notify a user of the erroneous attachment of the at least one of the multiple components detected by the erroneous attachment detector.

9. The inspection support system for the erroneous attachment according to claim 3 , further comprising a notifier comprising a display screen and configured to notify a user of the erroneous attachment of the at least one of the multiple components detected by the erroneous attachment detector.

10. The inspection support system for the erroneous attachment according to claim 7 , wherein
the display screen is configured to display the comparative image, and the notifier is configured to
highlight, on the display screen, a region, of the comparative image, that contains any of the multiple components in which the erroneous attachment is detected by the erroneous attachment detector, or
highlight, on the display screen, a region, of the comparative image, that contains a part in which any of the multiple components, supposed to be attached, is determined by the erroneous attachment detector as not being attached.

11. The inspection support system for the erroneous attachment according to claim 8 , wherein
the display screen is configured to display the comparative image, and the notifier is configured to
highlight, on the display screen, a region, of the comparative image, that contains any of the multiple components in which the erroneous attachment is detected by the erroneous attachment detector, or
highlight, on the display screen, a region, of the comparative image, that contains a part in which any of the multiple components, supposed to be attached, is determined by the erroneous attachment detector as not being attached.

12. The inspection support system for the erroneous attachment according to claim 9 , wherein
the display screen is configured to display the comparative image, and the notifier is configured to
highlight, on the display screen, a region, of the comparative image, that contains any of the multiple components in which the erroneous attachment is detected by the erroneous attachment detector, or
highlight, on the display screen, a region, of the comparative image, that contains a part in which any of the multiple components, supposed to be attached, is determined by the erroneous attachment detector as not being attached.

13. The inspection support system for the erroneous attachment according to claim 1 , further comprising a storage communicably connected with the controller and configured to store, as a history, the comparative image in which the erroneous attachment is not detected by the erroneous attachment detector.

14. The inspection support system for the erroneous attachment according to claim 2 , further comprising a storage communicably connected with the controller and configured to store, as a history, the comparative image in which the erroneous attachment is not detected by the erroneous attachment detector.

15. The inspection support system for the erroneous attachment according to claim 3 , further comprising a storage communicably connected with the controller and configured to store, as a history, the comparative image in which the erroneous attachment is not detected by the erroneous attachment detector.

16. The inspection support system for the erroneous attachment according to claim 1 , wherein the controller is configured to store the correct image in a database.

17. The inspection support system for the erroneous attachment according to claim 2 , wherein the controller is further configured to store the correct image in a database.

18. The inspection support system for the erroneous attachment according to claim 3 , wherein the controller is configured to store the correct image in a database.

19. An inspection support system for erroneous attachment, the inspection support system comprising:
an imaging device communicably connected with circuitry and configured to capture an image that represents a state in which multiple components are attached

Assignees

• Subaru Corp

Inventors

• Sato Yoichi
• Sako Shoichiro
• Kikuchi Marosuke

Classifications

• G06T11/10

  Texturing; Colouring; Generation of textures or colours (retouching, inpainting or scratch removal G06T5/77) · CPC title

• G06T7/90 Primary

  Determination of colour characteristics · CPC title

• G06F16/51

  Indexing; Data structures therefor; Storage structures · CPC title

• G06T7/97

  Determining parameters from multiple pictures (depth or shape recovery from multiple images G06T7/55; stereo camera calibration G06T7/85) · CPC title

• G06T11/001

  Physics · mapped topic