Intelligent inspection devices

Patent metadata

Field	Value
Publication number	US-11574445-B2
Application number	US-201916677235-A
Country	US
Kind code	B2
Filing date	Nov 7, 2019
Priority date	Aug 15, 2019
Publication date	Feb 7, 2023
Grant date	Feb 7, 2023

Abstract

An intelligent inspection device is disclosed. The intelligent inspection device includes a camera configured to photograph a refrigerator that is installed and a processor configured to position a first refrigerator image photographed by the camera in a preset area of an inspection frame, compare the first refrigerator image with the inspection frame, inspect whether a step is present in the refrigerator based on a result of the comparison, and learn a solution corresponding to a result of the inspection to provide the solution to a user. The intelligent inspection device may be associated with an artificial intelligence module, an unmanned aerial vehicle (UAV), a robot, an augmented reality (AR) device, a virtual reality (VR) device, devices related to 5G services, and the like.

First claim

What is claimed is:

1. An intelligent inspection device, comprising:
a communication unit;
a memory configured to store a learning model;
a camera configured to photograph a refrigerator having two or more doors; and
one or more processors configured to inspect a step difference between the two or more doors of the refrigerator,
wherein the one or more processors are configured to:
capture a first refrigerator image of the refrigerator via the camera, wherein the refrigerator is positioned in a preset area of an inspection frame;
determine whether the step difference between the two or more doors of the refrigerator is present in the refrigerator using the learning model stored in the memory based on a comparison of the first refrigerator image of the refrigerator and the inspection frame;
generate solution method data for the step difference based on the comparison; and
transmit the solution method data to an augmented reality electronic device via the communication unit to cause the augmented reality electronic device to display a real image of the refrigerator superimposed with a virtual image of an area of the step difference to be repaired and an image of a tool to be used for the solution method data, and to sequentially display, in augmented reality, a process of repairing the step difference in the refrigerator.

2. The intelligent inspection device of claim 1 , wherein the one or more processors are further configured to:
determine the learning model to be used for the refrigerator based on the first refrigerator image and set the inspection frame based on the determined learning model.

3. The intelligent inspection device of claim 2 , wherein the one or more processors are further configured to:
pre-process the first refrigerator image; and
extract a feature value from the pre-processed first refrigerator image.

4. The intelligent inspection device of claim 3 , wherein the one or more processors is further configured to pre-process the first refrigerator image by using one of gray-scaling, blurring, sharpening, canny edge detection, thresholding, erosion, morphology, or noise removal.

5. The intelligent inspection device of claim 2 , wherein the one or more processors is further configured to align the first refrigerator image with the inspection frame by adjusting a size or a position of the first refrigerator image.

6. The intelligent inspection device of claim 2 , wherein the inspection frame comprises:
a first reference line formed at an upper side of the inspection frame;
a second reference line vertically spaced apart from the first reference line and formed at a lower side of the inspection frame;
a third reference line formed at a left side of the inspection frame; and
a fourth reference line laterally spaced apart from the third reference line and formed at a right side of the inspection frame.

7. The intelligent inspection device of claim 6 , wherein comparing the first refrigerator image with the inspection frame comprises:
comparing an upper side of a left door of the refrigerator or an upper side of a right door of the refrigerator with the first reference line or comparing a lower side of the left door of the refrigerator or a lower side of the right door of the refrigerator with the second reference line; and
determining whether the step difference is present comprises using the learning model to recognize whether the step difference is present.

8. The intelligent inspection device of claim 2 , further comprising a display unit,
wherein the one or more processors is further configured to display information of the solution method data via the display unit.

9. The intelligent inspection device of claim 1 , wherein the transmitted solution method data is further configured to cause the augmented reality electronic device to display the process of repairing the step difference while maintaining a constant time interval.

10. The intelligent inspection device of claim 1 , wherein the one or more processors is further configured to:
receive via the communication unit, from a network, downlink control information (DCI) to be used for scheduling transmission of information of the inspection performed by the intelligent inspection device; and
transmit, via the communication unit, the information of the inspection to the network based on the DCI.

11. The intelligent inspection device of claim 10 , wherein the one or more processors is further configured to:
perform an initial access procedure with the network based on a synchronization signal block (SSB); and
transmit, via the communication unit, the information on the result of the inspection to the network through a physical uplink shared channel (PUSCH),
wherein demodulation reference signals (DM-RS) of the SSB and the PUSCH are quasi co-located (QCLed) for QCL type D.

12. An intelligent inspection device, comprising:
a communication unit;
a camera configured to photograph a refrigerator having two or more doors; and
one or more processors configured to inspect a step difference between the two or more doors of the refrigerator,
wherein the one or more processors are configured to:
capture a first refrigerator image of the refrigerator via the camera, wherein the refrigerator is positioned in a preset area of an inspection frame;
determine a reference model for the refrigerator based on the first refrigerator image; and
obtain a determination of whether the step difference between the two or more doors of the refrigerator is present in the refrigerator based on a comparison of the first refrigerator image of the refrigerator and the inspection frame;
generate solution method data for the step difference based on the comparison; and
transmit the solution method data to an augmented reality electronic device via the communication unit to cause the augmented reality electronic device to display a real image of the refrigerator superimposed with a virtual image of an area of the step difference to be repaired and an image of a tool to be used for the solution method data, and to sequentially display, in augmented reality, a process of repairing the step difference in the refrigerator.

13. The intelligent inspection device of claim 12 , wherein the one or more processors are further configured to provide the first refrigerator image and the determined reference model to an artificial intelligence (AI) processing server via the communication unit, and obtain the determination of whether the step difference is present from the AI server.

Assignees

• Lg Electronics Inc

Inventors

• Han Junsoo
• Yang Younghun
• Eom Yonghwan
• Jeong Junseong
• Han Cholok

Classifications

• G06N3/09

  Supervised learning · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06Q50/10

  Services · CPC title

• H04N23/57

  Mechanical or electrical details of cameras or camera modules specially adapted for being embedded in other devices · CPC title

• G06T19/006 Primary

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title