Processing device relating to inspection of inspection object, inspection system of inspection object, processing method relating to inspection of inspection object, and non-transitory storage medium

Patent metadata

Field	Value
Publication number	US-12136238-B2
Application number	US-202217675851-A
Country	US
Kind code	B2
Filing date	Feb 18, 2022
Priority date	Sep 3, 2021
Publication date	Nov 5, 2024
Grant date	Nov 5, 2024

Abstract

In an embodiment, a processing device relating to an inspection of an inspection object by a photography unit is provided. A processor of the processing device calculates a plurality of photography points as positions photographing the inspection object based on shape data in which a shape of a surface of the inspection object is indicated by a point group, and information relating to a position and a normal vector on the surface of the inspection object is defined by the point group. The processor executes analysis regarding a path that passes through all of the calculated photography points and minimizes a sum of a movement cost from each of the photography points to a photography point of a next movement destination, and calculates a path corresponding to an analysis result as a path for moving the photography unit.

First claim

What is claimed is:

1. A processing device relating to an inspection of an inspection object with use of a photography unit, the photography unit being connected to a movable body, a position and an attitude of the photography unit varying in accordance with a movement of the movable body, the processing device comprising:
a processor configured to:
acquire shape data, a shape of a surface of the inspection object being indicated by a point group, information relating to a position and a normal vector on the surface of the inspection object being defined by using the point group in the shape data, the surface of the inspection object being divided into polygons based on the point group in the shape data;
calculate a plurality of photography points as positions for photographing the inspection object by the photography unit based on the shape data in such a state that, in each of the polygons disposed in an inspection range of the surface of the inspection object, any part of each polygon is included in the photography range of photography at one or more of the photography points; and

calculate a solution of an optimization problem regarding a path that passes through all of the calculated photography points and minimizes a sum of a movement cost from each of the photography points to a photography point that is a next movement destination, so that a path corresponding to the calculated solution of the optimization problem is set as a path for moving the photography unit between the photography points, the movement cost from each of the photography points to the photography point that is the next movement destination being calculated based on one or more of a variation of a position of a connection portion of the movable body to the photography unit, a variation of an attitude of the connection portion of the movable body to the photography unit, and a variation of a control amount of the movable body in addition to a variation of the position of the photography unit and a variation of the attitude of the photography unit.

2. The processing device of claim 1 , wherein the processor is configured to calculate the solution of the optimization problem regarding the path that minimizes the sum of the movement cost, under a constraint condition that the photography points are successively passed one by one and each of the photography points is passed only once.

3. The processing device of claim 1 , wherein the processor is configured to calculate, as the path for moving the photography unit between the photography points, any one of an optimal solution, a local optimal solution and an approximate solution of the optimization problem regarding the path that passes through all of the photography points and minimizes the sum of the movement cost.

4. The processing device of claim 1 , wherein the processor is configured to determine a state of the surface of the inspection object, by image-processing an image that the photography unit photographs at each of the photography points.

5. The processing device of claim 4 , wherein the processor is configured to implement displaying either an image indicative of a defect determined to be present on the surface of the inspection object, or an image in which a determination result of the defect is reflected on the shape data.

6. The processing device of claim 1 , wherein the processor is configured to implement displaying any one of an image in which a real-time position and a real-time attitude of the photography unit are reproduced, an image that the photography unit is photographing in real time, and an image indicative of information relating to the normal vector in the range that is being photographed in real time on the surface of the inspection object, in a state in which the photography unit is moving between the photography points along the calculated path.

7. An inspection system comprising:
the processing device of claim 1 ;
the photography unit including an image-forming optical system and configured to photograph the inspection object by forming an image of the inspection object with use of the image-forming optical system; and
a movable body connected to the photography unit and configured to move, thereby varying the position and the attitude of the photography unit and rendering the photography unit movable between the photography points.

8. The inspection system of claim 7 , wherein the photography unit is configured to photograph the inspection object at each of the photography points, in such an attitude that an angle of an optical axis of the image-forming optical system to the surface of the inspection object falls within a predetermine angle range.

9. The inspection system of claim 7 , wherein the photography unit comprises:
an imaging element configured to form an image-forming surface of the image-forming optical system and configured to capture an image of light that is made incident through the image-forming optical system; and
a color filter including a wavelength selection region that passes light of a predetermined wavelength range, the color filter being disposed on an optical path of light that is incident on the imaging element through the image-forming optical system, and being configured to render the light of the predetermined wavelength range, which has passed through the wavelength selection region, incident on the imaging element.

10. A processing method relating to an inspection of an inspection object with use of a photography unit, the photography unit being connected to a movable body, a position and an attitude of the photography unit varying in accordance with a movement of the movable body, the processing method comprising:
acquiring shape data, a shape of a surface of the inspection object being indicated by a point group, information relating to a position and a normal vector on the surface of the inspection object being defined by using the point group in the shape data, the surface of the inspection object being divided into polygons based on the point group in the shape data;
calculating a plurality of photography points as positions for photographing the inspection object by the photography unit based on the shape data in such a state that, in each of the polygons disposed in an inspection range of the surface of the inspection object, any part of each polygon is included in the photography range of photography at one or more of the photography points; and
calculating a solution of an optimization problem regarding a path that passes through all of the calculated photography points and minimizes a sum of a movement cost from each of the photography points to a photography point that is a next movement destination so that a path corresponding to the calculated solution of the optimization problem is set as a path for moving the photography unit between the photography points, the movement cost from each of the photography points to the photography point that is the next movement destination being calculated based on one or more of a variation of a position of a connection portion of the movable body to the photography unit, a variation of an attitude of the connection portion of the movable body to the photography unit, and a variation of a control amount of the movable body in addition to a variation of the position of the photography unit and a variation of the attitude of the photography unit.

11. A non-transitory storage medium storing a processing program relating to an inspection of an inspection object with use of a photography unit, the photography unit being connected to a movable body, a position and an attitude of the photography unit varying in accordance with a movement of the movable body, the processing pr

Assignees

• Toshiba Kk

Inventors

• Sugiyama Nobukatsu
• Ooga Junichiro
• Ishihara Yoshiyuki
• Oaki Junji
• Ohno Hiroshi
• Okano Hideaki
• Kano Hiroya
• Kamikawa Takahiro

Classifications

• H04N7/183

  for receiving images from a single remote source · CPC title

• G06T2207/30108

  Industrial image inspection · CPC title

• G06T7/40

  Analysis of texture (depth or shape recovery from texture G06T7/529) · CPC title

• G06T7/0004

  Industrial image inspection · CPC title

• H04N23/695

  Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects · CPC title