Image data processing system, unmanned aerial vehicle, image data processing method, and non-transitory computer-readable recording medium

Patent metadata

Field	Value
Publication number	US-12106447-B2
Application number	US-202017627814-A
Country	US
Kind code	B2
Filing date	Jul 21, 2020
Priority date	Jul 23, 2019
Publication date	Oct 1, 2024
Grant date	Oct 1, 2024

Abstract

An image data processing system is provided including: a discharge portion for discharging a content in a container onto a subject to form a sprayed symbol; a camera for capturing an image of a region including the sprayed symbol on the subject; and an image superimposition processing portion for performing an image superimposition process on the image using the sprayed symbol as a reference point. The camera may include a visible light camera for a visible light image and an infrared camera for capturing a thermal image of a subject.

First claim

What is claimed is:

1. An image data processing system comprising: a discharge portion for discharging a content in a container onto a subject to form a sprayed symbol; a camera for capturing an image of a region including the sprayed symbol on the subject; and an image superimposition processing portion for performing an image superimposition process on the image using the sprayed symbol as a reference point, wherein the camera includes a visible light camera for a visible light image and an infrared camera for capturing a thermal image of the subject, the discharge portion is configured to use the content to form, on a surface of the subject, a different temperature area having a temperature that is different from those of other parts of the subject, the infrared camera is configured to capture the thermal image of a region including the different temperature area, and the image superimposition processing portion is configured to, in an image superimposition process on the thermal image and the visible light image, use the different temperature area and the sprayed symbol as reference points in the image superimposition process.

2. The mage data processing system according to claim 1 , wherein the content includes a vaporizable component and the different temperature area is formed by vaporization heat generated when the content discharged onto the subject evaporates.

3. The image data processing system according to claim 1 , further comprising a connectivity evaluation portion for evaluating a difficulty of the image superimposition process based on a continuity of a superimposed part of the image in the image superimposition process, wherein
if the connectivity evaluation portion evaluates that the image superimposition process on an image where the subject is captured is difficult, the discharge portion forms the sprayed symbol.

4. The image data processing system according to claim 3 , wherein the connectivity evaluation portion is configured to perform a process to extract feature points in an image captured by the camera, and evaluate a difficulty of the image superimposition process based on a quantity of the obtained feature points.

5. The image data processing system according to claim 3 , wherein the connectivity evaluation portion is configured to sense whether there is a periodic repeated pattern, and evaluate the difficulty based on a sensing result.

6. The image data processing system according to claim 3 , wherein the connectivity evaluation portion is configured to attempt the image superimposition process on the image and evaluate the difficulty based on the attempt result.

7. The image data processing system according to claim 3 , wherein the image superimposition process includes a connecting process on a plurality of images.

8. The image data processing system according to claim 1 , wherein the image superimposition process includes a synthetic process on a plurality of images.

9. The image data processing system according to claim 1 , further comprising a light for shooting in a dark.

10. The image data processing system according to claim 1 , further comprising a temperature adjustment apparatus for adjusting a temperature of the content to a temperature that is different from a surface temperature of the subject.

11. The image data processing system according to claim 1 , wherein the content is colored.

12. The image data processing system according to claim 1 , wherein the container is an aerosol container.

13. An unmanned aerial vehicle that incorporates the image data processing system according to claim 1 .

14. The image data processing system according to claim 1 , further comprising a connectivity evaluation portion for evaluating a difficulty of the image superimposition process based on a continuity of a superimposed part of the image in the image superimposition process, wherein if the connectivity evaluation portion evaluates that the image superimposition process on an image where the subject 1s captured 1s difficult, the discharge portion forms the sprayed symbol.

15. An image data processing method comprising: discharging a content from a container incorporated in an unmanned aerial vehicle onto a subject to form a sprayed symbol; capturing a plurality of images related to a region of the subject including the sprayed symbol; and processing image data of the plurality of images using the sprayed symbol as a reference point for an image superimposition process, wherein the capturing includes a visible light camera for a visible light image and an infrared camera for capturing a thermal image of the subject, the discharging is configured to use the content to form, on a surface of the subject, a different temperature area having a temperature that is different from those of other parts of the subject, the infrared camera is configured to capture the thermal image of a region including the different temperature area, and the image superimposition process is configured to, in an image superimposition process on the thermal image and the visible light image, use the different temperature area and the sprayed symbol as reference points in the image superimposition process.

16. The image data processing method according to claim 15 , comprising:
capturing a surface profile of the subject; and
determining whether to discharge the content onto the subject based on the captured surface profile.

17. The image data processing method according to claim 16 , wherein the determining whether to discharge the content onto the subject includes, in a case where the surface profile does not have unevenness deeper than a predetermined first threshold value, determining that the surface profile is a smooth surface and then determining to discharge the content.

18. The image data processing method according to claim 16 , wherein the determining whether to discharge the content onto the subject includes, in a case where the surface profile has a scratch or unevenness deeper than a predetermined second threshold value, determining to discharge the content onto the subject, and
capturing the plurality of images includes capturing the subject at a predetermined first distance, and capturing the subject at a second distance, which is farther than the first distance.

19. A non-transitory computer-readable recording medium that records a program that, when executed by a computer, causes the computer to perform steps of: discharging a content from a container incorporated in an unmanned aerial vehicle onto a subject to form a sprayed symbol; capturing a plurality of images related to a region of the subject including the sprayed symbol; and processing image data of the plurality of images using the sprayed symbol as a reference point for an image superimposition process, wherein the capturing includes a visible light camera for a visible light image and an infrared camera for capturing a thermal image of the subject, the discharging is configured to use the content to form, on a surface of the subject, a different temperature area having a temperature that is different from those of other parts of the subject, the infrared camera is configured to capture the thermal image of a region including the different temperature area, and the image superimposition process is configured to, in an image superimposition process on the thermal image and the visible light image, use the different temperature area and the sprayed symbol as reference points in the image superimposition process.

Assignees

• Toyo Seikan Co Ltd

Inventors

• Kominami Atsushi
• Araki Souji

Classifications

• H04N23/23

  from thermal infrared radiation · CPC title

• B64U30/21

  Rotary wings · CPC title

• B64U10/14

  with four distinct rotor axes, e.g. quadcopters · CPC title

• B64U20/87

  Mounting of imaging devices, e.g. mounting of gimbals · CPC title

• B64U2201/10

  autonomous, i.e. by navigating independently from ground or air stations, e.g. by using inertial navigation systems [INS] · CPC title