Texture classification of digital images in aerial inspection

Patent metadata

Field	Value
Publication number	US-2020320293-A1
Application number	US-201916377956-A
Country	US
Kind code	A1
Filing date	Apr 8, 2019
Priority date	Apr 8, 2019
Publication date	Oct 8, 2020
Grant date	—

Abstract

Unmanned aerial inspection systems and associated methods. In one embodiment, an aerial platform (e.g., an Unmanned Aerial Vehicle (UAV)) navigates to a location of a geographic region, and captures a digital image of the geographic region with an imaging device. The aerial platform segments the digital image into superpixels, selects a region of interest from the digital image to define one or more patches associated with the superpixels, assigns terrain texture categories to the patches, and assigns the terrain texture categories to the superpixels based on the terrain texture categories of the patches to generate a texture classified representation of the digital image. The aerial platform determines whether a site contamination is present at the geographic region based on the texture classified representation of the digital image, and reports an alert upon identifying that the site contamination is present.

First claim

1 . An unmanned aerial inspection system, comprising:
an aerial platform comprising an imaging device, and at least one processor and at least one memory that:
navigate the aerial platform to a location of a geographic region;
capture a digital image of the geographic region with the imaging device while the aerial platform is airborne;
segment the digital image into superpixels;
select a region of interest from the digital image to define one or more patches associated with the superpixels;
assign terrain texture categories to the patches;
assign the terrain texture categories to the superpixels based on the terrain texture categories of the patches to generate a texture classified representation of the digital image;
determine whether a site contamination is present at the geographic region based on the texture classified representation of the digital image; and
report an alert upon determining that the site contamination is present.

2 . The unmanned aerial inspection system of claim 1 wherein the at least one processor and at least one memory:
designate one or more of the terrain texture categories as a site contamination category;
identify a percentage of the superpixels in the texture classified representation that are assigned the site contamination category; and
determine that the site contamination is present at the geographic region when the percentage exceeds a threshold.

3 . The unmanned aerial inspection system of claim 1 wherein the at least one processor and at least one memory:
designate one or more of the terrain texture categories as a site contamination category;
identify a total number of the superpixels in the texture classified representation that are assigned the site contamination category; and
determine that the site contamination is present at the geographic region when the total number exceeds a threshold.

4 . The unmanned aerial inspection system of claim 1 wherein the at least one processor and at least one memory assign the terrain texture categories to the superpixels by:
for each individual superpixel of the superpixels,
identifying pixels in the individual superpixel that belong to at least one of the patches;
identifying one or more of the terrain texture categories assigned to each of the pixels; and
assigning one of the terrain texture categories that is assigned to a majority of the pixels as a terrain texture category for the individual superpixel.

5 . The unmanned aerial inspection system of claim 1 wherein:
the at least one processor and at least one memory assign the terrain texture categories to the patches based on a patch classification model; and
the at least one processor and at least one memory:
present one or more test images to a user;
receive input from the user indicating areas in the test images as test patches;
receive input from the user assigning one of the terrain texture categories to the test patches; and
train the patch classification model based on the test patches.

6 . The unmanned aerial inspection system of claim 5 wherein:
the site contamination comprises vegetation encroachment, and the terrain texture categories include at least a high vegetation category and a vegetation-free category;
the at least one processor and at least one memory receive input from the user indicating first areas of concentrated vegetation within the test images as first test patches, and receive input from the user assigning the high vegetation category to the first test patches; and
the at least one processor and at least one memory receive input from the user indicating second areas of non-vegetation within the test images as second test patches, and receive input from the user assigning the vegetation-free category to the second test patches.

7 . The unmanned aerial inspection system of claim 5 wherein:
the site contamination comprises erosion, and the terrain texture categories include at least a high erosion category and an erosion-free category;
the at least one processor and at least one memory receive input from the user indicating first areas of concentrated erosion within the test images as first test patches, and receive input from the user assigning the high erosion category to the first test patches; and
the at least one processor and at least one memory receive input from the user indicating second areas of non-erosion within the test images as second test patches, and to receive input from the user assigning the erosion-free category to the second test patches.

8 . The unmanned aerial inspection system of claim 1 wherein the at least one processor and at least one memory:
send an alert message via wireless signals while the aerial platform is airborne when the site contamination is present.

9 . The unmanned aerial inspection system of claim 1 wherein the at least one processor and at least one memory:
send an alert message via wireless signals while the aerial platform is airborne that a high vegetation contamination is present at the geographic region such that a vegetation removal service can be directed to the location of the geographic region.

10 . The unmanned aerial inspection system of claim 1 wherein the aerial platform is one of a rotary-wing Unmanned Aerial Vehicle (UAV) and a fixed-wing UAV.

11 . A method of performing a site inspection, the method comprising:
navigating an aerial platform to a location of a geographic region; capturing a digital image of the geographic region with an imaging device onboard the aerial platform while the aerial platform is airborne; segmenting the digital image into superpixels at the aerial platform; selecting a region of interest from the digital image to define one or more patches associated with the superpixels; assigning terrain texture categories to the patches; assigning the terrain texture categories to the superpixels based on the terrain texture categories of the patches to generate a texture classified representation of the digital image; determining whether a site contamination is present at the geographic region based on the texture classified representation of the digital image; and reporting an alert upon determining that the site contamination is present.

12 . The method of claim 11 wherein determining whether a site contamination is present comprises:
designating one or more of the terrain texture categories as a site contamination category;
identifying a percentage of the superpixels in the texture classified representation that are assigned the site contamination category; and
determining that the site contamination is present at the geographic region when the percentage exceeds a threshold.

13 . The method of claim 11 wherein determining whether a site contamination is present comprises:
designating one or more of the terrain texture categories as a site contamination category;
identifying a total number of the superpixels in the texture classified representation that are assigned the site contamination category; and
determining that the site contamination is present at the geographic region when the total number exceeds a threshold.

14 . The method of claim 11 wherein assigning the terrain texture categories to the superpixels comprises:
for each individual superpixel of the superpixels,
identifying pixels in the individual superpixel that belong to at least one of the patches;
identifying one or more of the terrain texture categories assigned to each of the pixels; and
assigning one of the terrain texture categories that is assigned to a majority of the pixels as a terrain texture category for the individual super

Assignees

• Boeing Co

Inventors

• Yang Yan

Classifications

• G06T11/10 Primary

  Texturing; Colouring; Generation of textures or colours (retouching, inpainting or scratch removal G06T5/77) · CPC title

• G06V10/54

  relating to texture · CPC title

• G06V20/17

  taken from planes or by drones · CPC title

• G06V10/774

  Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title

• G06V10/235

  based on user input or interaction · CPC title