Visual localization of unmanned aerial vehicles based on marker detection and processing

Patent metadata

Field	Value
Publication number	US-9619709-B2
Application number	US-201514721162-A
Country	US
Kind code	B2
Filing date	May 26, 2015
Priority date	Jul 25, 2013
Publication date	Apr 11, 2017
Grant date	Apr 11, 2017

Abstract

A method for locating markers in an image captured by a mobile device moving about an operating space. The method includes preprocessing an image to generate a set of image data, locating fixed features of markers by tracing edges of the fixed features, and extracting variable data payloads of each of the markers associated with the located fixed features. The fixed features of each of the markers may include a pair of parallel lines extending along opposite sides of a data area containing the variable data payload, and each of the lines extends a distance beyond each exposed end of the data area to avoid missing data when markers are not arranged orthogonally to the scan direction. The preprocessing involves rotating or skewing the image to provide rotated or skewed versions of the image to facilitate locating markers regardless of their angular orientation in the image.

First claim

We claim:

1. A system for facilitating determination of positions of mobile devices such as unmanned aerial vehicles within an operating space containing a number of objects and markers placed on exposed surfaces of the objects, with each of the markers including a pattern of linear bars adjacent a data area, comprising:
a mobile device operable to move about the operating space in response to control signals; and
a controller on the mobile device generating the control signals based on a determination of a position of the mobile device within the operating space relative to locations of the markers,
wherein the controller executes a localization module to locate each of the markers, to extract data from the data areas, and to estimate the position of the mobile device based on the extracted data,
wherein the pattern of linear bars comprises first and second pairs of spaced apart and parallel bars with the first and second pairs being disposed on opposite sides of the data area, and
wherein each of the linear bars in the first and second pairs of parallel bars has a length greater than a maximum length of the data area such that each of the linear bars extends at each end a distance beyond the data area.

2. The system of claim 1 , wherein the localization module uses a single direction scan of an image to initially locate the markers in the image.

3. The system of claim 1 , wherein at least one of the markers is positioned to be oriented in the image such that the pattern of linear bars is non-orthogonal to the single direction scan and wherein the localization module repeats the single direction scan on a rotated or skewed version of the image to locate the markers in the image.

4. The system of claim 1 , wherein each of the data areas comprises one or more linear patterns extending parallel to inner edges of the pair of spaced apart and parallel bars and at a predefined distance from the inner edge of a proximate one of the parallel bars.

5. The system of claim 1 , wherein the pattern of linear bars comprises parallel stripes on a pair of opposite sides of the data area and wherein the parallel stripes in the first and second pairs are spaced apart a predefined distance.

6. The system of claim 1 , wherein the distance that each of the linear bars extends at each end beyond the data area is at least 20 percent of the maximum length of the data area.

7. The system of claim 6 , wherein the distance that each of the linear bars extends at each end beyond the data area is at least 30 percent of the maximum length of the data area.

8. A system for facilitating determination of positions of mobile devices such as unmanned aerial vehicles within an operating space containing a number of objects and markers placed on exposed surfaces of the objects, with each of the markers including a pattern of linear bars adjacent a data area, comprising:
a mobile device operable to move about the operating space in response to control signals; and
a controller on the mobile device generating the control signals based on a determination of a position of the mobile device within the operating space relative to locations of the markers,
wherein the controller executes a localization module to locate each of the markers using a single direction scan, to extract data from the data areas, and to estimate the position of the mobile device based on the extracted data,
wherein at least one of the markers is positioned to be oriented in the image such that the pattern of linear bars is non-orthogonal to the single direction scan, and
wherein the localization module repeats the single direction scan on a rotated or skewed version of the image to locate the markers in the image.

9. The system of claim 8 , wherein the pattern of linear bars comprises at least a pair of spaced apart and parallel bars disposed on either side of the data area.

10. The system of claim 9 , wherein the pair of parallel bars has a length greater than a maximum length of the pattern in the data area such that each of the linear bars extends a distance beyond each exposed end of the pattern in the data area.

11. The system of claim 9 , wherein the pattern of each of the data areas comprises one or more linear patterns extending parallel to inner edges of the pair of spaced apart and parallel bars and at predefined distances from the inner edges.

12. The system of claim 9 , wherein the pattern of linear bars comprises parallel stripes on a pair of opposite sides of the data area and wherein adjacent pairs of the parallel stripes are spaced apart a predefined distance.

13. A system for facilitating determination of positions of mobile devices such as unmanned aerial vehicles within an operating space containing a number of objects and markers placed on exposed surfaces of the objects, with each of the markers including a pattern of linear bars adjacent a data area, comprising:
a mobile device operable to move about the operating space in response to control signals; and
a controller on the mobile device generating the control signals based on a determination of a position of the mobile device within the operating space relative to locations of the markers,
wherein the controller executes a localization module to locate each of the markers, to extract data from the data areas, and to estimate the position of the mobile device based on the extracted data,
wherein the pattern of linear bars comprises parallel stripes on a pair of opposite sides of the data area, and
wherein the parallel stripes on each of opposite sides of the data area are spaced apart a predefined distance.

14. The system of claim 13 , wherein the localization module uses a single direction scan of an image to initially locate the markers in the image.

15. The system of claim 13 , wherein at least one of the markers is positioned to be oriented in the image such that the pattern of linear bars is non-orthogonal to the single direction scan and wherein the localization module repeats the single direction scan on a rotated or skewed version of the image to locate the markers in the image.

16. The system of claim 13 , wherein the pair of parallel bars has a length greater than a maximum length of the data area such that the pair of parallel bars extends a distance beyond each exposed end of the pattern in the data area.

17. A system for facilitating determination of positions of mobile devices such as unmanned aerial vehicles within an operating space containing a number of objects and markers placed on exposed surfaces of the objects, with each of the markers including a pattern of linear bars adjacent a data area, comprising:
a mobile device operable to move about the operating space in response to control signals; and
a controller on the mobile device generating the control signals based on a determination of a position of the mobile device within the operating space relative to locations of the markers,
wherein the controller executes a localization module to locate each of the markers, to extract data from the data areas, and to estimate the position of the mobile device based on the extracted data,
wherein the pattern of linear bars comprises at least a pair of spaced apart and parallel bars disposed on either side of the data area, and
wherein the pair of parallel bars has a length greater than a maximum length of the data area such that the pair of parallel bars extends a distance beyond each exposed end of the pattern in the data area.

18. The system of claim 17 , wherein the pattern of each of the data areas comprises one or more l

Assignees

• Disney Entpr Inc

Inventors

• Stark James Alexander
• Wong Clifford

Classifications

• G06V20/13

  Satellite images · CPC title

• G06V10/245

  by locating a pattern; Special marks for positioning · CPC title

• G05D1/101

  specially adapted for aircraft · CPC title

• G06K19/06131

  the marking comprising a target pattern, e.g. for indicating the center of the bar code or for helping a bar code reader to properly orient the scanner or to retrieve the bar code inside of an image · CPC title

• G06K2009/3225

  Physics · mapped topic