System and method for determining autonomous vehicle location using incremental image analysis

Patent metadata

Field	Value
Publication number	US-11079242-B2
Application number	US-201816226353-A
Country	US
Kind code	B2
Filing date	Dec 19, 2018
Priority date	Dec 29, 2017
Publication date	Aug 3, 2021
Grant date	Aug 3, 2021

Abstract

Systems, methods, and computer-readable storage media for determining autonomous vehicle location using incremental image analysis. An exemplary method can include identifying an expected position of an autonomous vehicle which is moving, and identifying, an actual position of the autonomous vehicle. The identifying of the actual position occurs by obtaining images of the autonomous vehicle's surroundings, initiating an iterative image comparison of those images to previously stored images within a given geographic radius of the autonomous vehicle, and iteratively extending the radius (and the pictures being compared) until a match is found or until the maximum radius is reached.

First claim

We claim:

1. A method comprising:
identifying, at a central location and via a processor accessing a memory device, an expected position of an autonomous vehicle which is moving; and
identifying, via the processor, an actual position of the autonomous vehicle, by:
transmitting, from the processor at the central location to the autonomous vehicle, a request for on-board location information, the on-board location information comprising:
surrounding images comprising surroundings of the autonomous vehicle; and
peer GPS (Global Positioning System) data for proximate peers;

receiving, at the central location from the autonomous vehicle and in response to the request, the surrounding images;

receiving, at the central location from the autonomous vehicle and in response to the request, the peer GPS data;
determining, based on the peer GPS data in the on-board location information, that the actual position of the autonomous vehicle is outside a threshold range of the expected position, to yield a determination;
initiating, based on the determination, an iterative image comparison, wherein each iteration of the iterative image comparison:
extends a geographic radius extending from the expected position; and
compares stored images within the geographic radius to the surrounding images,
until the comparison of the stored images to the surrounding images within a given geographic radius yields the actual position with a predetermined certainty.

2. The method of claim 1 , wherein the surrounding images comprise a panoramic image with a 360° view around the autonomous vehicle.

3. The method of claim 1 , wherein the autonomous vehicle is an aerial drone, and the surrounding images comprise an image of ground beneath the aerial drone.

4. The method of claim 1 , wherein the proximate peers comprise a plurality of other autonomous vehicles, and the peer GPS data comprise GPS coordinates of the other autonomous vehicles.

5. The method of claim 1 , wherein the identifying of the actual position of the autonomous vehicle occurs in real-time, with the surrounding images are taken by the autonomous vehicle immediately upon receiving the request, and the initiation of the iterative image comparison begins immediately upon receiving the on-board location information.

6. The method of claim 1 , wherein each iteration of the iterative image comparison further comprises:
filtering landmarks from the stored images which are located outside the geographic radius for a given iteration, to yield filtered images; and
wherein the comparing of the stored images to the surrounding images uses the filtered images.

7. The method of claim 6 , wherein comparing the stored images to the surrounding images using the filtered images is more computationally efficient than comparing the stored images to an entirety of the surrounding images.

8. The method of claim 1 , wherein the on-board location information further comprises distance data corresponding to the surrounding images, the distance data identifying a distance between the autonomous vehicle and an object pictured within the surrounding images.

9. A system comprising:
a processor; and
a computer-readable storage medium having instructions stored which, when executed by the processor, cause the processor to perform operations comprising:
identifying, at a central location, an expected position of an autonomous vehicle which is moving; and
identifying an actual position of the autonomous vehicle, by:
transmitting, from the central location to the autonomous vehicle, a request for on-board location information, the on-board location information comprising:
surrounding images comprising surroundings of the autonomous vehicle; and
peer GPS (Global Positioning System) data for proximate peers;

receiving, at the central location from the autonomous vehicle and in response to the request, the surrounding images;
receiving, at the central location from the autonomous vehicle and in response to the request, the peer GPS data;
determining, based on the peer GPS data in the on-board location information, that the actual position of the autonomous vehicle is outside a threshold range of the expected position, to yield a determination;
initiating, based on the determination, an iterative image comparison, wherein each iteration of the iterative image comparison:
extends a geographic radius extending from the expected position; and
compares stored images within the geographic radius to the surrounding images,
until the comparison of the stored images to the surrounding images within a given geographic radius yields the actual position with a predetermined certainty.

10. The system of claim 9 , wherein the surrounding images comprise a panoramic image with a 360° view around the autonomous vehicle.

11. The system of claim 9 , wherein the autonomous vehicle is an aerial drone, and the surrounding images comprise an image of ground beneath the aerial drone.

12. The system of claim 9 , wherein the proximate peers comprise a plurality of other autonomous vehicles, and the peer GPS data comprise GPS coordinates of the other autonomous vehicles.

13. The system of claim 9 , wherein the identifying of the actual position of the autonomous vehicle occurs in real-time, with the surrounding images are taken by the autonomous vehicle immediately upon receiving the request, and the initiation of the iterative image comparison begins immediately upon receiving the on-board location information.

14. The system of claim 9 , wherein each iteration of the iterative image comparison further comprises:
filtering landmarks from the stored images which are located outside the geographic radius for a given iteration, to yield filtered images; and
wherein the comparing of the stored images to the surrounding images uses the filtered images.

15. The system of claim 14 , wherein comparing the stored images to the surrounding images using the filtered images is more computationally efficient than comparing the stored images to an entirety of the surrounding images.

16. The system of claim 9 , wherein the on-board location information further comprises distance data corresponding to the surrounding images, the distance data identifying a distance between the autonomous vehicle and an object pictured within the surrounding images.

17. A non-transitory computer-readable storage medium having instructions stored which, when executed by a computing device, cause the computing device to perform operations comprising:
identifying, at a central location, an expected position of an autonomous vehicle which is moving; and
identifying an actual position of the autonomous vehicle, by:
transmitting, at the central location to the autonomous vehicle, a request for on-board location information, the on-board location information comprising:
surrounding images comprising surroundings of the autonomous vehicle; and
peer GPS (Global Positioning System) data for proximate peers;

receiving, at the central location from the autonomous vehicle and in response to the request, the surrounding images;
receiving, at the central location from the autonomous vehicle and in response to the request, the peer GPS data;
determining, based on the peer GPS data in the on-board location information, that the actual position of the autonomous vehicle is outside a threshold range of the expected position, to yield a determination;
initiating, based on the determination, an iterative image comparison, wherein each iteration of the iterative image comparison:
extends a geographic rad

Assignees

• Walmart Apollo Llc

Inventors

• Cantrell Robert L
• Winkle David C
• O'Brien John J

Classifications

• B64U2201/10

  autonomous, i.e. by navigating independently from ground or air stations, e.g. by using inertial navigation systems [INS] · CPC title

• B64U2201/00

  UAVs characterised by their flight controls · CPC title

• G06T7/74 Primary

  involving reference images or patches · CPC title

• B64U2101/30

  for imaging, photography or videography · CPC title

• B64U20/87

  Mounting of imaging devices, e.g. mounting of gimbals · CPC title