Systems and methods for dynamic planning and operation of autonomous systems using image observation and information theory

Patent metadata

Field	Value
Publication number	US-10618673-B2
Application number	US-201715488182-A
Country	US
Kind code	B2
Filing date	Apr 14, 2017
Priority date	Apr 15, 2016
Publication date	Apr 14, 2020
Grant date	Apr 14, 2020

Abstract

Systems and methods described herein incorporate autonomous navigation using a vision-based guidance system. The vision-based guidance system enables autonomous trajectory planning and motion execution by the described systems and methods without feedback or communication with external operators. The systems and methods described herein can autonomously track an object of interest while seeking to obtain a diversity of views of the object of interest to aid in object identification. The systems and methods described include a robust reacquisition methodology. By handling navigation and tracking autonomously, systems described herein can react more quickly to non-cooperative moving objects of interest and can operate in situations where communications with external operators is compromised or absent.

First claim

The invention claimed is:

1. An autonomous vehicle system, comprising:
a chassis including one or more motors;
an imaging system attached to the chassis;
a vision-based guidance system including a memory and at least one of a central processing unit (CPU) or a graphics processing unit (GPU) configured to:
acquire images of an object of interest using the imaging system;
analyze the images of the object to determine an object position and an object size relative to the imaging system;
determine a path and trajectory of the autonomous vehicle system relative to the object of interest, the trajectory identifying a sequence of waypoints and the path identifying a route through the sequence of waypoints, each waypoint indicating a position in space relative to the object of interest, at least one waypoint in the sequence of waypoints identifying a distance and specific viewing angle from the autonomous vehicle to the object of interest; and
control the one or more motors to move the autonomous vehicle system along the trajectory while keeping the object of interest in view of the imaging system.

2. The system of claim 1 , wherein analyzing the images of the object of interest to determine the object position and the object size relative to the imaging system includes updating an object model based on at least a portion of the acquired images using a discriminative learning-based tracking algorithm.

3. The system of claim 2 , wherein the vision-based guidance system is further configured to:
compute a tracking confidence score between at least a first image and a second image as determined by the discriminative learning-based tracking algorithm;
in response to the tracking confidence score being greater than or equal to a threshold value, update the object model using the second image using the discriminative learning-based tracking algorithm; and
in response to the tracking confidence score being less than the threshold value, halt updating the object model.

4. The system of claim 3 , wherein the tracking confidence score is a rate of change of a score output by the discriminative learning-based tracking algorithm.

5. The system of claim 4 , wherein the discriminative learning-based tracking algorithm is a support vector machine.

6. The system of claim 3 , wherein the vision-based guidance system is further configured to:
automatically enter a search mode to reacquire the object of interest in an image in response to the tracking confidence score being less than the threshold value.

7. The system of claim 2 , wherein the vision-based guidance system is further configured to apply a feature-based matching algorithm to the images to compensate for relative motion of the imaging system with respect to the object of interest.

8. The system of claim 1 , further comprising an altimeter.

9. The system of claim 1 , wherein the vision-based guidance system is configured to communicate with a remote operator to send or receive information related to the object of interest including images.

10. The system of claim 1 , wherein the imaging system is attached to a gimbal of the chassis.

11. The system of claim 10 , wherein the vision-based guidance system is further configured to move the imaging system using the gimbal to keep the object of interest in view of the imaging system.

12. The system of claim 1 , wherein the vision-based guidance system is implemented entirely onboard the chassis.

13. The system of claim 1 , wherein determining a trajectory relative to an object of interest includes:
computing a variation of information metric using one or more of the acquired images of the object of interest;
determining a waypoint based on a measure of information gain; and
planning a path to reach the waypoint.

14. The system of claim 13 , wherein determining a waypoint based on a measure of information gain includes:
accessing a lookup table specific to a class of the object of interest; and
identifying the waypoint corresponding to the estimated next-best view in the lookup table based upon the current location of the autonomous vehicle system in relation to the object of interest.

15. The system of claim 14 , wherein the class of the object of interest is at least one of a building, a vehicle, an individual, or a natural feature.

16. The system of claim 13 , wherein determining a waypoint based on a measure of information gain includes:
selecting the waypoint from a pre-determined path.

17. A method of autonomously tracking an object of interest, comprising:
acquiring images of the object of interest using an imaging system attached to a chassis of an autonomous vehicle system, the chassis including one or more motors;
analyzing the images of the object of interest to determine an object position and an object size relative to the imaging system;
determining a path and trajectory of the autonomous vehicle system relative to the object of interest, the trajectory identifying a sequence of waypoints and the path identifying a route through the sequence of waypoints, each waypoint indicating a position in space relative to the object of interest, at least one waypoint in the sequence of waypoints identifying a distance and specific viewing angle from the autonomous vehicle to the object of interest; and
controlling the one or more motors to move the chassis along the trajectory while keeping the object of interest in view of the imaging system.

18. The method of claim 17 , wherein analyzing the images of the object of interest to determine the object position and the object size relative to the imaging system includes updating an object model based on at least a portion of the acquired images using a discriminative learning-based tracking algorithm.

19. The method of claim 18 , further comprising
computing a tracking confidence score between at least a first image and a second image in the as determined by the discriminative learning-based tracking algorithm;
in response to the tracking confidence score being greater than or equal to a threshold value, updating the object model using the second image using the discriminative learning-based tracking algorithm; and
in response to the tracking confidence score being less than the threshold value, halt updating the object model.

20. The method of claim 19 , wherein the tracking confidence score is a rate of change of a score output by the discriminative learning-based tracking algorithm.

21. The method of claim 20 , wherein the discriminative learning-based tracking algorithm is a support vector machine.

22. The method of claim 19 , further comprising automatically entering a search mode to reacquire the object of interest in an image in response to the tracking confidence score being less than the threshold value.

23. The method of claim 17 , further comprising applying a feature-based matching algorithm to the images to compensate for relative motion of the imaging system with respect to the object of interest.

24. The method of claim 17 wherein determining a trajectory relative to an object of interest includes:
computing a variation of information metric using one or more of the acquired images of the object of interest;
determining a waypoint based on a measure of information gain; and
planning a path to reach the waypoint.

25. The method of claim 24 , wherein determining a waypoint based on a measure of information gain includes:
accessing a loo

Assignees

• Chan Michael T
• Duarte Ronald
• Massachusetts Inst Technology

Inventors

• Chan Michael T
• Duarte Ronald

Classifications

• G06T2207/10032

  Satellite or aerial image; Remote sensing · CPC title

• G06T7/246

  using feature-based methods, e.g. the tracking of corners or segments · CPC title

• G06T2207/30244

  Camera pose · CPC title

• G06T7/73

  using feature-based methods · CPC title

• G06K9/6201

  Physics · mapped topic