Combining multiple estimates of an environment into a consolidated estimate for an autonomous vehicle

Patent metadata

Field	Value
Publication number	US-9684836-B1
Application number	US-201514939232-A
Country	US
Kind code	B1
Filing date	Nov 12, 2015
Priority date	Jul 31, 2013
Publication date	Jun 20, 2017
Grant date	Jun 20, 2017

Abstract

A vehicle is provided that may combine multiple estimates of an environment into a consolidated estimate. The vehicle may receive first data indicative of the region of interest in an environment from a sensor of the vehicle. The first data may include a first accuracy value and a first estimate of the region of interest. The vehicle may also receive second data indicative of the region of interest in the environment, and the second data may include a second accuracy value and a second estimate of the region of interest. Based on the first data and the second data, the vehicle may combine the first estimate of the region of interest and the second estimate of the region of interest.

First claim

What is claimed is:

1. A method performed by a computing device comprising one or more processors, the method comprising:
receiving, via at least one sensor coupled to an autonomous vehicle operating in a lane of travel, first data including a first estimate of a boundary location of a boundary of the lane of travel, the first data having been acquired at a first time;
based on the first data, determining a first variance in the first estimate of the boundary location;
receiving, via the at least one sensor, second data including a second estimate of the boundary location, the second data having been acquired at a second time different from the first time;
based on the second data, determining a second variance in the second estimate of the boundary location; and
based on the first estimate, the first variance, the second estimate, and the second variance, determining third data including a combined estimate of the boundary location and further including a combined variance in the combined estimate.

2. The method of claim 1 , further comprising:
based on the third data, providing instructions to control the autonomous vehicle in the lane of travel.

3. The method of claim 1 , wherein the first estimate comprises a first plurality of coordinate points corresponding to the boundary location, and
wherein the second estimate comprises a second plurality of coordinate points corresponding to the boundary location.

4. The method of claim 3 , wherein the first variance is determined further based on one or more of (i) an operational state of the at least one sensor, (ii) a distance between the first plurality of coordinate points and the at least one sensor, and (iii) first sensor data indicative of environmental conditions present at the first time, and
wherein the second variance is determined further based on one or more of (i) the operational state of the at least one sensor, (ii) a distance between the second plurality of coordinate points and the at least one sensor, and (iii) second sensor data indicative of environmental conditions present at the second time.

5. The method of claim 1 , wherein the first estimate comprises a first image portion, and
wherein the second estimate comprises a second image portion.

6. The method of claim 1 , wherein the first estimate comprises a first plurality of coordinate points corresponding to the boundary location,
wherein the second estimate comprises a second plurality of coordinate points corresponding to the boundary location,
wherein the combined estimate comprises a third plurality of coordinate points corresponding to the boundary location,
wherein the first variance comprises, for each coordinate point of the first plurality of coordinate points, an associated respective variance value that indicates how accurately the coordinate point corresponds to the boundary location,
wherein the second variance comprises, for each coordinate point of the second plurality of coordinate points, an associated respective variance value that indicates how accurately the coordinate point corresponds to the boundary location, and
wherein the combined variance comprises, for each coordinate point of the third plurality of coordinate points, an associated respective variance value that indicates how accurately the coordinate point corresponds to the boundary location.

7. The method of claim 6 , wherein the each of the first, second, and third pluralities of coordinate points is a plurality of two-dimensional (2D) coordinate points.

8. An autonomous vehicle comprising:
at least one sensor configured to sense a lane of travel in an environment of the autonomous vehicle, wherein the autonomous vehicle is operating in the lane of travel in the environment;
at least one processor; and
data storage comprising instructions executable by the at least one processor to cause the system to perform operations comprising:
receiving, via the at least one sensor, first data including a first estimate of a boundary location of a boundary of the lane of travel, the first data having been acquired at a first time;
based on the first data, determining a first variance in the first estimate of the boundary location;
receiving, via the at least one sensor, second data including a second estimate of the boundary location, the second data having been acquired at a second time different from the first time;
based on the second data, determining a second variance in the second estimate of the boundary location; and
based on the first estimate, the first variance, the second estimate, and the second variance, determining third data including a combined estimate of the boundary location and further including a combined variance in the combined estimate.

9. The vehicle of claim 8 , wherein the operations further comprise:
based on the third data, providing instructions to control the autonomous vehicle in the lane of travel.

10. The vehicle of claim 8 , wherein the at least one sensor includes a first sensor and a second, different sensor,
wherein the first data is received via the first sensor, and
wherein the second data is received via the second sensor.

11. The vehicle of claim 8 , wherein the first estimate comprises a first plurality of coordinate points corresponding to the boundary location,
wherein the second estimate comprises a second plurality of coordinate points corresponding to the boundary location,
wherein the first variance is determined further based on one or more of (i) an operational state of the at least one sensor, (ii) a distance between the first plurality of coordinate points and the at least one sensor, and (iii) first sensor data indicative of environmental conditions present at the first time, and
wherein the second variance is determined further based on one or more of (i) the operational state of the at least one sensor, (ii) a distance between the second plurality of coordinate points and the at least one sensor, and (iii) second sensor data indicative of environmental conditions present at the second time.

12. The vehicle of claim 8 , wherein one or more of the first estimate, the second estimate, and the combined estimate is based on a predetermined map of the environment, the predetermined map including the boundary location.

13. The vehicle of claim 8 ,
wherein the first estimate comprises a first plurality of coordinate points corresponding to the boundary location,
wherein the second estimate comprises a second plurality of coordinate points corresponding to the boundary location,
wherein the combined estimate comprises a third plurality of coordinate points corresponding to the boundary location,
wherein the first variance comprises, for each coordinate point of the first plurality of coordinate points, an associated respective variance value that indicates how accurately the coordinate point corresponds to the boundary location,
wherein the second variance comprises, for each coordinate point of the second plurality of coordinate points, an associated respective variance value that indicates how accurately the coordinate point corresponds to the boundary location, and
wherein the combined variance comprises, for each coordinate point of the third plurality of coordinate points, an associated respective variance value that indicates how accurately the coordinate point corresponds to the boundary location.

14. The vehicle of claim 8 , wherein determining the third data based on the first estimate, the first variance, the second estimate, and the second variance comprises:
making a comparison between one or both of the first and second variances and a pred

Assignees

• Google Inc
• Waymo Llc

Inventors

• Ferguson David Ian Franklin
• Silver David Harrison

Classifications

• G06V10/80

  Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level (multimodal speaker identification or verification G10L17/10) · CPC title

• G06V20/588

  Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road · CPC title

• G06F18/25

  Fusion techniques · CPC title

• G06T2207/30256

  Lane; Road marking · CPC title

• G06K9/209

  Physics · mapped topic