Object velocity and/or yaw rate detection and tracking

Patent metadata

Field	Value
Publication number	US-2021237761-A1
Application number	US-202016866865-A
Country	US
Kind code	A1
Filing date	May 5, 2020
Priority date	Jan 31, 2020
Publication date	Aug 5, 2021
Grant date	—

Abstract

Tracking a current and/or previous position, velocity, acceleration, and/or heading of an object using sensor data may comprise determining whether to associate a current object detection generated from recently received (e.g., current) sensor data with a previous object detection generated from formerly received sensor data. In other words, a track may identify that an object detected in former sensor data is the same object detected in current sensor data. However, multiple types of sensor data may be used to detect objects and some objects may not be detected by different sensor types or may be detected differently, which may confound attempts to track an object. An ML model may be trained to receive outputs associated with different sensor types and/or a track associated with an object, and determine a data structure comprising a region of interest, object classification, and/or a pose associated with the object.

First claim

What is claimed is:

1 . A method comprising:
receiving a first object detection associated with a first sensor type and a second object detection associated with a second sensor type, the first object detection and the second object detection identifying an object in an environment surrounding an autonomous vehicle; receiving a track associated with the object, the track identifying at least one of an estimated previous position of the object, a previous region of interest, or a previous velocity of the object; inputting the first object detection, the second object detection, and at least part of the track into a machine-learning (ML) model; receiving, from the ML model, a region of interest associated with the object and a plurality of velocities associated with the object, a velocity of the plurality of velocities associated with a portion of the object; determining, based at least in part on the plurality of velocities, an estimated velocity and an estimated yaw rate associated with the object; determining an updated track associated with the object based at least in part on the region of interest, the updated track comprising at least a portion of the track and at least one of the estimated velocity or the estimated yaw rate; and controlling the autonomous vehicle based at least in part on the updated track.

2 . The method of claim 1 , further comprising receiving, from the ML model, at least one of:
an indication that the object is stationary or dynamic, a top-down segmentation of the environment, a current position of the object, or an acceleration associated with the object.

3 . The method of claim 1 , wherein determining the estimated velocity and the estimated yaw rate comprises performing a regression over the velocities according to a system of equations.

4 . The method of claim 1 , further comprising:
receiving ground truth data indicating a ground truth velocity associated with a pixel; determining a difference between the ground truth velocity and the velocity; and altering one or more parameters of the ML model to reduce the difference.

5 . The method of claim 1 , wherein inputting the first object detection, the second object detection, and at least part of the track comprises:
generating a multi-channel image based at least in part on the first object detection, the second object detection, and at least part of the track; and inputting the multi-channel image to the ML model.

6 . The method of claim 1 , wherein the velocity of the plurality of velocities is associated with a pixel of an image and the plurality of velocities is associated with different pixels of the image.

7 . The method of claim 1 , wherein:
the first object detection is an output of a first perception pipeline; and the second object detection is an output of a second perception pipeline.

8 . A system comprising:
one or more processors; and a memory storing processor-executable instructions that, when executed by the one or more processors, cause the system to perform operations comprising:
receiving a first object detection associated with a first sensor type and a second object detection associated with a second sensor type, the first object detection and the second object detection identifying an object in an environment surrounding an autonomous vehicle;
receiving a track associated with the object, the track identifying at least one of an estimated previous position of the object, a previous region of interest, or a previous velocity of the object;
inputting the first object detection, the second object detection, and at least part of the track into a machine-learning (ML) model;
receiving, from the ML model, a region of interest associated with the object and a plurality of velocities associated with the object, a velocity of the plurality of velocities associated with a portion of the object;
determining, based at least in part on the plurality of velocities, an estimated velocity and an estimated yaw rate associated with the object;
determining an updated track associated with the object based at least in part on the region of interest, the updated track comprising at least a portion of the track and at least one of the estimated velocity or the estimated yaw rate; and
controlling the autonomous vehicle based at least in part on the updated track.

9 . The system of claim 8 , wherein the operations further comprise receiving, from the ML model, at least one of:
an indication that the object is stationary or dynamic, a top-down segmentation of the environment, a current position of the object, or an acceleration associated with the object.

10 . The system of claim 8 , wherein inputting the first object detection, the second object detection, and at least part of the track comprises:
generating a multi-channel image based at least in part on the first object detection, the second object detection, and at least part of the track; and inputting the multi-channel image to the ML model.

11 . The system of claim 8 , wherein the velocity of the plurality of velocities is associated with a pixel of an image and the plurality of velocities is associated with different pixels of the image.

12 . The system of claim 8 , wherein:
the first object detection is an output of a first perception pipeline; and the second object detection is an output of a second perception pipeline.

13 . The system of claim 8 , wherein at least one of the first object detection or the second object detection comprises at least one of:
a representation of the environment from a top-down perspective; an indication of a classification, location, region occupied by, or state of an object; a velocity, acceleration, yaw, or yaw rate associated with the object; a sensor data segmentation; or a representation of an occluded portion of the environment.

14 . The system of claim 8 , wherein:
the plurality of velocities are a subset of a set of velocities output by the ML model; the set of velocities are each associated with a covariance; and the method further comprises determining to output the plurality of velocities based at least in part on determining that the plurality of velocities are associated with covariances that meet or exceed a threshold covariance.

15 . A non-transitory computer-readable medium storing processor-executable instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:
receiving a first object detection associated with a first sensor type and a second object detection associated with a second sensor type, the first object detection and the second object detection identifying an object in an environment surrounding an autonomous vehicle; receiving a track associated with the object, the track identifying at least one of an estimated previous position of the object, a previous region of interest, or a previous velocity of the object; inputting the first object detection, the second object detection, and at least part of the track into a machine-learning (ML) model; receiving, from the ML model, a region of interest associated with the object and a plurality of velocities associated with the object, a velocity of the plurality of velocities associated with a portion of the object; determining, based at least in part on the plurality of velocities, an estimated velocity and an estimated yaw rate associated with the object; determining an updated track associated with the object based at least in part on the region of interest, the updated track comprising at least a portion of

Assignees

• Zoox Inc

Inventors

• Das Subhasis
• Yu Kai
• Zwiebel Benjamin Isaac

Classifications

• G06T7/248 Primary

  involving reference images or patches · CPC title

• G06V10/25

  Determination of region of interest [ROI] or a volume of interest [VOI] · CPC title

• B60W60/001 Primary

  Planning or execution of driving tasks · CPC title

• G06F18/25

  Fusion techniques · CPC title

• B60W60/00272

  relying on extrapolation of current movement · CPC title