Articulated object determination

Patent metadata

Field	Value
Publication number	US-12406506-B2
Application number	US-202418651548-A
Country	US
Kind code	B2
Filing date	Apr 30, 2024
Priority date	Sep 30, 2021
Publication date	Sep 2, 2025
Grant date	Sep 2, 2025

Abstract

A vehicle computing system may implement techniques to determine whether two objects in an environment are related as an articulated object. The techniques may include applying heuristics and algorithms to object representations (e.g., bounding boxes) to determine whether two objects are related as a single object with two portions that articulate relative to each other. The techniques may include predicting future states of the articulated object in the environment. One or more model(s) may be used to determine presence of the articulated object and/or predict motion of the articulated object in the future. Based on the presence and/or motion of the articulated object, the vehicle computing system may control operation of the vehicle.

First claim

What is claimed is:

1. A system comprising:
one or more processors; and
one or more non-transitory computer-readable media storing instructions executable by the one or more processors, wherein the instructions, when executed, cause the system to perform operations comprising:
receiving sensor data from a sensor associated with a vehicle in an environment;
determining, based at least in part on the sensor data, a first representation of a first object in the environment and a second representation of a second object in the environment;
applying, by a model, one or more heuristics to the first representation and to the second representation; and
joining, by the model and based at least in part on the applying, the first object and the second object as an articulated object, wherein joining the first representation of the first object and the second representation of the second object as the articulated object is further based at least in part on a control policy comprising information identifying a right of way or a rule of an intersection associated with the first object and the second object in the environment.

2. The system of claim 1 , wherein:
the first representation or the second representation includes a top-down representation, and
the one or more heuristics comprises a size heuristic, a distance heuristic, or a joining point heuristic.

3. The system of claim 1 , wherein applying the one or more heuristics to the first representation and the second representation comprises:
determining a first size of the first representation and a second size of the second representation; and
comparing the first size or the second size to a size threshold,
wherein joining the first representation of the first object and the second representation of the second object is further based at least in part on the first size or the second size meeting or exceeding the size threshold.

4. The system of claim 1 , wherein applying the one or more heuristics to the first representation and the second representation comprises:
determining a distance between a first point associated with the first representation and a second point of the second representation; and
comparing the distance to a distance threshold,
wherein joining the first representation of the first object and the second representation of the second object is based at least in part on the distance being less than the distance threshold.

5. The system of claim 1 , the operations further comprising:
determining a predicted position of the articulated object at a future time; and
controlling the vehicle in the environment relative to the articulated object at least in part by determining a planned trajectory for the vehicle based at least in part on the predicted position of the articulated object.

6. A method comprising:
receiving sensor data from a sensor associated with a vehicle in an environment;
determining, based at least in part on the sensor data, a first representation of a first object in the environment and a second representation of a second object in the environment;
applying, by a model, one or more heuristics to the first representation and the second representation; and
joining, by the model and based at least in part on the applying, the first object and the second object as an articulated object, wherein joining the first representation of the first object and the second representation of the second object as the articulated object is further based at least in part on a control policy comprising information identifying a right of way or a rule of an intersection associated with the first object and the second object in the environment.

7. The method of claim 6 , wherein the model is a first model, and further comprising:
determining, by a second model and based at least in part on the sensor data, data comprising a top-down representation of the environment;
inputting the data into the first model; and
controlling the vehicle in the environment relative to the articulated object.

8. The method of claim 6 , wherein the one or more heuristics applied by the model comprises one or more of: a physical heuristic, a physics algorithm, or a linear algebra algorithm.

9. The method of claim 6 , wherein applying the one or more heuristics to the first representation and the second representation comprises:
determining a first size of the first representation and a second size of the second representation; and
comparing the first size or the second size to a size threshold,
wherein joining the first representation of the first object and the second representation of the second object is based at least in part on the first size or the second size meeting or exceeding the size threshold.

10. The method of claim 6 , wherein applying the one or more heuristics to the first representation and the second representation comprises:
determining a distance between a first point of the first representation and a second point of the second representation; and
comparing the distance to a distance threshold,
wherein joining the first representation of the first object and the second representation of the second object is based at least in part on the distance being less than the distance threshold.

11. The method of claim 6 , further comprising:
determining a predicted position of the articulated object at a future time; and
determining a planned trajectory for the vehicle based at least in part on the predicted position of the articulated object.

12. The method of claim 6 , wherein applying the one or more heuristics to the first representation and the second representation comprises:
performing at least one of:
determining that the first representation and the second representation overlap; or
determining that a first midline of the first representation intersects with a second midline of the second representation,
wherein joining the first representation of the first object and the second representation of the second object is based at least in part on the first representation and the second representation overlapping or the first midline and the second midline intersecting.

13. The method of claim 6 , wherein the joining is associated with a first time, the method further comprising:
receiving additional sensor data from the sensor at a second time after the first time;
applying, by the model, the one or more heuristics to the first representation and the second representation of the articulated object at the second time; and
disjoining the first object from the second object.

14. The method of claim 6 , wherein:
the first representation of the first object is a first shape having a first boundary,
the second representation of the second object is a second shape having a second boundary, and
the first shape or the second shape includes two dimensions or three-dimensions.

15. The method of claim 6 , further comprising:
determining a first object type of the first object and a second object type of the second object, the first object type or the second object type including at least one of: a car, a truck, a trailer, or a boat; and
comparing, as a comparison, the first object type and the second object type,
wherein joining the first representation of the first object and the second representation of the second object as the articulated object is further based at least in part on the comparison.

16. One or more non transitory computer readable media storing instructions executable by one or more processors, wherein the instructions, when executed, cause the one or more processors to perform

Assignees

• Zoox Inc

Inventors

• Costantino Adrian Michael
• Das Subhasis
• Papi Francesco

Classifications

• G06V20/17 Primary

  taken from planes or by drones · CPC title

• G06V20/58

  Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads · CPC title

• G06V20/64

  Three-dimensional [3D] objects · CPC title

• G06V20/56 Primary

  exterior to a vehicle by using sensors mounted on the vehicle · CPC title