System and method for training and utilizing an object-centric occupancy estimation model

Patent metadata

Field	Value
Publication number	US-12597230-B2
Application number	US-202418425062-A
Country	US
Kind code	B2
Filing date	Jan 29, 2024
Priority date	Jan 29, 2024
Publication date	Apr 7, 2026
Grant date	Apr 7, 2026

Abstract

A method of updating parameters of an occupancy estimation model. The method includes receiving images that are two-dimensional. An occupancy estimation model is utilized to generate a voxel based on the images. An occupancy loss is determined by comparing the voxel to an occupancy ground truth corresponding to the voxel. An object loss is determined by comparing the voxel to an object ground truth. The object loss is combined with the occupancy loss to determine a total loss for the voxel. Parameters of the occupancy estimation model are updated to reduce the total loss determined.

First claim

What is claimed is:

1 . A method of updating parameters of an occupancy estimation model, the method comprising:
receiving a plurality of images, wherein the plurality of images are two-dimensional; utilizing the occupancy estimation model to generate a voxel based on the plurality of images; determining an occupancy loss by comparing the voxel to an occupancy ground truth corresponding to the voxel; determining an object loss by comparing the voxel to an object ground truth; combining the object loss with the occupancy loss to determine a total loss for the voxel; and updating parameters of the occupancy estimation model to reduce the total loss determined.

2 . The method of claim 1 , including applying a weight to the object loss when combined with the occupancy loss to determine the total loss.

3 . The method of claim 1 , wherein comparing the voxel to the occupancy ground truth is performed on a voxel-wise cross-entropy basis.

4 . The method of claim 1 , wherein the object loss is multiplied by a weight before being combined with the occupancy loss to determine the total loss.

5 . The method of claim 1 , wherein the object loss includes an object detection loss and utilizing an object detection model to identify detected objects in the voxel.

6 . The method of claim 5 , wherein the object detection loss includes comparing an object ground truth for the voxel to the detected objects.

7 . The method of claim 6 , wherein comparing the object ground truth for the voxel to the detected objects includes at least one of cross-entropy or shape regression.

8 . The method of claim 1 , wherein the object loss includes an object fullness loss and determining the object fullness loss includes comparing a set of occupied voxels defined within a bounding box from the object ground truth compared to a probability of occupancy estimated by the occupancy estimation model.

9 . A non-transitory computer-readable storage medium embodying programmed instructions which, when executed by a processor, are operable for performing a method comprising:
receiving a plurality of images, wherein the plurality of images are two-dimensional; utilizing an occupancy estimation model to generate a voxel based on the plurality of images; determining an occupancy loss by comparing the voxel to an occupancy ground truth corresponding to the voxel; determining an object loss by comparing the voxel to an object ground truth; combining the object loss with the occupancy loss to determine a total loss for the voxel; and updating parameters of the occupancy estimation model to reduce the total loss determined.

10 . The computer-readable storage medium of claim 9 , wherein the method includes applying a weight to the object loss when combining with the occupancy loss to determine the total loss.

11 . The computer-readable storage medium of claim 9 , wherein comparing the voxel to the occupancy ground truth is performed on a voxel-wise cross-entropy basis.

12 . The computer-readable storage medium of claim 9 , wherein the object loss is multiplied by a weight before being combined with the occupancy loss to determine the total loss.

13 . The computer-readable storage medium of claim 9 , wherein the object loss includes an object detection loss and utilizing an object detection model to identify detected objects in the voxel.

14 . The computer-readable storage medium of claim 13 , wherein the object detection loss includes comparing an object ground truth for the voxel to the detected objects.

15 . The computer-readable storage medium of claim 14 , wherein comparing the object ground truth for the voxel to the detected objects includes at least one of cross-entropy or shape regression.

16 . The computer-readable storage medium of claim 9 , wherein the object loss includes an object fullness loss and determining the object fullness loss includes comparing a set of occupied voxels defined within a bounding box from the object ground truth compared to a probability of occupancy estimated by the occupancy estimation model.

17 . A vehicle system comprising:
at least one optical sensor and a controller in data communication with the at least one optical sensor, wherein the controller configured to:
receive a plurality of images, wherein the plurality of images are two-dimensional optical images;
perform an occupancy estimation on the plurality of images with an occupation estimation model to generate a voxel;
perform a three-dimensional object detection on the plurality of images to identify at least one region associated with an object of interest; and
vary a probability threshold for identifying an object in the voxel based on the three-dimensional object detection, wherein the probability threshold for identifying objects is reduced for a portion of the voxel that corresponds to the at least one region associated with the object of interest when compared to the probability threshold for identifying the objects outside of the at least one region associated with the object of interest.

18 . The vehicle system of claim 17 , wherein the occupation estimation model is trained by:
receiving a plurality of training images, wherein the plurality of training images are two-dimensional; utilizing an occupancy estimation model to generate a voxel based on the plurality of training images; determining an occupancy loss by comparing the voxel to an occupancy ground truth corresponding to the voxel; determining an object loss by comparing the voxel to an object ground truth; combining the object loss with the occupancy loss to determine a total loss for the voxel; and updating parameters of the occupancy estimation model to reduce the total loss determined.

19 . The vehicle system of claim 18 , wherein
the object loss includes an object detection loss and utilizing an object detection model to identify detected objects in the voxel; the object detection loss includes comparing an object ground truth for the voxel to the detected objects; and comparing the object ground truth for the voxel to the detected objects includes at least one of cross-entropy or shape regression.

20 . The vehicle system of claim 18 , wherein the object loss includes an object fullness loss and determining the object fullness loss includes comparing a set of occupied voxels defined within a bounding box from the object ground truth compared to a probability of occupancy estimated by the occupancy estimation model.

Assignees

• Gm Global Tech Operations Llc

Inventors

• Baltaxe Michael
• Levi Dan

Classifications

• G06V20/58 Primary

  Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads · CPC title

• G06V10/751 Primary

  Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching · CPC title

• G06V10/82

  using neural networks · CPC title

• G06V10/774

  Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title

• G06V20/56 Primary

  exterior to a vehicle by using sensors mounted on the vehicle · CPC title