Artificial intelligence modeling techniques for vision-based occupancy determination

Patent metadata

Field	Value
Publication number	US-12469160-B2
Application number	US-202418440764-A
Country	US
Kind code	B2
Filing date	Feb 13, 2024
Priority date	Sep 9, 2022
Publication date	Nov 11, 2025
Grant date	Nov 11, 2025

Abstract

Disclosed herein are methods and systems for using artificial intelligence modeling techniques to train and execute an artificial intelligence model to analyze camera feed received from an ego to generate an occupancy data indicating whether different voxels within the ego's surroundings are occupied by an object having mass. A method comprises inputting, using a camera of an ego object, image data of a space around the ego object into an artificial intelligence model; predicting, by executing the artificial intelligence model, an occupancy attribute of a plurality of voxels; and generating a dataset based on the plurality of voxels and their corresponding occupancy attribute.

First claim

What is claimed is:

1 . A method for generating a three-dimensional occupancy grid of a space around an ego object based on two-dimensional visual data, the method comprising:
inputting, by a processor using a camera of an ego object configured to navigate a space, periodically captured two-dimensional (2D) visual data from the camera of the space around the ego object into an artificial intelligence model to cause the artificial intelligence model to generate an output using image data comprising only the 2D visual data from the camera; periodically predicting, by the processor executing the artificial intelligence model using only the captured 2D visual data from the camera, an occupancy attribute of 3D occupancy data; and generating, by the processor using only the captured 2D visual data from the camera, a dataset based on the 3D occupancy data and their corresponding occupancy attribute.

2 . The method of claim 1 , further comprising:
generating, by the processor, an output representing an environment of the ego object and illustrating the 3D occupancy data and corresponding occupancy attributes, wherein the output comprises a graphical indicator of the occupancy attribute for at least a portion of the 3D occupancy data.

3 . The method of claim 2 , wherein the graphical indicator corresponds to a detected object associated with the at least the portion of the plurality of vexels 3D occupancy data.

4 . The method of claim 2 , further comprising:
displaying, by the processor, the output on a screen associated with the ego object.

5 . The method of claim 1 , wherein the dataset is a queryable dataset configured to transmit the occupancy attribute of the 3D occupancy data to an autonomous driving protocol of the ego object.

6 . The method of claim 1 , wherein the artificial intelligence model is trained using a sensor attribute of the 3D occupancy data.

7 . The method of claim 1 , wherein the ego object is an autonomous vehicle executing a driving protocol based on the dataset.

8 . The method of claim 1 , further comprising:
featurizing, by the processor, the 2D visual data prior to executing the artificial intelligence model.

9 . The method of claim 1 , wherein the 2D visual data comprises a plurality of camera feeds from a plurality of cameras of the ego object, the method further comprising:
temporally aligning, by the processor, the plurality of camera feeds.

10 . An ego object comprising:
a camera; a first processor; a second processor; a non-transitory computer-readable medium containing an artificial intelligence model configured to be executed by the first processor, wherein the first processor is configured to:
input, using the camera of the ego object configured to navigate a space, periodically captured two-dimensional (2D) visual data from the camera of the space around the ego object into the artificial intelligence model to cause the artificial intelligence model to generate an output using image data comprising only the 2D visual data from the camera;
periodically predict, executing the artificial intelligence model using only the captured 2D visual data from the camera, an occupancy attribute of a plurality of voxels 3D occupancy data; and
generate, using only the captured 2D visual data from the camera, a dataset based on the 3D occupancy data and their corresponding occupancy attribute,
wherein the second processor is configured to:
autonomously navigate the ego object using the dataset.

11 . The ego object of claim 10 , wherein the first processor is further configured to:
generate an output representing an environment of the ego object and illustrating the 3D occupancy data and their corresponding occupancy attribute, wherein the output comprises a graphical indicator of the occupancy attribute for at least a portion of the 3D occupancy data.

12 . The ego object of claim 11 , wherein the graphical indicator corresponds to a detected object associated with the at least the portion of the 3D occupancy data.

13 . The ego object of claim 11 , wherein the first processor is further configured to:
display the output on a screen associated with the ego object.

14 . The ego object of claim 10 , wherein the artificial intelligence model is trained using a sensor attribute of the 3D occupancy data.

15 . The ego object of claim 10 , wherein the ego object is an autonomous vehicle executing a driving protocol based on the dataset.

16 . A method comprising:
training, by a processor, an artificial intelligence model using a training dataset comprising first two-dimensional (2D) visual data received from a camera of an ego object, the training dataset having a first set of data points where each data point within the set of data points corresponds to a location and an image attribute of 3D occupancy data of space around the ego object, whereby the artificial intelligence model correlates each data point within the first set of data points with a corresponding data point within a second set of data points using locations for each data point, whereby, when the artificial intelligence model is trained, the artificial intelligence model is configured to receive a camera feed comprising second 2D visual data from a second ego object configured to navigate a space and periodically predict, using only the second 2D visual data, a third set of data points using only the camera feed, where each data point within the third set of data points corresponds an occupancy attribute indicating whether at least a portion of the 3D occupancy data of space around the second ego object is occupied by any object having mass.

17 . The method of claim 16 , wherein the artificial intelligence model is further configured to generate an output representing an environment of the ego object and illustrating the 3D occupancy data and their corresponding occupancy attribute.

18 . The method of claim 16 , wherein the training dataset further comprises a second set of data points where each data point within the second set of data points corresponds to the location and a sensor attribute of 3D occupancy data of the space around the ego object.

19 . The method of claim 17 , wherein a graphical indicator corresponds to a detected object associated with at least portion of the 3D occupancy data.

20 . The method of claim 17 , wherein the artificial intelligence model uses a three-dimensional multiview reconstruction protocol to generate the output.

Assignees

• Tesla Inc

Inventors

• Duan Pengfei
• Desai Nishant
• Lee Philip
• Elluswamy Ashok

Classifications

• G06T2207/30252

  Vehicle exterior; Vicinity of vehicle · CPC title

• G06T2207/20081

  Training; Learning · CPC title

• G06T15/08

  Volume rendering · CPC title

• B60W2420/403

  Image sensing, e.g. optical camera · CPC title

• B60W60/001

  Planning or execution of driving tasks · CPC title