Depth estimation using a neural network

Patent metadata

Field	Value
Publication number	US-12450763-B2
Application number	US-201916714359-A
Country	US
Kind code	B2
Filing date	Dec 13, 2019
Priority date	Dec 13, 2019
Publication date	Oct 21, 2025
Grant date	Oct 21, 2025

Abstract

Apparatuses, systems, and techniques to identify object distance with one or more cameras. In at least one embodiment, one or more cameras capture at least two images, where one image is transformed to the other, and a neural network determines whether said object is in front of or behind a known distance, whereby an object's distance may be determined after a set of known distances are analyzed.

First claim

What is claimed is:

1. One or more processors, comprising:
circuitry to use one or more neural networks to estimate a distance of an object to a camera based, at least in part, on changes made to a first image of the object in order to substantially match a second image of the object, wherein the one or more neural networks further estimate the distance by determining a direction the object shifts in the first image after the first image is changed.

2. The one or more processors of claim 1 , wherein the distance is estimated by:
changing the first image to substantially match the second image based on a point shared by the first image and the second image; and
determining, based on changes made to the first image, whether the object is in front of or beyond a depth plane of a plurality of available depth planes.

3. The one or more processors of claim 2 , wherein the distance is further estimated by:
selecting, if additional depth planes are available, a new depth plane based on a binary search of a plurality of available depth planes; and
changing the first image to substantially match the second image based on a new point shared by the first image and the second image, the point being on the new depth plane.

4. The one or more processors of claim 3 , wherein the plurality of available depth planes comprise depth planes at variable distances from the object depending on the distance to the object.

5. The one or more processors of claim 1 , wherein the first image and the second image are captured by a plurality of image capture devices.

6. The one or more processors of claim 1 , wherein the circuitry is to change the first image by transforming points in the first image to the second image by a homography.

7. The one or more processors of claim 1 , wherein the first image is captured from an initial camera position and the second image is captured from a secondary camera position.

8. A system, comprising:
one or more processors to use one or more neural networks to estimate a distance of an object to one or more image capture devices based, at least in part, on changes made to a first image of the object in order to substantially match a second image of the object, wherein the one or more neural networks further estimate the distance by determining a direction the object shifts in the first image after the first image is changed.

9. The system of claim 8 , further comprising:
memory containing instructions that, when executed, further cause the system to:
select a depth plane from a plurality of available depth planes;
determine, using the one or more neural networks, if the object is in front of or beyond the depth plane; and
indicate, based on whether the object is in front of or beyond the depth plane, that the depth plane defines a boundary for the object.

10. The system of claim 9 , wherein the one or more processors:
select, if additional depth planes are available, a new depth plane based on the plurality of available depth planes; and
change the first image to substantially match the second image based on a new point shared by the first image and the second image, the point being on the new depth plane.

11. The system of claim 10 , wherein the new depth plane is selected according to a binary search of the plurality of available depth planes.

12. The system of claim 9 , wherein the plurality of available depth planes comprise one or more depth planes that define the one or more available boundaries for the object.

13. The system of claim 8 , wherein the first image and the second image are captured by a plurality of image capture devices.

14. The system of claim 8 , wherein one or more processors change the first image by transforming all points in the first image to the second image by a homography, the homography relating two images of a planar surface captured at least in part by the first image and the second image.

15. One or more processors, comprising:
circuitry to help train one or more neural networks to estimate a distance of an object to a camera based, at least in part, on changes made to a first image of the object in order to substantially match a second image of the object, wherein the one or more neural networks further estimate the distance by determining a direction the object shifts in the first image after the first image is changed.

16. The one or more processors of claim 15 , wherein the estimated distance is determined by:
changing the first image to substantially match the second image based on a point shared by the first image and the second image, the point being on a depth plane of a plurality of available depth planes displayed in each of the first image and the second image;
determining, based on changes made to the first image, whether the object is in front of or beyond the depth plane; and
indicating, based on whether the object is in front of or beyond the depth plane, that the depth plane defines a boundary for the object.

17. The one or more processors of claim 16 , wherein the distance is further determined by:
selecting, if additional depth planes are available in the plurality of available depth planes, a new depth plane based on a search of a plurality of available depth planes; and
changing the first image to substantially match the second image based on a new point shared by the first image and the second image, the point being on the new depth plane.

18. The one or more processors of claim 16 , wherein the plurality of available depth planes are predetermined.

19. The one or more processors of claim 15 , wherein the first image and the second image are captured by a plurality of cameras.

20. The one or more processors of claim 15 , wherein the circuitry is to change the first image by transforming the first image to match the second image.

21. The one or more processors of claim 15 , wherein the first image is captured from an initial camera position and the second image is captured from a secondary camera position.

22. A method, comprising:
training one or more neural networks to estimate a distance of an object to a camera based, at least in part, on changes made to a first image of the object in order to substantially match a second image of the object, wherein the one or more neural networks further estimate the distance by determining a direction the object shifts in the first image after the first image is changed.

23. The method of claim 22 , further comprising:
capturing two or more images from one or more image capture devices;
selecting a depth plane from a plurality of available depth planes;
determining if the object is in front of the depth plane; and
generating, if the object is in front of the depth plane, an indication that the depth plane is an outer boundary for the object.

24. The method of claim 23 , further comprising generation of an indication, if the object is beyond the depth plane, that the depth plane is an inner boundary for the object.

25. The method of claim 23 , further comprising:
selecting a new depth plane from the plurality of available depth planes; and
determining a position of the object based on the new depth plane.

26. The method of claim 23 , wherein the depth plane is selected from the plurality of available depth planes according to a binary search.

27. The method of claim 22 , wherein the first and second images are captured by a plurality of camer

Assignees

• Nvidia Corp

Inventors

• Gallo Orazio
• Badki Abhishek
• Troccoli Alejandro

Classifications

• G06T2207/20084

  Artificial neural networks [ANN] · CPC title

• G06T2207/20081

  Training; Learning · CPC title

• G06T2207/10028

  Range image; Depth image; 3D point clouds · CPC title

• G06N3/08

  Learning methods · CPC title

• G06N3/04

  Architecture, e.g. interconnection topology · CPC title