Method and apparatus for estimating depth information of images

Patent metadata

Field	Value
Publication number	US-12586221-B2
Application number	US-202318297396-A
Country	US
Kind code	B2
Filing date	Apr 7, 2023
Priority date	Apr 7, 2022
Publication date	Mar 24, 2026
Grant date	Mar 24, 2026

Abstract

A method and apparatus for estimating depth information of an image are disclosed. The depth information estimation method of the image includes providing a confidence map for a ground truth depth map; and learning a depth information estimation model for estimating depth information of an image based on the ground truth depth map and the confidence map.

First claim

What is claimed is:

1 . A method of estimating the depth information of an image, the method comprising:
providing, by an apparatus including a memory and a processor, a confidence map for a ground truth depth map; and learning, by the apparatus, a depth information estimation model that estimates depth information of an image based on the ground truth depth map and the confidence map, wherein the providing the confidence map includes:
filling, by the apparatus, an empty depth value of the ground truth depth map using a depth completion technique and providing the confidence map by generating the confidence map for the ground truth depth map filled with the empty depth value,
wherein the generating the confidence map includes:
configuring, by the apparatus, a confidence value to 1 for a depth value generated in a same pixel position as the ground truth depth map; and
generating, by the apparatus, the confidence map for the ground truth depth map filled with the empty depth value, by configuring the confidence value based on a distance from a number of pixels having a confidence value present in a certain radius for a depth value of a remaining pixel position.

2 . The method of claim 1 ,
wherein the providing the confidence map includes: generating, by the apparatus, the confidence map using a pre-learned confidence estimation model that inputs at least one image corresponding to the ground truth depth map and the ground truth depth map.

3 . The method of claim 1 ,
wherein the providing the confidence map includes: for a ground truth depth map with a predetermined ratio of the empty depth value among the ground truth depth map, filling, by the apparatus, the empty depth value using the depth completion technique.

4 . The method of claim 1 ,
wherein the learning the depth information estimation model includes: learning, by the apparatus, the depth information estimation model by reflecting the confidence map in a loss function.

5 . A method for estimating depth information of image, the method comprising:
obtaining, by an apparatus including a memory and a processor, a ground truth depth map; filling, by the apparatus, an empty depth value for each of the obtained ground truth depth map using a depth completion technique; generating, by the apparatus, a confidence map for the ground truth depth map filled with the empty depth value; and learning, by the apparatus, a depth information estimation model for estimating depth information of an image based on the ground truth depth map filled with the empty depth value and the confidence map, wherein the providing the confidence map includes:
configuring, by the apparatus, a confidence value to 1 for a depth value generated in a same pixel position as the ground truth depth map; and
generating, by the apparatus, the confidence map for the ground truth depth map filled with the empty depth value, by configuring a confidence value based on a distance from a number of pixels having a confidence value present in a certain radius for a depth value of a remaining pixel position.

6 . The method of claim 5 ,
wherein the generating the confidence map includes: generating the confidence map based on the ground truth depth map filled with the empty depth value and at least one image corresponding to the obtained ground truth depth map.

7 . The method of claim 6 , wherein:
the filling using the depth completion technique includes: for a ground truth depth map with a predetermined ratio of the empty depth value among the ground truth depth map, filling, by the apparatus, the empty depth value using the depth completion technique.

8 . An apparatus for estimating depth information of an image, the apparatus comprising:
a memory; a transceiver; and a processor, wherein the processor is configured to: provide a confidence map for a ground truth depth map; learn a depth information estimation model for estimating depth information of an image based on the ground truth depth map and the confidence map; and fill an empty depth value of the ground truth depth map using depth completion technology and provide the confidence map by generating the confidence map for the ground truth depth map filled with the empty depth value, wherein the processor is configured to:
configure a confidence value to 1 for a depth value generated in a same pixel position as the ground truth depth map; and
generate the confidence map for the ground truth depth map filled with the empty depth value, by configuring the confidence value based on a distance from a number of pixels having the confidence value present in a certain radius for a depth value of a remaining pixel position.

9 . The apparatus of claim 8 ,
wherein the processor is configured to: generate the confidence map using a pre-learned confidence estimation model that inputs at least one image corresponding to the ground truth depth map and the ground truth depth map.

10 . The apparatus of claim 8 ,
wherein the processor is configured to: for a ground truth depth map with a predetermined ratio of the empty depth value among the ground truth depth map, fill the empty depth value using the depth completion technology.

11 . The apparatus of claim 8 ,
wherein the processor is configured to: learn the depth information estimation model by reflecting the confidence map in a loss function.

Assignees

• Electronics & Telecommunications Res Inst
• Univ Indiana Trustees

Inventors

• Jung Soon-Heung
• Crandall David
• Vats Vibhas Kumar
• Shubham Shaurya
• Reza Md Alimoor
• Wang Chuhua

Classifications

• G06V10/761

  Proximity, similarity or dissimilarity measures · CPC title

• G06T2207/20081

  Training; Learning · CPC title

• G06T2207/10028

  Range image; Depth image; 3D point clouds · CPC title

• G06V10/778

  Active pattern-learning, e.g. online learning of image or video features · CPC title

• G06T7/50 Primary

  Depth or shape recovery · CPC title