Method for managing driving and electronic device

Patent metadata

Field	Value
Publication number	US-12530905-B2
Application number	US-202318096480-A
Country	US
Kind code	B2
Filing date	Jan 12, 2023
Priority date	Jun 22, 2022
Publication date	Jan 20, 2026
Grant date	Jan 20, 2026

Abstract

A method for managing driving applied in an electronic device which assesses distances to objects in a path of autonomous driving obtains RGB images of a scene in front of a vehicle, processes the RGB images based on a trained depth estimation model, and obtain depth images corresponding to the RGB images. The depth images are converted to 3D point cloud maps, 3D regions of interest from the 3D point cloud maps are determined according to a size of the vehicle, and the 3D regions of interest are converted into 2D regions of interest according to internal parameters of a camera. The 2D regions of interest are analyzed for obstacles. Driving continues when the 2D regions of interest have no obstacles, the vehicle is controlled to issue an alarm when obstacles are discovered.

First claim

What is claimed is:

1 . A method for managing driving comprising:
when detecting a vehicle is in a driving state, obtaining Red-Green-Blue (RGB) images of a scene in front of the vehicle; processing the RGB images based on a trained depth estimation model, and obtaining depth images corresponding to the RGB images; converting the depth images to three dimensional (3D) point cloud maps; determining 3D regions of interest of the vehicle from the 3D point cloud maps according to a size of the vehicle; converting the 3D regions of interest into 2D regions of interest according to internal parameters of a camera, comprising: obtaining a size of the vehicle, the size comprising a length, a width and a height of the vehicle; determining the 3D regions of interest of the vehicle from the 3D point cloud maps according to the length, the width and the height of the vehicle; determining whether the 2D regions of interest have obstacles; in response that the 2D regions of interest have no obstacles, controlling the vehicle to continue driving; in response that the 2D regions of interest have the obstacles, controlling the vehicle to issue an alarm.

2 . The method as claimed in claim 1 , further comprising:
converting the 3D regions of interest into the 2D regions of interest according to a formula of

z
[

x
1

y
1

1

]

=

K
⁢
B

=

[

f
x

0

c
x

0

f
y

c
y

0

0

1

]

[

x

y

z

]

,

in which (x1, y1) represents a coordinate of one 2D region of interest, K represents the internal parameters of the camera, B represents a coordinate (x, y, z) of one 3D region of interest.

3 . The method as claimed in claim 1 , wherein the depth estimation model comprises a depth estimation convolutional neural network and a pose estimation convolutional neural network.

4 . The method as claimed in claim 3 , further comprising:
training the depth estimation model, and obtaining the trained depth estimation model.

5 . The method as claimed in claim 4 , further comprising:
obtaining training images; inputting the training images into the depth estimation convolutional neural network, and obtaining the depth images corresponding to the training images; inputting adjacent frames of the training images into the pose estimation convolutional neural network, and obtaining a pose information of a camera corresponding to the adjacent frames; reconstructing the training images based on the depth images, the pose information of the camera, and internal parameters of the camera corresponding to the RGB images, and obtaining reconstructed images; calculating loss values between the training images and the reconstructed images by using a preset loss function; adjusting parameters of the depth estimation model to minimize the loss values, and obtaining the trained depth estimation model.

6 . The method as claimed in claim 5 , further comprising:
calculating the reconstructed images according to a second formula of P t+1 =K{circumflex over (T)} t→t+1 {circumflex over (D)}(P t )K −1 P t , in which P t+1 represents one reconstructed image, K represents the internal parameters of the camera, {circumflex over (T)} t→t+1 represents the pose information of the adjacent frames, {circumflex over (D)}(P t ) represents the depth value of a pixel coordinate, P t represents the pixel coordinate of the training images.

7 . The method as claimed in claim 1 , further comprising:
converting the depth images to the 3D point cloud maps according to a formula of

D
[

a
1

b
1

1

]

=

K
⁢
U

=

[

f
x

0

c
x

Assignees

• Hon Hai Prec Ind Co Ltd

Inventors

• Lee Chieh
• Yang Jung-Hao
• Chien Shih-Chao
• Kuo Chin-Pin

Classifications

• G06V10/25

  Determination of region of interest [ROI] or a volume of interest [VOI] · CPC title

• G06T2207/30244

  Camera pose · CPC title

• G06T2207/20081

  Training; Learning · CPC title

• G06T2207/20084

  Artificial neural networks [ANN] · CPC title

• G06T2207/30261

  Obstacle · CPC title