Method and apparatus for generating three-dimensional (3D) road model

Patent metadata

Field	Value
Publication number	US-11632536-B2
Application number	US-202017091535-A
Country	US
Kind code	B2
Filing date	Nov 6, 2020
Priority date	Nov 7, 2016
Publication date	Apr 18, 2023
Grant date	Apr 18, 2023

Abstract

A method for generating a three-dimensional (3D) lane model, the method including calculating a free space indicating a driving-allowed area based on a driving image captured from a vehicle camera, generating a dominant plane indicating plane information of a road based on either or both of depth information of the free space and a depth map corresponding to a front of the vehicle, and generating a 3D short-distance road model based on the dominant plane.

First claim

What is claimed is:

1. A method of generating a three-dimensional (3D) lane model, the method comprising:
generating a dominant plane indicating plane information of a road based on a depth information of a driving-allowed area of a vehicle on a driving image and a depth map corresponding to a front of the vehicle;
generating road shape information of the road based on travel route information of the vehicle acquired from map information;
generating a 3D short-distance road model based on the dominant plane;
acquiring height information from the dominant plane; and
generating a 3D long-distance road model by projecting a result of matching the height information, the road shape information, and the 3D short-distance road model onto the dominant plane.

2. The method of claim 1 , wherein the driving-allowed area is acquired by:
acquiring the driving image from a camera included in the vehicle;
generating a depth map including depth information corresponding to a visual field criterion of the front of the vehicle; and
calculating the driving-allowed area based on the driving image and the depth map.

3. The method of claim 1 , wherein the generating of the road shape information further comprises:
extracting the road shape information from the driving image, the road shape information including any one or any combination of any two or more of a surface of the road, a lane marking on the road, and a center line of the road.

4. The method of claim 3 , wherein the generating of the 3D short-distance road model comprises:
generating 3D coordinates based on two-dimensional (2D) coordinates of vertices configuring the lane marking and depth information corresponding to the 2D coordinates of the vertices on the dominant plane;
generating 3D lane information by performing inverse projection on the generated 3D coordinates; and
generating the 3D short-distance road model based on the 3D lane information.

5. The method of claim 3 , wherein the extracting of the road shape information comprises:
acquiring location information including either one or both of current location information of the vehicle and direction information of the vehicle; and
extracting the road shape information currently viewed by a camera included in the vehicle based on the location information and camera information included in the driving image.

6. The method of claim 5 , wherein the acquiring of the location information comprises:
acquiring the travel route information based on a starting point and a destination of the vehicle from the map information; and
acquiring the location information including either one or both of the current location information of the vehicle and the direction information of the vehicle, based on the travel route information.

7. The method of claim 5 , wherein the extracting of the road shape information comprises extracting the road shape information entering a viewing frustum of the camera from the map information based on the location information and the camera information included in the driving image.

8. The method of claim 1 , wherein the acquiring the height information comprises:
acquiring the height information by projecting 2D coordinates obtained from the 2D map to the dominant plane.

9. The method of claim 1 , wherein the matching comprises matching the road shape information and the 3D short-distance road model using a least square method (LSM).

10. The method of claim 1 , wherein the generating of the dominant plane comprises generating the dominant plane based on a depth map corresponding to a front of the vehicle.

11. A non-transitory computer-readable storage medium storing instructions that, when executed by one or more processors, cause the one or more processors to perform the method of claim 1 .

12. An apparatus for generating a three-dimensional (3D) road model, the apparatus comprising:
a camera configured to capture a driving image from a vehicle; and
one or more processors configured to:
generate a dominant plane indicating plane information of a road based on a depth information of a driving-allowed area of a vehicle on a driving image and a depth map corresponding to a front of the vehicle;
generate road shape information of the road based on travel route information of the vehicle acquired from map information;
generate a 3D short-distance road model based on the dominant plane;
acquire height information from the dominant plane; and
generate a 3D long-distance road model by projecting a result of matching the height information, the road shape information, and the 3D short-distance road model onto the dominant plane.

13. The apparatus of claim 12 , wherein the apparatus comprises at least one of a head-up display (HUD) apparatus, a three-dimensional (3D) digital information display (DID), a navigation apparatus, a 3D mobile apparatus, a smartphone, a smart television (TV), and a smart vehicle.

Assignees

• Samsung Electronics Co Ltd

Inventors

• Jung Hwiryong
• Sung Young Hun
• Lee Keechang
• Jung Kyungboo

Classifications

• G01C21/3848

  Data obtained from both position sensors and additional sensors · CPC title

• G01C21/3867

  Geometry of map features, e.g. shape points, polygons or for simplified maps · CPC title

• G01C21/3819

  Road shape data, e.g. outline of a route · CPC title

• H04N13/271 Primary

  wherein the generated image signals comprise depth maps or disparity maps · CPC title

• G01C21/3602

  Input other than that of destination using image analysis, e.g. detection of road signs, lanes, buildings, real preceding vehicles using a camera · CPC title