Method and apparatus for generating three-dimensional (3D) road model

Patent metadata

Field	Value
Publication number	US-10863166-B2
Application number	US-201715452476-A
Country	US
Kind code	B2
Filing date	Mar 7, 2017
Priority date	Nov 7, 2016
Publication date	Dec 8, 2020
Grant date	Dec 8, 2020

Abstract

A method for generating a three-dimensional (3D) lane model, the method including calculating a free space indicating a driving-allowed area based on a driving image captured from a vehicle camera, generating a dominant plane indicating plane information of a road based on either or both of depth information of the free space and a depth map corresponding to a front of the vehicle, and generating a 3D short-distance road model based on the dominant plane.

First claim

What is claimed is:

1. A method of generating a three-dimensional (3D) lane model, the method comprising:
calculating a free space indicating a driving-allowed area based on a driving image captured from a camera of a vehicle;
generating a dominant plane indicating plane information of a road based on a depth information of the free space;
generating road shape information of the road based on travel route information of the vehicle acquired from map information;
generating a 3D short-distance road model based on the dominant plane; and
generating, by matching the generated 3D short-distance road model to the travel route information and by expanding the 3D short-distance road model based on the road shape information, a 3D long-distance road model to indicate a second area of the road further from the vehicle than a first area of the road in front of the vehicle,
wherein the 3D short-distance road model indicates the first area of a road.

2. The method of claim 1 , wherein the calculating of the free space comprises:
acquiring the driving image from the camera included in the vehicle;
generating a depth map including depth information corresponding to a visual field criterion of the front of the vehicle; and
calculating the free space based on the driving image and the depth map.

3. The method of claim 2 , wherein the driving image includes any one or any combination of any two or more of camera information, a black-and-white image, and a color image captured by the camera.

4. The method of claim 2 , wherein the generating of the depth map comprises acquiring the depth information using any one or any combination of any two or more of a radar, a lidar, and a stereo matching method using a stereo camera.

5. The method of claim 1 , wherein the generating of the road shape information further comprises:
extracting the road shape information from the driving image, the road shape information including any one or any combination of any two or more of a surface of the road, a lane marking on the road, and a center line of the road.

6. The method of claim 5 , wherein the generating of the 3D short-distance road model comprises:
generating 3D coordinates based on two-dimensional (2D) coordinates of vertices configuring the lane marking and depth information corresponding to the 2D coordinates of the vertices on the dominant plane;
generating 3D lane information by performing inverse projection on the generated 3D coordinates; and
generating the 3D short-distance road model based on the 3D lane information.

7. The method of claim 5 , wherein the extracting of the road shape information comprises:
acquiring location information including either one or both of current location information of the vehicle and direction information of the vehicle; and
extracting the road shape information currently viewed by the camera based on the location information and camera information included in the driving image.

8. The method of claim 7 , wherein the acquiring of the location information comprises acquiring the location information using any one or any combination of any two or more of a global positioning system (GPS), an inertial measurement unit (IMU), and a visual odometry method.

9. The method of claim 7 , wherein the acquiring of the location information comprises:
acquiring the travel route information based on a starting point and a destination of the vehicle from the map information; and
acquiring the location information including either one or both of the current location information of the vehicle and the direction information of the vehicle, based on the travel route information.

10. The method of claim 9 , wherein the extracting of the road shape information comprises extracting the road shape information entering a viewing frustum of the camera from the map information based on the location information and the camera information included in the driving image.

11. The method of claim 10 , further comprising:
generating the 3D long-distance road model based on the road shape information and the 3D short-distance road model.

12. The method of claim 11 , wherein the generating of the 3D long-distance road model comprises:
matching the road shape information and the 3D short-distance road model; and
projecting a result of the matching onto the dominant plane for generating the 3D long-distance road model.

13. The method of claim 12 , wherein the matching comprises matching the road shape information and the 3D short-distance road model using a least square method (LSM).

14. The method of claim 11 , wherein the generating of the 3D long-distance road model comprises:
acquiring height information from the dominant plane; and
generating the 3D long-distance road model based on the height information, the road shape information, and the 3D short-distance road model.

15. A non-transitory computer-readable storage medium storing instructions that, when executed by one or more processors, cause the one or more processors to perform the method of claim 1 .

16. An apparatus for generating a three-dimensional (3D) road model, the apparatus comprising:
a camera configured to capture a driving image from a vehicle; and
one or more processors configured to:
calculate a free space indicating a driving-allowed area based on the driving image,
generate a dominant plane indicating plane information of a road based on a depth information of the free space,
generate road shape information of the road based on travel route information of the vehicle acquired from map information,
generate a 3D short-distance road model based on the dominant plane, and
generating, by matching the generated 3D short-distance road model to the travel route information and by expanding the 3D short-distance road model based on the road shape information, a 3D long-distance road model to indicate a second area of the road further from the vehicle than a first area of the road in front of the vehicle,

wherein the 3D short-distance road model indicates the first area of a road.

17. The apparatus of claim 16 , wherein the one or more processors are further configured to acquire the driving image from a camera included in the vehicle, generate a depth map including depth information corresponding to a visual field criterion of the front of the vehicle, and calculate the free space based on the driving image and the depth map.

18. An apparatus for generating a three-dimensional (3D) road model, the apparatus comprising:
a camera configured to capture a driving image from a vehicle; and
one or more processors configured to:
calculate a free space indicating a driving-allowed area based on the driving image,
generate a dominant plane indicating plane information of a road based on a depth information of the free space,
generate a 3D short-distance road model based on the dominant plane, and

generating a 3D long-distance road model by matching the generated 3D short-distance road model to map-based travel route information, wherein the one or more processors are further configured to:
extract road shape information including any one or any combination of any two or more of a surface of the road, a lane marking on the road, and a center line of the road from the driving image,
generate 3D coordinates based on two-dimensional (2D) coordinates of vertices configuring the lane marking and depth information corresponding to the 2D coordinates of the vertices on the dominant plane, and
generate the 3D short-distance road model based on 3D lane information generate

Assignees

• Samsung Electronics Co Ltd

Inventors

• Jung Hwiryong
• Sung Young Hun
• Lee Keechang
• Jung Kyungboo

Classifications

• G06V20/588

  Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road · CPC title

• G01C21/3822

  Road feature data, e.g. slope data · CPC title

• G01C21/3848

  Data obtained from both position sensors and additional sensors · CPC title

• G01C21/3867

  Geometry of map features, e.g. shape points, polygons or for simplified maps · CPC title

• G01C21/3819

  Road shape data, e.g. outline of a route · CPC title