Method and apparatus for displaying virtual route

Patent metadata

Field	Value
Publication number	US-10732004-B2
Application number	US-201815919432-A
Country	US
Kind code	B2
Filing date	Mar 13, 2018
Priority date	Nov 9, 2017
Publication date	Aug 4, 2020
Grant date	Aug 4, 2020

Abstract

A method and apparatus for displaying a virtual route estimates a position of a vehicle based on sensing data received asynchronously from sensors, and outputs a three-dimensional (3D) virtual route generated by registering a driving environment model corresponding to a driving environment of the vehicle and the position of the vehicle in map information.

First claim

What is claimed is:

1. A method of displaying a virtual route, the method comprising:
estimating a first position of a vehicle based on first sensing data received from a first sensor at a first time;
estimating a second position of the vehicle by correcting the estimated first position based on second sensing data received from a second sensor at a second time;
generating a three-dimensional (3D) virtual route by registering, in map information, a driving environment model corresponding to a driving environment of the vehicle and the first and second positions of the vehicle; and
outputting the 3D virtual route.

2. The method of claim 1 , wherein the first and second sensors comprise any one or any combination of a global positioning system (GPS) sensor, an inertial measurement unit (IMU) sensor, an on-board diagnostics (OBD) sensor, and a camera sensor.

3. The method of claim 1 , wherein the first and second sensing data are sampled separately for each of the first and second sensors.

4. The method of claim 1 , wherein the estimating comprises:
inserting, into each of the first and second sensing data, information related to a processing time at which the corresponding piece of the sensing data is processed; and
estimating the first and second positions of the vehicle based on the first and second sensing data, each of the estimated first and second positions including the information related to the processing time.

5. A method of displaying a virtual route, the method comprising:
estimating a position of a vehicle based on sensing data received asynchronously from sensors;
generating a three-dimensional (3D) virtual route by registering a driving environment model corresponding to a driving environment of the vehicle and the position of the vehicle in map information; and
outputting the 3D virtual route,
wherein the estimating comprises:
estimating a first position of the vehicle based on first sensing data received at a first time from a first sensor of the sensors;
updating the first position of the vehicle to a second position based on second sensing data received at a second time from the first sensor or a second sensor of the sensors; and
updating the second position of the vehicle to a third position based on third sensing data received at a third time from the first sensor, or the second sensor or a third sensor of the sensors.

6. The method of claim 1 , further comprising:
measuring an odometry based on image data acquired from a camera sensor among the sensors,
wherein the second sensing data includes the image data and the second sensor includes the camera sensor.

7. A method of displaying a virtual route, the method comprising:
estimating a position of a vehicle based on sensing data received asynchronously from sensors;
generating a three-dimensional (3D) virtual route by registering a driving environment model corresponding to a driving environment of the vehicle and the position of the vehicle in map information;
outputting the 3D virtual route; and
measuring an odometry based on image data acquired from a camera sensor among the sensors,
wherein the estimating comprises estimating the position of the vehicle based on the sensing data and the odometry, and
wherein the estimating based on the sensing data and the odometry comprises:
estimating the position of the vehicle based on one piece of the sensing data, except for the image data; and
correcting the position of the vehicle based on the odometry.

8. A method of displaying a virtual route, the method comprising:
estimating a position of a vehicle based on sensing data received asynchronously from sensors;
generating a three-dimensional (3D) virtual route by registering a driving environment model corresponding to a driving environment of the vehicle and the position of the vehicle in map information; and
outputting the 3D virtual route,
wherein the generating of the 3D virtual route comprises:
generating a segmentation image based on image data acquired from a camera sensor among the sensors;
detecting objects included in the segmentation image;
generating the driving environment model based on depth values of the objects and a driving lane of the vehicle identified from the objects; and
generating the 3D virtual route by registering the driving environment model and the position of the vehicle in the map information.

9. The method of claim 1 , wherein the generating of the 3D virtual route comprises:
estimating a lighting and a texture of a region including a driving lane of the vehicle based on image data acquired from a camera sensor among the sensors; and
generating the 3D virtual route by applying the lighting and the texture.

10. The method of claim 1 , further comprising:
transforming the 3D virtual route to match to a viewpoint of a driver of the vehicle,
wherein the outputting comprises displaying the transformed 3D virtual route.

11. The method of claim 10 , wherein the outputting of the transformed 3D virtual route comprises displaying the transformed 3D virtual route through a head-up display (HUD) of the vehicle.

12. The method of claim 10 , wherein the transforming comprises:
tracking the viewpoint of the driver by tracking 3D positions of both eyes of the driver;
predicting a transformation relation between the viewpoint of the driver and a virtual image displayed through the HUD based on the 3D positions of both the eyes of the user; and
transforming the 3D virtual route based on the transformation relation.

13. A non-transitory computer-readable storage medium storing instructions that, when executed by one or more processors, cause the one or more processors to perform the method of claim 1 .

14. An apparatus for displaying a virtual route, the apparatus comprising:
sensors including a first sensor configured to sense first sensing data of a vehicle and a second sensor configured to sense second sensing data of the vehicle;
one or more processors configured to:
estimate a first position of the vehicle based on the first sensing data received from the first sensor at a first time,
estimate a second position of the vehicle by correcting the estimated first position based on the second sensing data received from the second sensor at a second time, and
generate a three-dimensional (3D) virtual route by registering, in map information, a driving environment model corresponding to a driving environment of the vehicle and the first and second positions of the vehicle; and

a display configured to display the 3D virtual route.

15. The apparatus of claim 14 , wherein the sensors comprise any two or any combination of any three or more of:
a global positioning system (GPS) sensor configured to measure an absolute route of the vehicle;
an inertial measurement unit (IMU) sensor configured to measure a relative route of the vehicle;
an on-board diagnostics (OBD) sensor configured to measure a driving distance of the vehicle; and
a camera sensor configured to capture image data including the driving environment of the vehicle.

16. The apparatus of claim 15 , wherein the first sensing data and the second sensing data are sampled separately for each of the first sensor and the second sensor, respectively.

17. The apparatus of claim 14 , wherein the one or more processors are further configured to insert, into each of the first sensing data and the second sensing data, information related to a processing time at which a corresponding one of the first sensing data and the second sensing data is processed, and to estimate the first

Assignees

• Samsung Electronics Co Ltd

Inventors

• Jung Kyungboo
• Kang Nahyup
• Lee Keechang
• Hong Sunghoon

Classifications

• G01C21/3819

  Road shape data, e.g. outline of a route · CPC title

• G01C21/3867

  Geometry of map features, e.g. shape points, polygons or for simplified maps · CPC title

• G01C21/3848

  Data obtained from both position sensors and additional sensors · CPC title

• G01C21/188

  for accumulated errors, e.g. by coupling inertial systems with absolute positioning systems · CPC title

• G01C21/3658

  Lane guidance · CPC title