Superimposition device of virtual guiding indication and reality image and the superimposition method thereof

Patent metadata

Field	Value
Publication number	US-10282915-B1
Application number	US-201715857230-A
Country	US
Kind code	B1
Filing date	Dec 28, 2017
Priority date	Dec 27, 2017
Publication date	May 7, 2019
Grant date	May 7, 2019

Abstract

A superimposition device of virtual guiding indication and reality image includes at least an image capturing device, a processor, a graphic processing unit (GPU), and a display device. The image capturing device captures reality image including instant scene. The processor receives the reality image and obtains height variation information. The GPU performs image correction processing on the reality image to obtain corrected image, generates updated transformation matrix according to the height variation information, and performs inverse perspective projection transformation by using the updated transformation matrix to generate bird's-eye view image of the corrected image and superimposes virtual guiding indication on the bird's-eye view image and performs perspective projection transformation on the bird's-eye view image to transform the bird's-eye view image into three-dimensional (3D) navigation image which includes the guiding indication. The display device displays the 3D navigation image which superimposes the virtual guiding indication into the reality image.

First claim

What is claimed is:

1. A superimposition device of a virtual guiding indication and a reality image, comprising:
at least one image capturing device deposited in front of a vehicle for capturing a reality image including a real-time scene,
a processor connected to the image capturing device for receiving the reality image and obtaining height variation information, wherein the height variation information includes a current altitude data of the vehicle and an altitude data for the road in front of the vehicle;
a graphic processing unit (GPU) connected to the processor for performing an image correction processing on the reality image to obtain a corrected image, generating an updated transformation matrix according to the height variation information, performing an inverse perspective projection transformation on the corrected image using the updated transformation matrix to generate a bird's-eye view image of the corrected image, generating a virtual guiding indication according to a route planning information, and superimposing the virtual guiding indication on the bird's-eye view image and then performing a perspective projection transformation on the bird's-eye view image to transform the bird's-eye view image into a three-dimensional (3D) navigation image, wherein the 3D navigation image includes the virtual guiding indication; and
a display device connected to the processor for displaying the 3D navigation image.

2. The superimposition device as claimed in claim 1 , further comprising:
a global positioning system (GPS) unit, providing and outputting a real-time location of the vehicle;
a geographic information system (GIS) unit, storing map information; and
a height information unit, providing the current altitude data of the vehicle and the altitude data for the road in front of the vehicle.

3. The superimposition device as claimed in claim 2 , wherein the height information unit is an altimeter.

4. The superimposition device as claimed in claim 2 , wherein height information provided by the GIS unit is compared with height information provided by the GPS unit to obtain the height variation information that represents a gradient of the road traffic ahead.

5. The superimposition device as claimed in claim 1 , wherein the virtual guiding indication is a 3D indication which is in equally proportional changes with distance and height variation and translucently attached to the ground.

6. The superimposition device as claimed in claim 1 , wherein the display device is a head-up display (HUD) device disposed next to the driving position of the vehicle to display the 3D navigation image that includes information indicating road traffic ahead superimposed with the virtual guiding indication.

7. The superimposition device as claimed in claim 1 , wherein the virtual guiding indication generated by the GPU is performed with the inverse perspective projection transformation according to the updated transformation matrix to correspond to an image coordinate system of the bird's-eye view image with the height variation, and to correspond to an image coordinate system of the reality image using the perspective projection transformation so as to superimpose the virtual guiding indication on the reality image.

8. The superimposition device as claimed in claim 1 , wherein the display device further comprises a user interface for displaying the current altitude data of the vehicle and the altitude data for the road ahead image.

9. The superimposition device as claimed in claim 1 , wherein the GPU performs the inverse perspective projection transformation using an inverse perspective projection algorithm that takes into account a height variation to generate the virtual guiding indication in the reality image.

10. The superimposition device as claimed in claim 1 , wherein the virtual guiding indication has a proportional relation with a driving lane that the vehicle is driving in within the reality image.

11. A superimposition method for superimposing a virtual guiding indication and a reality image applied to a superimposition device deposited on a vehicle and implemented by the superimposition device, the method comprising:
capturing a reality image including a real-time scene and obtaining height variation information, wherein the height variation information includes a current altitude data of the superimposition device and an altitude data for the road in front of the vehicle;
performing an image correction processing on the reality image to obtain a corrected image;
generating an updated transformation matrix according to the height variation information and performing an inverse perspective projection transformation on the corrected image using the updated transformation matrix to generate a bird's-eye view image of the corrected image;
generating a virtual guiding indication according to a route planning information and superimposing the virtual guiding indication on the bird's-eye view image; and
performing a perspective projection transformation on the bird's-eye view image to transform the bird's-eye view image into a three-dimensional (3D) navigation image, wherein the 3D navigation image includes the virtual guiding indication.

12. The superimposition method as claimed in claim 11 , wherein the corrected image is a non-distorted image and the correction processing includes a camera parameter correction processing and a distortion correction processing.

13. The superimposition method as claimed in claim 11 , further comprising:
receiving height information from a geographic information system (GIS) unit and height information from a global positioning system (GPS) unit; and
compared the height information provided by the GIS unit with the height information provided by the GPS unit to obtain the height variation information that represents a gradient of the road traffic ahead.

14. The superimposition method as claimed in claim 11 , wherein the virtual guiding indication is a 3D indication which is in equally proportional changes with distance and height variation and translucently attached to the ground.

15. The superimposition method as claimed in claim 11 , further comprising:
performing the inverse perspective projection transformation using an inverse perspective projection algorithm that takes into account a height variation to generate the virtual guiding indication in the reality image.

16. The superimposition method as claimed in claim 11 , further comprising:
displaying the 3D navigation image.

Assignees

• Ind Tech Res Inst

Inventors

• Lin Jing-Jhih
• Liu Wei-Cheng
• Lin Yu-Chen
• Wang Cheng-Hsien
• Chen Peng-Cheng

Classifications

• G06T5/50

  using two or more images, e.g. averaging or subtraction · CPC title

• G06T19/003

  Navigation within 3D models or images · CPC title

• G01C21/3602 Primary

  Input other than that of destination using image analysis, e.g. detection of road signs, lanes, buildings, real preceding vehicles using a camera · CPC title

• G06T3/4038

  Image mosaicing, e.g. composing plane images from plane sub-images · CPC title

• H04N5/265

  Mixing · CPC title