Method for displaying AR navigation screen, and AR navigation system

Patent metadata

Field	Value
Publication number	US-11715273-B2
Application number	US-201917756894-A
Country	US
Kind code	B2
Filing date	Dec 3, 2019
Priority date	Dec 3, 2019
Publication date	Aug 1, 2023
Grant date	Aug 1, 2023

Abstract

The present specification discloses a method for displaying an AR navigation screen, and an AR navigation system. The method for displaying an AR navigation screen according to the present specification may comprise the steps of: overlapping and displaying images of the surroundings of a vehicle and a plurality of AR images corresponding to the images of the surroundings; generating a window for separately displaying a first area including AR images overlapping at least one of the plurality of AR images; and overlapping and displaying the window and the images of the surroundings. The invention of the present specification has the effect of improving user convenience and promoting safe vehicle operation by using a window when AR images and the like are difficult to recognize due to being overlapped.

First claim

The invention claimed is:

1. A method of displaying an augmented reality (AR) navigation screen comprising:
displaying, on a display device, a surrounding image of a vehicle and a plurality of AR images related to the surrounding image in an overlapping manner;
generating a window separately displaying a first area including a particular AR image overlapping at least one of the plurality of AR images; and
displaying, on the display device, the window relative to the surrounding image in the overlapping manner,
wherein the generating of the window includes:
selecting AR images overlapping a vanishing line or vanishing point of the surrounding image from among the plurality of AR images;
setting the first area including an AR image overlapping at least one of the selected AR images; and
adjusting a size of the first area based on a size of the window.

2. The method of claim 1 , wherein the plurality of AR images are respectively related to a plurality of objects displayed on the surrounding image and respectively displayed at positions corresponding to coordinates of the plurality of objects.

3. The method of claim 1 , wherein the displaying of the plurality of AR images includes:
distinguishing a plurality of objects displayed on the surrounding image;
extracting coordinate information of each of the plurality of objects;
transmitting the coordinate information to an external server through wireless communication;
generating the plurality of AR images related to the coordinate information in the external server; and
transmitting the plurality of AR images to the vehicle through the wireless communication.

4. The method of claim 3 , wherein the external server and the vehicle transmit information using vehicle-to-everything (V2X) communication.

5. The method of claim 1 , wherein the adjusting includes:
calculating a scale ratio at which the size of the first area can be included in the size of the window; and
displaying the first area on the window based on the calculated scale ratio.

6. The method of claim 1 , wherein further comprising:
removing the window based on an overlap of the particular AR image being released.

7. The method of claim 1 , wherein further comprising:
removing the window based on a distance between the vehicle and a destination being less than or equal to a preset distance.

8. The method of claim 1 , wherein the generating of the window includes:
setting a plurality of second areas each of which includes the plurality of AR images; and
setting the first area including the particular AR image overlapping at least one of the plurality of second areas.

9. A method of displaying an augmented reality (AR) navigation screen comprising:
displaying, on a display device, a surrounding image of a vehicle and a plurality of AR images related to the surrounding image in an overlapping manner;
generating a window separately displaying a first area including a particular AR image overlapping at least one of the plurality of AR images; and
displaying, on the display device, the window relative to the surrounding image in the overlapping manner,
wherein the generating of the window includes:
converting the first area into a two-dimensional image; and
displaying the converted two-dimensional image on the window, and
wherein the displaying of the converted two-dimensional image on the window includes:
calculating a scale ratio at which a size of the converted two-dimensional image can be included in the window;
converting the converted two-dimensional image into a 2.5-dimensional image based on the calculated scale ratio being greater than a preset threshold; and
displaying the converted 2.5-dimensional image on the window.

10. A method of displaying an augmented reality (AR) navigation screen comprising:
displaying, on a display device, a surrounding image of a vehicle and a plurality of AR images related to the surrounding image in an overlapping manner;
generating a window separately displaying a first area including a particular AR image overlapping at least one of the plurality of AR images; and
displaying, on the display device, the window relative to the surrounding image in the overlapping manner,
wherein the window is positioned above a vanishing line or vanishing point of the surrounding image, and
wherein a width of the window is equal to or greater than a length of a line formed by connecting two points at which the vanishing line meets a leftmost solid line and a rightmost solid line, both boundaries of a road on which the vehicle is traveling, being displayed on the surrounding image as the leftmost solid line and the rightmost solid line.

11. The method of claim 10 , wherein further comprising:
based on the window overlapping the at least one of the plurality of AR images, deleting the at least one of the plurality of AR images overlapped with the window.

12. An augmented reality (AR) navigation system comprising:
a camera configured to photograph a surrounding image of a vehicle;
a position data generation device configured to generate current position data of the vehicle;
a processor configured to calculate coordinate information of each of a plurality of objects displayed on the surrounding image based on the current position data and the surrounding image;
an external server configured to receive the coordinate information through wireless communication and generate a plurality of AR images related to the coordinate information;
a communication device configured to receive the plurality of AR images from the external server through the wireless communication; and
a display device configured to display the surrounding image and the plurality of AR images,
wherein the external server generates a window separately displaying a first area including a particular AR image overlapping at least one of the plurality of AR images, and transmits the window to the communication device, and
the display device displays the surrounding image, the plurality of AR images, and the window in an overlapping manner, and
wherein the external server selects AR images overlapping a vanishing line or vanishing point of the surrounding image from among the plurality of AR images, sets the first area including an AR image overlapping at least one of the selected AR images, and adjusts a size of the first area based on a size of the window.

13. The AR navigation system of claim 12 , wherein the external server and the communication device transmit information using V2X vehicle-to-everything (V2X) communication.

14. The AR navigation system of claim 12 , wherein the external server removes the window based on an overlap of the particular AR image being released.

15. An augmented reality (AR) navigation system comprising:
a camera configured to photograph a surrounding image of a vehicle;
a position data generation device configured to generate current position data of the vehicle;
a processor configured to calculate coordinate information of each of a plurality of objects displayed on the surrounding image based on the current position data and the surrounding image;
an external server configured to receive the coordinate information through wireless communication and generate a plurality of AR images related to the coordinate information;
a communication device configured to receive the plurality of AR images from the external server through the wireless communication; and
a display device configured to display the surrounding image and the plurality of AR images,
wherein the external server generates a window separately displaying a first area incl

Assignees

• Lg Electronics Inc

Inventors

• LEE Kihyung
• Jung Junyoung
• Ji Aettie
• Jung Kyungjin

Classifications

• G06T19/006 Primary

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• G06T7/70

  Determining position or orientation of objects or cameras (camera calibration G06T7/80) · CPC title

• G06T19/20

  Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts · CPC title

• G01C21/3632

  Guidance using simplified or iconic instructions, e.g. using arrows (G01C21/365 takes precedence) · CPC title

• G06T2207/30252

  Vehicle exterior; Vicinity of vehicle · CPC title