Method and apparatus for providing augmented reality

Patent metadata

Field	Value
Publication number	US-9336630-B2
Application number	US-201314068547-A
Country	US
Kind code	B2
Filing date	Oct 31, 2013
Priority date	Dec 5, 2012
Publication date	May 10, 2016
Grant date	May 10, 2016

Abstract

A method and apparatus for providing augmented reality are provided and include a controller that is configured to match a head up display area of a windshield and an input image and determine an area in which an information amount is a minimum in the matched image as a position of a display window to display virtual information. In addition, the controller is configured to output virtual information at the determined position.

First claim

What is claimed is:

1. A method of providing augmented reality of an augmented reality providing apparatus mounted within a vehicle, the method comprising:
matching, by a controller, a head up display area of a windshield and an input image;
determining, by the controller, an area in which an information amount is a minimum in the matched image as a position of a display window to display virtual information; and
outputting, by the controller, virtual information at the determined position,
wherein:
(1) the determining of an area comprises:
determining, by the controller, whether a marker related to the virtual information is included in the matched image;
when the marker is included in the matched image, dividing, by the controller, an area in which virtual information may be displayed in the matched image into a plurality of sub-blocks;
calculating, by the controller, the information amount based on the plurality of sub-blocks;
searching, by the controller, for a position at which the information amount is the minimum based on the plurality of sub-blocks; and
determining, by the controller, the position at which the information amount is the minimum as a position of the display window; and
(2) the calculating of an information amount comprises:
acquiring, by the controller, included edges based on the plurality of sub-blocks;
counting, by the controller, the number of the edges;
calculating, by the controller, an average distance based on the plurality of sub-blocks;
normalizing, by the controller, the number of the edges; and
calculating, by the controller, the information amount based on the sub-block using the calculated average distance and the number of the normalized edges.

2. The method of claim 1 , wherein the searching of for a position includes:
generating, by the controller, an integral image based on the information amount; and
searching, by the controller, for a position at which the information amount is a first least amount using a window size as a search unit in the integral image.

3. The method of claim 2 ; wherein the outputting of virtual information includes:
when an already output virtual information display window exists at the position at which the information amount is the first least amount, outputting, by the controller, virtual information at a position at which an information amount is a second least amount;
when the already output virtual information display window exists at the position at which the information amount is the second least amount, determining, by the controller, whether to output the virtual information display window according to a priority of previously defined display information;
determining, by the controller, whether output space exists at the determined position;
when output space exists, outputting, by the controller, the virtual information display window according to the priority at the output space; and
when output space does not exist, searching for, by the controller, a periphery and outputting the virtual information display window according to the priority on the already output virtual information display window in which the priority is below a predetermined threshold in an area in which the information amount is the minimum.

4. The method of claim 3 , wherein the outputting the virtual information display window according to the priority on the already output virtual information display window includes:
determining, by the controller, whether an ‘OFF’ flag is listed in the overwrite field in the area in which the information amount is the minimum; and
when the ‘OFF’ flag is not listed in the overwrite field, outputting, by the controller, a virtual information display window according to the priority on the already output virtual information display window in which the priority is below the predetermined threshold in the area in which the information amount is the minimum.

5. An augmented reality providing apparatus mounted within a vehicle, comprising:
a controller configured to:
capture an input image positioned at a front side of a driver using an imaging device; match the input image and a head up display area of a windshield;
determine whether a marker is detected in a matched image;
determine an area in which an information amount is a minimum in the matched image as a position of a display window to display virtual information that corresponds to the marker, when the marker is detected; and
output an augmented reality screen in which virtual information is displayed at the determined position,
wherein the controller is further configured to:
divide an area in which virtual information is displayed in the matched image into a plurality of sub-blocks;
calculate the information amount based on the plurality of sub-blocks;
determine a position at which the information amount is the minimum in the plurality of sub-blocks as the position to display the virtual information;
normalize a number of edges based on the plurality of sub-block;
calculate the information amount based on the sub-block together with the sub-block average distance and the number of the normalized edges; and
search for the position at which the information amount is a first least amount using a size of a window as a search unit in a generated integral image based on the information amount.

6. The augmented reality providing apparatus of claim 5 , wherein the controller is further configured to:
output virtual information at the position at which the information amount is a second least amount, when an already output virtual information display window exists at the position at which the information amount is the first least amount;
determine whether to output the virtual information display window according to a priori of previously defined display information, when the already output virtual information display window exists at the position at which the information amount is the second least amount; and
search for a periphery, when output space does not exist, and output the virtual information display window according to the priority on the already output virtual information display window having a priority below a predetermined threshold in the area in which the information amount is the minimum.

7. A non-transitory computer readable medium containing program instructions executed by a controller, the computer readable medium comprising:
program instructions that control an imaging device to capture an input image positioned at a front side of a driver;
program instructions that match the input image and a head up display area of a windshield;
program instructions that determine whether a marker is detected in a matched image;
program instructions that determine an area in which an information amount is a minimum in the matched image as a position of a display window to display virtual information that corresponds to the marker, when the marker is detected;
program instructions that output an augmented reality screen in which virtual information is displayed at the determined position;
program instructions that divide an area in which virtual information is displayed in the matched image into a plurality of sub-blocks;
program instructions that calculate the information amount based on the plurality of sub-blocks; and
program instructions that determine a position at which the information amount is the minimum in the plurality of sub-blocks as the position to display the virtual information;
program instructions that normalize a number of edges based on the plurality of sub-block;
program instructions that calculate the information amount based on the sub-block together with the sub-block average distance and the number of the normalized edges; and
program instructions th

Assignees

• Hyundai Motor Co Ltd

Inventors

• Shin Dongsoo

Classifications

• G06T2207/30248

  Vehicle exterior or interior · CPC title

• G06T2207/20021

  Dividing image into blocks, subimages or windows · CPC title

• G02B27/01 Primary

  Head-up displays · CPC title

• G06T7/403

  Physics · mapped topic

• G02B2027/014

  comprising information/image processing systems · CPC title