Method for operating a display device in a motor vehicle

Patent metadata

Field	Value
Publication number	US-11100718-B2
Application number	US-201816754544-A
Country	US
Kind code	B2
Filing date	Sep 12, 2018
Priority date	Oct 9, 2017
Publication date	Aug 24, 2021
Grant date	Aug 24, 2021

Abstract

Surroundings data, relating to at least one surrounding area in the surroundings of a motor vehicle situated in a field of vision of a vehicle occupant, are obtained by a sensor system of the motor vehicle. At least one piece of information, which needs to be output to the user, is provided by an information source. The information is at least partially overlaid on a display of the surrounding area by a display device with the type of information presentation and/or the position at, and/or the orientation in, which the information presentation is visualized being dependent on the surroundings data.

First claim

The invention claimed is:

1. A method for operating a display device in a motor vehicle, comprising:
capturing surroundings data relating to at least one surrounding area where another vehicle is located, in surroundings of the motor vehicle, situated in a field of vision of a user, by a sensor system of the motor vehicle;
ascertaining a three-dimensional surroundings model describing the surrounding area from the surroundings data;
providing at least one piece of information, to be output to the user, from an information source;
generating, based on the at least one piece of information and the surroundings model, one of a three-dimensional information object and a two-dimensional information object in at least one of a position and an orientation in a coordinate system prescribed for the three-dimensional surroundings model; and
visualizing an at least partially overlaid presentation of an information presentation reproducing the at least one piece of information by virtual projection of the one of the three-dimensional information object and the two-dimensional information object onto a surface of the other vehicle in the at least one surrounding area for the user by the display device, positioning of the information presentation and presentation orientation of the information presentation being dependent on the surroundings data.

2. The method according to claim 1 , wherein the display device is separate from the motor vehicle and secured to a head and in the field of vision of the user.

3. The method according to claim 2 , wherein the display device includes a position sensing device configured to detect at least one of a relative position and/or a device orientation of the display device in relation to at least one of a reference position and a reference direction, fixed relative to the motor vehicle, and a piece of reference information from which the relative position and/or the device orientation of the display device are ascertained.

4. The method according to claim 3 , wherein the display device is a video-based display device,
wherein the visualizing of the at least one surrounding area includes one of
capturing an image of the at least one surrounding area by one of the sensor system and a further sensor system, and displaying the image on the video-based display device with the information presentation overlaid thereon,
overlaying the information presentation via a semitransparent display device with at least part of the at least one surrounding area visible to the user through the semitransparent display device, and
projecting, via a virtual retinal display, the information presentation onto at least one retina of the user.

5. The method according to claim 4 ,
further comprising evaluating the surroundings data to detect at least one external object or at least one position, and
wherein the providing of the at least one piece of information is based on a property of the at least one external object or the position.

6. The method according to claim 5 ,
wherein the evaluating of the surroundings data results in the at least one external object being detected, and
wherein the method further comprises determining, based on at least one of an object position and an object orientation of the at least one external object, the at least one of the type of information presentation, the positioning of the information presentation and the presentation orientation of the information presentation visualized by the visualizing.

7. The method according to claim 6 , further comprising: providing at least one of the surroundings data and the surroundings model to a motor-vehicle-external device; and prescribing, by the motor-vehicle-external device, the at least one of the type of information presentation, the positioning of the information presentation and the presentation orientation of the information presentation visualized by the visualizing.

8. The method according to claim 1 , wherein the display device is a video-based display device,
wherein the visualizing of the at least one surrounding area includes one of
capturing an image of the at least one surrounding area by one of the sensor system and a further sensor system, and displaying the image on the video-based display device with the information presentation overlaid thereon,
overlaying the information presentation via a semitransparent display device with at least part of the at least one surrounding area visible to the user through the semitransparent display device, and
projecting, via a virtual retinal display, the information presentation onto at least one retina of the user.

9. The method according to claim 1 ,
further comprising evaluating the surroundings data to detect at least one object or at least one position, and
wherein the providing of the at least one piece of information is based on a property of the at least one object or the position.

10. The method according to claim 1 ,
further comprising evaluating the surroundings data to detect at least one object, and
wherein the method further comprises determining, based on at least one of a position and an orientation of the at least one object, the at least one of the type of information presentation, the positioning of the information presentation and the presentation orientation of the information presentation visualized by the visualizing.

11. The method according to claim 1 , wherein the at least one piece of information virtually projected onto the surface of the other vehicle is at least one of a speed of the other vehicle and a distance of the other vehicle from the motor vehicle of the user.

12. A motor vehicle, comprising:
a sensor system configured to capture at least one surrounding area where another vehicle is located, in surroundings of the motor vehicle, situated in a field of vision of a user;
an information source configured to provide at least one piece of information, to be output to the user;
a processor, coupled to the sensor system and the information source, configured to
ascertain a three-dimensional surroundings model describing the surrounding area from the surroundings data, and
generate, based on the at least one piece of information and the surroundings model, one of a three-dimensional information object and a two-dimensional information object in at least one of a position and an orientation in a coordinate system prescribed for the three-dimensional surroundings model; and

a display, coupled to the processor, configured to visualize an at least partially overlaid presentation of an information presentation reproducing the at least one piece of information by virtual projection of the one of the three-dimensional information object and the two-dimensional information object onto a surface of the other vehicle in the at least one surrounding area for the user, positioning of the information presentation and presentation orientation of the information presentation being dependent on the surroundings data.

13. The motor vehicle according to claim 12 , wherein the display includes a position sensing device configured to detect at least one of a relative position and/or an orientation of the display in relation to at least one of a reference position and a reference direction, fixed relative to a coordinate system of the motor vehicle, and a piece of reference information from which the relative position and/or the orientation of the display are ascertained.

14. The motor vehicle according to claim 12 , wherein the display device is one of
a video-based display configured to display an image of the at least one surrounding area, captured by the sensor syst

Assignees

• Audi Ag

Inventors

• Kuehne Marcus
• Profendiner Daniel
• Wollny Nils

Classifications

• G06T19/006 Primary

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• G02B2027/0138

  comprising image capture systems, e.g. camera · CPC title

• G02B27/017

  Head mounted · CPC title

• G02B27/0101 Primary

  characterised by optical features (G02B27/0172 takes precedence) · CPC title

• G02B2027/014

  comprising information/image processing systems · CPC title