Method, mobile user device, and computer program for producing visual information for at least one occupant of a vehicle

Patent metadata

Field	Value
Publication number	US-11062598-B2
Application number	US-202016748870-A
Country	US
Kind code	B2
Filing date	Jan 22, 2020
Priority date	Sep 4, 2017
Publication date	Jul 13, 2021
Grant date	Jul 13, 2021

Abstract

A method produces visual information, by way of a mobile user device, for at least one occupant of the vehicle in connection with at least partially automated driving, in which method visual information, in particular an image sequence, is displayed on a display of the mobile user device. The image sequence represents a traffic situation detected by the vehicle and/or a need for action of an occupant which is requested or recommended in this context.

First claim

What is claimed is:

1. A method for producing visual information, by way of a mobile user device, for an occupant of the vehicle in a context of at least partially automated driving, the method comprising:
outputting visual information on a display device of the mobile user device, wherein the visual information represents a traffic situation detected by way of the vehicle and/or represents a required action by a vehicle occupant, which is requested or recommended in said context;
the mobile user device in a passenger compartment of the vehicle is detected as that of the vehicle occupant in a driver's seat; and
the outputting takes place on the display device of the detected mobile user device of the vehicle occupant in the driver's seat; and
receiving information from the mobile user device characterizing that the vehicle occupant has,
interrupted operation of the mobile user device, and/or
stopped paying attention to the mobile user device using a sensor of the mobile device, and

changing a control action of the vehicle based or the received information.

2. The method according to claim 1 , wherein
the visual information output is an image sequence.

3. The method according to claim 1 , wherein
the visual information representing the traffic situation and/or the required action by the vehicle occupant, relating to the traffic situation, represents a reality which is predicted in a near future.

4. The method according to claim 1 , wherein
an active display content of the mobile user device is reduced or prepared, and/or occupant information which is different from the active display content and which at least partially represents the respective traffic situation and/or the required action by the occupant of the vehicle, which is connected to the traffic situation, is output.

5. The method according to claim 1 , further comprising:
registering a traffic situation and/or determining a specific required action by at least one occupant of a vehicle in a driving mode,
transferring information which is dependent on the registered traffic situation and/or the at least one specific required action by the occupant of the vehicle, to the mobile user device of at least one occupant of the vehicle, and,
subsequently actuating the mobile user device in accordance with the transferred information, wherein visual information which represents the traffic situation and/or the required action at least symbolically is output.

6. The method according to claim 1 , wherein the output on the display device is representative of:
a request for the vehicle occupant to direct their attention to the surroundings of the vehicle,
a request for the vehicle occupant to direct their attention to information about the vehicle,
a request to grip the steering wheel,
a request for the vehicle occupant to briefly depress or operate a pedal of the vehicle or to confirm readiness to operate at least one pedal,
a request to change a vehicle speed,
a request to maintain a distance,
a request for the vehicle occupant to participate in a specific maneuver,
a request for the vehicle occupant to make a decision with respect to a specific maneuver,
a request for at least part of the control of the vehicle to be assumed by the vehicle occupant,
a request to a front seat passenger to wake up or to stir the vehicle occupant or to communicate specific information to said vehicle occupant.

7. The method according to claim 6 , wherein
the vehicle occupant is the vehicle occupant on a driver's seat.

8. The method according to claim 1 , wherein
the output on the display device takes place in accordance with one or more predetermined criteria relating to the determined traffic situation and/or the determination of a specific required action by at least one occupant of a vehicle which is driving in an at least partially automated fashion.

9. The method according to claim 1 , wherein
at least one mobile user device is detected as a mobile user device of a further vehicle occupant or of a vehicle occupant in another seat of the vehicle, and
the outputting takes place on the display device of the detected mobile user device of the vehicle occupant in the other seat of the vehicle.

10. The method according to claim 1 , wherein
different threshold values relating to the significance of the traffic situation and/or significance and urgency of the required action by the at least one vehicle occupant are differentiated, and
different actuation processes of one or more output units of the mobile user device are subsequently carried out.

11. The method according to claim 1 , further comprising:
detecting that the vehicle occupant is listening to audible content and/or is making a telephone call with the mobile user device, and
outputting auditory information by way of the mobile user device representing the information relating to the specific detected traffic situation and/or relating to a requested or recommended action by the vehicle occupant in connection with the detected traffic situation.

12. The method according to claim 11 , further comprising:
outputting of an acoustic or haptic request to direct one's attention to the traffic situation in the surroundings of the vehicle and/or to the image sequence on the mobile user device.

13. The method according to claim 1 , further comprising:
detecting a present application case of the mobile user device and/or of at least one perception channel of a driver or of the vehicle occupant on a driver's seat which is taken up by the use of the mobile user device, and subsequently
actuating the output unit of the mobile user device in accordance with the detected application case of the mobile user device and/or the determined perception channel of the driver or of the vehicle occupant on the driver's seat which is taken up.

14. The method according to claim 1 , further comprising:
detecting at least one specific perception channel of the driver which is taken up by the mobile user device including detection of an intensity with which the at least one perception channel is taken up, and subsequently,
actuating the output unit of the mobile user device in accordance with the detected perception channel including the detected intensity.

15. The method according to claim 1 , further comprising:
in response to the received information, transmitting a request to the mobile user device of a further vehicle occupant to carry out an operator control action in the vehicle.

16. The method according to claim 15 , wherein
the operator control action influences at least one of a lateral guidance, longitudinal guidance, or execution of a maneuver of the vehicle.

17. The method according to claim 1 , wherein
the sensor of the mobile user device is a camera.

18. A computer program product comprising a non-transitory computer readable medium having stored thereon program code that, when executed, carries out the acts of:
determining, using a mobile user device, that a vehicle occupant of a vehicle is actively using a mobile user device;
determining that a driving action is planned or started by the vehicle in at least a partially automated fashion; and
outputting visual information on a display device of the mobile user device, wherein
based on the determining that the vehicle is actively using the mobile user device and the determining that a driving action is planned or started, the visual information instructs the vehicle occupant to refrain from carrying out a specific control action during the driving action and charact

Assignees

• Bayerische Motoren Werke Ag

Inventors

• Augst Alexander

Classifications

• B60K35/26

  using acoustic output · CPC title

• B60K35/10

  Input arrangements, i.e. from user to vehicle, associated with vehicle functions or specially adapted therefor · CPC title

• B60K35/21

  using visual output, e.g. blinking lights or matrix displays · CPC title

• B60K35/80

  Arrangements for controlling instruments · CPC title

• B60K35/28

  characterised by the type of the output information, e.g. video entertainment or vehicle dynamics information; characterised by the purpose of the output information, e.g. for attracting the attention of the driver · CPC title