Pedestrian device and traffic safety assistance method

Patent metadata

Field	Value
Publication number	US-11900795-B2
Application number	US-202017610864-A
Country	US
Kind code	B2
Filing date	May 18, 2020
Priority date	May 20, 2019
Publication date	Feb 13, 2024
Grant date	Feb 13, 2024

Abstract

The present invention relates to a pedestrian device and a traffic safety assistance method which can effectively and properly support pedestrian's safety confirmation by utilizing vehicle-to-pedestrian communications and an AR device. A pedestrian device of the present invention includes: an ITS communication device 21 (pedestrian-vehicle communication device) configured to perform vehicle-to-pedestrian communications with an in-vehicle terminal 2 ; a processor 32 configured to determine if there is a risk of collision based on information transmitted to and received from the in-vehicle device, and control provision of an alert to a user of the pedestrian device; and an AR display 26 for displaying a virtual object overlaid on a real space which can be seen by the user, wherein the processor controls display of the virtual object (virtual terminal) on the AR display as an alert operation to provide an alert to the user.

First claim

The invention claimed is:

1. A pedestrian device comprising:
a pedestrian-vehicle communication device for exchanging information with an in-vehicle device mounted in a vehicle via vehicle-to-pedestrian communications;
an AR display for displaying a virtual object(s) overlaid on a real space which can be seen by a user of the pedestrian device; and
a processor configured to:
determine if there is a risk of collision with the vehicle based on the information transmitted to and received from the in-vehicle device; and
when determining that there is a risk of collision, control display of the virtual object(s) on the AR display as an alert operation to provide an alert to the user, wherein the display of the virtual object(s) on the AR display comprises simulated images of the user and the vehicle, showing stepwise representation of a collision scene in which the user and the vehicle gradually approach each other and finally collide.

2. The pedestrian device according to claim 1 , further comprising a field-of-view camera for capturing the user's field of view,
wherein the processor is configured to:
detect a palm and a fingertip of the user from an image captured by the field-of-view camera;
display a virtual terminal as the virtual object at a position of the user's palm; and
detect the user's screen operation on the virtual terminal based on positions of the fingertip of the user, and control a screen displayed on the virtual terminal in response to the screen operation.

3. The pedestrian device according to claim 2 , further comprising a line-of-sight camera for capturing an eyeball of the user,
wherein the processor is configured to:
acquire position data of a point of regard of the user from an image captured by the line-of-sight camera;
determine whether or not the user is looking at the virtual terminal based on the position data of the point of regard;
when determining that there is a risk of collision and that the user is looking at the visual terminal, performs the alert operation to provide an alert to the user.

4. The pedestrian device according to claim 1 , wherein the processor is configured to, when determining that there is a risk of collision, predict a collision point where the collision is predicted to occur, and the display of the stepwise representation includes a mark image representing the collision point as the virtual object.

5. The pedestrian device according to claim 1 , wherein the processor is configured to display at least one of a mark image and a simulated image representing a position of the vehicle having a risk of collision as the virtual object(s).

6. The pedestrian device according to claim 1 , wherein, when detecting that the user has gotten in a vehicle as a driver, the processor transitions to an in-vehicle terminal mode and controls the display of the virtual object(s) so as to support safe driving of the driver.

7. The pedestrian device according to claim 6 , wherein the pedestrian-vehicle communication device is configured to receive information on a direction of line of sight of s nearby pedestrian from a pedestrian device carried by the nearby pedestrian, and
wherein the processor is configured to display an image representing the direction of the line of sight of the nearby pedestrian as the virtual object based on the information on the direction of the line of sight of the nearby pedestrian.

8. The pedestrian device according to claim 6 , wherein the processor is configured to display at least one of a mark image and a simulated image representing a position of a pedestrian having a risk of collision as the virtual object(s).

9. The pedestrian device according to claim 1 , wherein the pedestrian-vehicle communication device is configured to receive information on a direction of line of sight of a driver of a nearby vehicle from the in-vehicle device mounted therein, and
wherein the processor is configured to display an image representing the direction of the line of sight of the nearby driver as the virtual object based on the information on the direction of the line of sight of the nearby driver.

10. A traffic safety assistance method for exchanging information between a pedestrian device and an in-vehicle device to support safety of a pedestrian and a vehicle, the traffic safety assistance method comprising:
determining, by the pedestrian device, if there is a risk of collision between the vehicle and the pedestrian based on information transmitted to and received from the in-vehicle device, and
when determining that there is a risk of collision, controlling, by the pedestrian device, display of a virtual object on an AR display, wherein the AR display displays the virtual object overlaid on a real space which can be seen by the pedestrian which is a user of the pedestrian device, wherein the AR display displays simulated images of the pedestrian and the vehicle, showing stepwise representation of a collision scene in which the pedestrian and the vehicle gradually approach each other and finally collide.

11. The traffic safety assistance method according to claim 10 , comprising:
acquiring three-dimensional position data of the pedestrian and of the vehicle each time at unit intervals of time during a time period from a current time to when the collision is predicted to occur, and
generating three-dimensional rendering data of the pedestrian and the vehicle at each time in the collision scene based on the three-dimensional position data to display the simulated images of the pedestrian and the vehicle at each time.

12. The traffic safe assistance method according to claim 11 , wherein the three-dimensional position data are calculated based on current positions and moving speeds of the pedestrian and the vehicle.

13. The traffic safe assistance method according to claim 10 , comprising:
displaying predicted mobility areas of the pedestrian and the vehicle in the display of the stepwise representation, wherein the predicted mobility area is displayed in a fan shape extending from a current position of the pedestrian or the vehicle to a collision point where the collision is predicted to occur.

14. The traffic safe assistance method according to claim 10 , comprising:
displaying a mark image corresponding to a position of the vehicle having a risk of collision in case the vehicle is currently out of sight of the pedestrian.

15. The pedestrian device according to claim 1 , wherein the processor is configured to:
acquire three-dimensional position data of the user and of the vehicle each time at unit intervals of time during a time period from a current time to when the collision is predicted to occur, and
generate three-dimensional rendering data of the user and the vehicle at each time in the collision scene based on the three-dimensional position data to display the simulated images of the user and the vehicle at each time.

16. The pedestrian device according to claim 15 , wherein the processors is configured to calculate the three-dimensional position data based on current positions and moving speeds of the user and the vehicle.

17. The pedestrian device according to claim 1 , wherein the processor is configured to display predicted mobility areas of the user and the vehicle in the display of the stepwise representation.

18. The pedestrian device according to claim 17 , wherein the predicted mobility area is displayed in a fan shape extending from a current position of the user or the vehicle to a collision point where the collision is predicted to occur.

19. The

Assignees

• Panasonic Ip Man Co Ltd

Inventors

• Ueno Tsuyoshi
• Sugahara Masahito
• Muramatsu Shintaro
• Okubo Yoshiyuki

Classifications

• G08G1/005 Primary

  including pedestrian guidance indicator · CPC title

• B60W30/0956

  the prediction being responsive to traffic or environmental parameters · CPC title

• B60W50/14

  Means for informing the driver, warning the driver or prompting a driver intervention · CPC title

• G02B27/0093

  with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking · CPC title

• G02B27/0101

  characterised by optical features (G02B27/0172 takes precedence) · CPC title