Collision avoidance system for augmented reality environments

Patent metadata

Field	Value
Publication number	US-10360437-B2
Application number	US-201715604548-A
Country	US
Kind code	B2
Filing date	May 24, 2017
Priority date	Mar 22, 2017
Publication date	Jul 23, 2019
Grant date	Jul 23, 2019

Abstract

A collision avoidance system within an augmented reality environment determines a hazard and notifies the user of the hazard. By determining that the attention of the user is focused on the mobile computing device, and that there is a hazardous condition that the user is approaching, the device provides an alert to the user as to the hazardous condition to protect the user from the hazard. Known hazardous conditions can be stored and provided to a mobile computing device when that mobile computing device is within the vicinity of the known hazardous conditions. The system can also create geofences and provide notifications to the user when a boundary is approached or crossed.

First claim

What is claimed is:

1. A method of collision avoidance in an augmented reality environment, comprising:
determining that an augmented reality application is executing on a mobile computing device; and
based at least in part on determining that the augmented reality application is executing on the mobile computing device,
determining, based at least in part on imaging data representing at least a portion of an image of a user associated with the mobile computing device, that the user is looking at the mobile computing device,
determining a location of the mobile computing device,
receiving, from a database that is maintained by a wireless carrier associated with the mobile computing device, hazard data,
identifying, based at least in part on the hazard data, one or more hazards within a predetermined distance of the location,
determining that the mobile computing device is approaching at least one of the one or more hazards, and
generating a notification of the at least one of the one or more hazards in response to determining that the user is looking at the mobile computing device.

2. The method of claim 1 , wherein the database is remote from the mobile computing device.

3. The method of claim 1 , wherein identifying, as the one or more identified hazards and based at least in part on the hazard data, the one or more hazards comprises capturing image data from an image sensor associated with the mobile computing device.

4. The method of claim 1 , wherein generating the notification comprises exiting the augmented reality application.

5. The method of claim 1 , wherein generating the notification is provided by one or more of haptic, audio, or visual feedback.

6. An electronic device, comprising:
one or more processors;
an accelerometer;
an imaging sensor; and
one or more memories coupled to the one or more processors, the one or more memories having instructions stored thereon that, when executed by the one or more processors, cause the one or more processors to:
determine that the one or more processors are executing an augmented reality application;
receive motion data from the accelerometer;
determine, based at least in part upon the motion data, that the electronic device is moving;
receive imaging data from the imaging sensor representing at least a portion of an image of a user associated with the electronic device;
determine, based at least in part upon the imaging data, that the user associated with the electronic device is looking at the electronic device;
receive, from a database that is maintained by a wireless carrier associated with the electronic device, an indication of one or more hazards within a predetermined distance of the electronic device;
determine that the user associated with the electronic device is approaching at least one of the one or more hazards; and
in response to determining that the one or more processors are executing the augmented reality application and in response to determining that the user is looking at the electronic device, provide a notification indicating presence of the at least one of the one or more hazards.

7. The electronic device of claim 6 , wherein the motion data indicates that the user associated with the electronic device is walking.

8. The electronic device of claim 6 , wherein the instructions cause the one or more processors to analyze the imaging data with an object recognition algorithm.

9. The electronic device of claim 8 , wherein a gaze of the user is determined by the object recognition algorithm.

10. The electronic device of claim 8 , wherein the object recognition algorithm identifies at least one of the one or more hazards.

11. The electronic device of claim 6 , wherein the instructions cause the one or more processors to retrieve an identification and location of at least one of the one or more hazards from a hazards database.

12. The electronic device of claim 6 , wherein the instructions cause the one or more processors to receive an identification and location of at least one of the one or more hazards through a near field communication signal.

13. The electronic device of claim 6 , wherein the notification is one or more of haptic, audible, or visual feedback.

14. A method for collision avoidance, comprising:
determining that a mobile computing device is in motion;
determining, based at least in part on imaging data representing at least a portion of an image of a user associated with the mobile computing device, that the user is looking at the mobile computing device;
determining a location of one or more hazards within a predetermined distance of the mobile computing device;
determining that the mobile computing device is approaching at least one of the one or more hazards;
in response to determining that the user is looking at the mobile computing device, providing a notification of the at least one of the one or more hazards; and
sending the location of the at least one of the one or more hazards to a database that is maintained by a wireless carrier associated with the mobile computing device.

15. The method of claim 14 , further comprising receiving, from a second mobile computing device, an indication of a hazard and adding the indication of the hazard to a hazard database.

16. The method of claim 14 , wherein determining the one or more hazards comprises receiving, at the mobile computing device, a near field communication signal indicating a hazard.

17. The method of claim 14 , wherein determining the one or more hazards comprises receiving, at the mobile computing device, data from a news feed source.

18. The method of claim 14 , wherein determining the one or more hazards comprises creating a virtual boundary around the mobile computing device.

19. The method of claim 14 , wherein determining that the user associated with the mobile computing device is looking at the mobile computing device is based, at least in part, on tilt data generated by a tilt sensor associated with the mobile computing device.

20. The method of claim 14 , further comprising generating a three-dimensional spatial model that indicates the one or more hazards.

Assignees

• T Mobile Usa Inc

Inventors

• Obaidi Ahmad Arash
• Yocam Eric Wayne

Classifications

• G08G1/005 Primary

  including pedestrian guidance indicator · CPC title

• G06V20/64

  Three-dimensional [3D] objects · CPC title

• G06V10/17

  using hand-held instruments · CPC title

• G06K9/22

  Physics · mapped topic

• G06K9/00979

  Physics · mapped topic