Wearable smart device for hazard detection and warning based on image and audio data

Patent metadata

Field	Value
Publication number	US-9576460-B2
Application number	US-201514601506-A
Country	US
Kind code	B2
Filing date	Jan 21, 2015
Priority date	Jan 21, 2015
Publication date	Feb 21, 2017
Grant date	Feb 21, 2017

Abstract

A wearable smart device for providing hazard warning information to a user. The wearable smart device includes a microphone configured to detect audio data associated with a potential hazard. The wearable smart device also includes a camera configured to detect image data associated with the potential hazard. The wearable smart device also includes a processor coupled to the microphone and the camera and configured to determine whether the potential hazard presents a real hazard based on the detected audio data and the detected image data.

First claim

What is claimed is:

1. A wearable computing device for providing hazard warning information comprising:
a microphone configured to detect audio data associated with a potential hazard;
a camera configured to detect image data associated with the potential hazard; and
a processor coupled to the microphone and the camera and configured to determine which of the detected audio data or the detected image data has a higher quality, and to determine whether the potential hazard presents a real hazard to a user based on the detected audio data and the detected image data such that more weight is provided to whichever of the detected audio data or the detected image data has the higher quality.

2. The wearable computing device of claim 1 further comprising a memory coupled to the processor and configured to store memory audio data and memory image data associated with at least one hazard, and wherein the processor is configured to determine whether the potential hazard presents the real hazard by comparing at least one of the detected audio data or the detected image data to at least one of the memory audio data or the memory image data.

3. The wearable computing device of claim 1 further comprising an inertial measurement unit (IMU) coupled to the processor and configured to detect motion data and a GPS sensor configured to detect location data and wherein the processor is further configured to determine a current location of the wearable computing device based on at least two of the detected motion data, the detected location data, the detected audio data, and the detected image data.

4. The wearable computing device of claim 3 further comprising a memory coupled to the processor and configured to store a stored location and at least one hazard associated with the stored location, and wherein the processor is further configured to determine whether the at least one hazard presents a danger to the user based on a comparison of the current location to the stored location.

5. The wearable computing device of claim 1 wherein the microphone includes two microphones spaced apart and the processor is further configured to determine a direction of the potential hazard based on the detected audio data.

6. The wearable computing device of claim 5 wherein the processor is further configured to instruct the camera to focus towards the direction of the potential hazard in response to determining the direction of the potential hazard.

7. The wearable computing device of claim 1 wherein the processor is further configured to determine a distance to the potential hazard based on a volume of the detected audio data.

8. The wearable computing device of claim 1 wherein the processor is further configured to determine a severity level of the potential hazard based on at least one factor.

9. The wearable computing device of claim 8 wherein the at least one factor includes a distance to the potential hazard, whether a distance between the potential hazard and the user is decreasing, a size of the potential hazard, or an audio volume of the potential hazard.

10. The wearable computing device of claim 1 wherein the higher quality corresponds to at least one of a lack of obstruction of the detected audio data or the detected image data, an accuracy of the detected audio data or the detected image data, or a match of the detected audio data or the detected image data to data stored in a memory.

11. A wearable computing device for providing hazard warnings comprising:
an upper portion having a first end and a second end;
a first lower portion coupled to the first end of the upper portion;
a second lower portion coupled to the second end of the upper portion;
at least one microphone positioned on the first lower portion, the second lower portion or the upper portion and configured to detect audio data associated with a potential hazard;
a camera positioned on the first lower portion or the second lower portion and configured to detect image data associated with the potential hazard; and
a processor coupled to the camera and the at least one microphone and configured to determine which of the detected audio data or the detected image data has a higher quality, and to determine whether the potential hazard presents a real hazard to a user by analyzing whichever of the detected audio data or the detected image data has the higher quality.

12. The wearable computing device of claim 11 further comprising a memory coupled to the processor and configured to store memory audio data and memory image data associated with at least one hazard and wherein the processor is configured to determine whether the potential hazard presents the real hazard by comparing at least one of the detected audio data or the detected image data to at least one of the memory audio data or the memory image data.

13. The wearable computing device of claim 11 wherein the at least one microphone includes two microphones and the processor is further configured to determine a direction of the potential hazard based on the detected audio data.

14. The wearable computing device of claim 13 wherein the processor is further configured to cause the camera to focus towards the direction of the potential hazard in response to determining the direction of the potential hazard.

15. The wearable computing device of claim 11 wherein the processor is further configured to determine a distance to the potential hazard based on a volume of the detected audio data.

16. The wearable computing device of claim 11 wherein the processor is further configured to determine a severity level of the potential hazard based on at least one of a distance to the potential hazard, whether the potential hazard is approaching the user, a size of the potential hazard or an audio volume of the potential hazard.

17. The wearable computing device of claim 11 wherein the higher quality corresponds to at least one of a lack of obstruction of the detected audio data or the detected image data, an accuracy of the detected audio data or the detected image data, whether the detected audio data or the detected image data fails to indicate that the potential hazard is present, or a match of the detected audio data or the detected image data to data stored in a memory.

18. A method for providing hazard warnings to a user of a wearable computing device comprising:
detecting, by at least two microphones, audio data associated with a potential hazard and including volume information;
detecting, by a camera, image data associated with the potential hazard;
determining, by a processor, whether the detected audio data or the detected image data has a higher quality; and
determining, by the processor, whether the potential hazard presents a real hazard to the user based on the detected audio data and the detected image data by providing more weight to whichever of the detected audio data or the detected image data has the higher quality.

19. The method of claim 18 wherein the higher quality corresponds to at least one of a lack of obstruction of the detected audio data or the detected image data, an accuracy of the detected audio data or the detected image data, whether the detected audio data or the detected image data fails to indicate that the potential hazard is present, or a match of the detected audio data or the detected image data to data stored in a memory.

20. The method of claim 18 further comprising storing, in a memory, stored audio data and stored image data associated with at least one hazard, and wherein de

Assignees

• Toyota Motor Eng & Mfg North America Inc

Inventors

• Dayal Rajiv

Classifications

• G08B21/02 Primary

  Alarms for ensuring the safety of persons · CPC title

• G01C21/3691

  Retrieval, searching and output of information related to real-time traffic, weather, or environmental conditions (arrangements for giving variable traffic instructions G08G1/09) · CPC title

• A61H2201/5092

  Optical sensor · CPC title

• A61H2201/5064

  Position sensors · CPC title

• A61H2201/5082

  Temperature sensors · CPC title