System and method for headphones for monitoring an environment outside of a user's field of view

Patent metadata

Field	Value
Publication number	US-10567904-B2
Application number	US-201715684317-A
Country	US
Kind code	B2
Filing date	Aug 23, 2017
Priority date	Aug 23, 2017
Publication date	Feb 18, 2020
Grant date	Feb 18, 2020

Abstract

In at least one embodiment, a computer-program product embodied in a non-transitory computer readable medium that is programmed to provide an alert for a user of an environment outside of the user's visual field of view is provided. The computer-program product includes instructions to receive an echo profile indicative of at least one object outside of the user's visual field of view from headphones and to receive a command indicative of at least one object to detect on the echo profile from the user. The computer-program product includes instructions to generate an alert for the user to notify the user of a detected object in the user's visual field of view in the event the echo profile includes the at least one object.

First claim

What is claimed is:

1. A computer-program product embodied in a non-transitory computer readable medium that is programmed to provide an alert for a user of an environment outside of the user's visual field of view, the computer-program product comprising instructions to:
receive an echo profile on two channels, the echo profile being indicative of at least one object outside of the user's visual field of view from headphones;
receive a command, from a user interface, the command being indicative of at least one object to detect on the echo profile;
generate the alert for the user to notify the user of a detected object in the user's visual field of view in the event the echo profile includes the at least one object;
determine whether the echo profile includes a frequency that is within an ultrasonic frequency range prior to generating the alert; and
merge data of the echo profile as received on the two channels after determining that the echo profile includes the frequency that is within the ultrasonic frequency range to observe the at least one object in a blind view of the user prior to generating the alert.

2. The computer-program product of claim 1 wherein the ultrasonic frequency range corresponds to a frequency that is greater than 20 kHz.

3. The computer-program product of claim 1 , wherein the merged data of the echo profile is formed into a single stitched echo profile to provide an image depth map.

4. The computer-program product of claim 3 further comprising instructions to perform a deep learning based object detection on objects present in the image depth map to detect the at least one object.

5. The computer-program product of claim 4 further comprising instructions to extract shapes from the image depth map and to compress the extracted shapes.

6. The computer-program product of claim 5 further comprising instructions to compare the compressed extracted shapes against previously known objects stored in memory.

7. The computer-program product of claim 1 further comprising instructions to:
receive at least one of acceleration information and location information of a mobile device; and
execute a Kalman filter to estimate a future position of the at least one object in relation to the user.

8. The computer-program product of claim 7 further comprising instructions to receive the at least one of acceleration information and location information and to execute the Kalman filter prior to generating the alert for the user to notify the user of a detected object in the user's visual field of view.

9. The computer-program product of claim 1 wherein generating the alert for the user further includes streaming images on a display to alert the user of the detected object in the user's visual field of view in the event the echo profile includes the at least one object.

10. An apparatus for providing an alert for a user of an environment outside of the user's visual field of view, the apparatus comprising:
a mobile device being programmed to:
receive an echo profile on two channels, the echo profile being indicative of at least one object outside of the user's visual field of view from headphones;
receive a command from a user interface, the command being indicative of at least one object to detect on the echo profile;
generate the alert for the user to notify the user of a detected object in the user's visual field of view in the event the echo profile includes the at least one object;
determine whether the echo profile includes a frequency that is within an ultrasonic frequency range prior to generating the alert; and
merge data of the echo profile as received on the two channels after determining that the echo profile includes the frequency that is within the ultrasonic frequency range to observe the at least one object in a blind view of the user prior to generating the alert.

11. The apparatus of claim 10 wherein the mobile device is further programmed to transmit an audio stream to the headphones for playback while receiving the echo profile from the headphones.

12. The apparatus of claim 10 wherein the mobile device is further programmed to: provide at least one of acceleration information and location information to determine a future position of the user; and
execute a Kalman filter to estimate a future position of the at least one object in relation to the user.

13. The apparatus of claim 12 wherein the mobile device is further configured to provide at least one of acceleration information and location information to determine a future position of the user and execute the Kalman filter to estimate a future position of the at least one object in relation to the user.

14. The apparatus of claim 10 wherein the mobile device is further programmed to stream images on a display thereof to alert the user of the detected object in the user's visual field of view in the event the echo profile includes the at least one object.

15. A system for monitoring an environment outside of a user's visual field of view, the system comprising:
headphones including at least one audio speaker and at least one microphone, the headphones being programmed to:
receive an audio stream from a mobile device
playback the audio stream via the at least one audio speaker;
transmit a first signal in an ultrasonic range to an area exterior to the headphones;
receive, via the at least one microphone, a reflected first signal in the ultrasonic range from at least one object surrounding the user; and
transmit an echo profile indicative of at least one object outside of the user's visual field of view in response to the received reflected first signal; and

a mobile device being programmed to:
receive an echo profile on two channels;
determine whether the echo profile includes a frequency that is within an ultrasonic frequency range; and
merge data of the echo profile as received on the two channels after determining that the echo profile includes the frequency that is within the ultrasonic frequency range to observe the at least one object in a blind view of the user.

16. The system of claim 15 wherein the ultrasonic range corresponds to a frequency that is greater than 20 kHz.

17. The system of claim 15 wherein the at least one microphone is tuned to capture audio in a range of 20 Hz-20 kHz and tuned to capture the reflected first signal in the ultrasonic range of a frequency that is greater than 20 kHz.

Assignees

• Harman Int Ind

Inventors

• Sahay Pratyush

Classifications

• H04R2460/07

  Use of position data from wide-area or local-area positioning systems in hearing devices, e.g. program or information selection · CPC title

• H04R1/1041

  Mechanical or electronic switches, or control elements (switches in general H01H) · CPC title

• G08B3/10

  using electric transmission; using electromagnetic transmission · CPC title

• H04S7/304 Primary

  For headphones · CPC title

• G08B13/1618

  using ultrasonic detection means · CPC title