Auditory assistant module for autonomous vehicles

Patent metadata

Field	Value
Publication number	US-12134397-B2
Application number	US-202318339987-A
Country	US
Kind code	B2
Filing date	Jun 22, 2023
Priority date	Apr 26, 2019
Publication date	Nov 5, 2024
Grant date	Nov 5, 2024

Abstract

Disclosed are devices, systems and methods for an audio assistant in an autonomous or semi-autonomous vehicle. In one aspect the informational audio assistant receives a first set of data from a vehicle sensor and identifies an object or condition using the data from the vehicle sensor. Audio is generated representative of a perceived danger of an object or condition. A second set of data from the vehicle sensor subsystem is received and the informational audio assistant determines whether an increased danger exists based on a comparison of the first set of data to the second set of data. The informational audio assistant will apply a sound profile to the generated audio based on the increased danger.

First claim

What is claimed is:

1. A method of providing audio in a vehicle, comprising:
obtaining, by a computer located in a first vehicle, a first set of information about a second vehicle at a first time, wherein the second vehicle is located behind the first vehicle;
causing a first audio to be played in the first vehicle, wherein the first audio represents a direction in which the second vehicle moves towards the first vehicle, and wherein the first audio indicates a speed and the first location of the second vehicle relative to a fourth location of the first vehicle at the first time;
obtaining, by the computer located in the first vehicle, a second set of information about the second vehicle at a second time later than the first time;
performing a first determination, from the second set of information, that the second vehicle located at a second location at the second time poses an increased danger compared to the second vehicle located at a first location at the first time; and
causing, in response to the first determination, a second audio to be played in the first vehicle, wherein the second audio is different than the first audio.

2. The method of claim 1 ,
wherein the first audio is caused to be generated by a first speaker located at a first side of the vehicle that is on a same side as the first location of the second vehicle, and
wherein the second audio is caused to be generated by a second speaker located at a second side of the vehicle that is on a same side as the second location of the second vehicle.

3. The method of claim 2 , further comprising:
causing a third audio to be played in the first vehicle in response to a distance between the second vehicle and the first vehicle being greater at a third time than at the second time,
wherein the third time is later than the second time, and
wherein the third audio has an audio pattern or a rhythm that is slower than that of the second audio.

4. The method of claim 3 ,
wherein the third audio is caused to be generated by a third speaker located at a third side of the vehicle that is on a same side as a third location of the second vehicle at the third time, and
wherein the third side is different from the first side and the second side.

5. The method of claim 1 , further comprising:
performing a second determination, by the computer located in the first vehicle, that a plurality of pedestrians are present in an environment where the first vehicle is operated; and
causing a fourth audio to be played in the first vehicle in response to the second determination, wherein the fourth audio indicates a presence of the plurality of pedestrians.

6. The method of claim 1 , further comprising:
performing a third determination, by the computer located in the first vehicle, that a road on which the first vehicle is operating does not have one or more vehicles within a distance of a location of the first vehicle; and
causing a fifth audio to be played in the first vehicle in response to the third determination, wherein the fifth audio indicates that the road is clear for the distance.

7. The method of claim 1 , further comprising:
performing a fourth determination of a presence of water on a road on which the first vehicle will be driven;
causing a sixth audio to be played in the first vehicle in response to the fourth determination, wherein the sixth audio indicates that slippery conditions are present on the road ahead.

8. A non-transitory computer readable medium having code stored thereon, the code when executed by a processor, causing the processor to implement a method, comprising:
obtaining, by a computer located in a first vehicle, a first set of information about a second vehicle at a first time, wherein the second vehicle is located behind the first vehicle;
causing a first audio to be played in the first vehicle, wherein the first audio represents a direction in which the second vehicle moves towards the first vehicle, and wherein the first audio indicates a speed and the first location of the second vehicle relative to a fourth location of the first vehicle at the first time;
obtaining, by the computer located in the first vehicle, a second set of information about the second vehicle at a second time later than the first time;
performing a first determination, from the second set of information, that the second vehicle located at a second location at the second time poses an increased danger compared to the second vehicle located at a first location at the first time; and
causing, in response to the first determination, a second audio to be played in the first vehicle, wherein the second audio is different than the first audio.

9. The non-transitory computer readable medium of claim 8 , wherein the second vehicle is determined to pose the increased danger in response to determining that the second vehicle is approaching the first vehicle at a speed greater than a pre-determined threshold.

10. The non-transitory computer readable medium of claim 8 , wherein the second vehicle is determined to pose the increased danger in response to determining that a brake of the first vehicle has not been depressed since the first time until the second time.

11. The non-transitory computer readable medium of claim 8 , wherein the second audio has a volume higher than that of the first audio.

12. The non-transitory computer readable medium of claim 11 , wherein the second audio indicates the second location of the second vehicle relative to a fifth location of the first vehicle at the second time.

13. The non-transitory computer readable medium of claim 8 , wherein the second audio is caused to be played by applying a sound profile to the first audio, wherein the sound profile intensifies the first audio by modifying a frequency or an amplitude of the first audio.

14. An apparatus comprising:
at least one processor configured to:
obtain, by a computer located in a first vehicle, a first set of information about a second vehicle at a first time, wherein the second vehicle is located behind the first vehicle;
cause a first audio to be played in the first vehicle, wherein the first audio represents a direction in which the second vehicle moves towards the first vehicle, and wherein the first audio indicates a speed and the first location of the second vehicle relative to a fourth location of the first vehicle at the first time;
obtain, by the computer located in the first vehicle, a second set of information about the second vehicle at a second time later than the first time;
perform a first determination, from the second set of information, that the second vehicle located at a second location at the second time poses an increased danger compared to the second vehicle located at a first location at the first time; and
cause, in response to the first determination, a second audio to be played in the first vehicle, wherein the second audio is different than the first audio.

15. The apparatus of claim 14 , wherein each of the first set of information and the second set of information about the second vehicle includes a location, a size, the direction, and a velocity of the second vehicle.

16. The apparatus of claim 14 , wherein the first audio includes an audio pattern that includes beeps, chimes, or notes.

17. The apparatus of claim 14 , wherein the second audio has an audio pattern or a rhythm that is faster than that of the first audio, and wherein the audio pattern or the rhythm is based on a location of the second vehicle compared to that of the first vehicle.

18. The apparatus of cla

Assignees

• Tusimple Inc

Inventors

• Zhang Cheng
• Hou Xiaodi
• Kratz Sven

Classifications

• G05D1/646

  Following a predefined trajectory, e.g. a line marked on the floor or a flight path · CPC title

• H04R2499/13

  Acoustic transducers and sound field adaptation in vehicles · CPC title

• H04R5/02

  Spatial or constructional arrangements of loudspeakers · CPC title

• B60W2510/18

  Braking system · CPC title

• B60W40/08

  related to drivers or passengers · CPC title