Distance and object based external notification system for automated hailing service

Patent metadata

Field	Value
Publication number	US-10796174-B2
Application number	US-201816230960-A
Country	US
Kind code	B2
Filing date	Dec 21, 2018
Priority date	Dec 21, 2018
Publication date	Oct 6, 2020
Grant date	Oct 6, 2020

Abstract

Distance and object based external notification system for automated hailing service is described. An autonomous vehicle (AV) can include a processor configured to execute instructions stored on a non-transitory computer readable medium to detect, based on sensor information, an object within the AV; determine that the object belongs to a recent occupant of the AV; and, in response to the determining that the object belongs to the recent occupant of the AV, select, based on a proximity of the recent occupant to the AV, a notification modality for sending a message to the recent occupant regarding the object; and send the message using the notification modality.

First claim

What is claimed is:

1. An autonomous vehicle (AV), the AV comprising:
a processor configured to execute instructions stored on a non-transitory computer readable medium to:
detect, based on sensor information, an object within the AV;
determine that the object belongs to a recent occupant of the AV; and
in response to determining that the object belongs to the recent occupant of the AV:
select, based on a proximity of the recent occupant to the AV, a notification modality for sending a message to the recent occupant regarding the object, wherein to select the notification modality comprises to:
in a first case that the recent occupant is outside of the AV and the recent occupant is not beyond a threshold distance from the AV, select a first notification modality for sending the message to the recent occupant; and
in a second case that the recent occupant is outside of the AV and the recent occupant is beyond the threshold distance from the AV, select a second notification modality for sending the message to the recent occupant, wherein the second notification modality is different from the first notification modality and wherein the second notification modality comprises sending an electronic notification to the recent occupant; and send the message using the notification modality.

2. The AV of claim 1 , wherein in a case that the recent occupant is within the AV, the notification modality comprises displaying, on a display of the AV, an image of the object.

3. The AV of claim 1 , wherein in a case that the recent occupant is not within the AV, the notification modality comprises an external audible notification.

4. The AV of claim 3 , wherein a volume of the external audible notification is based on a distance between the recent occupant and the AV.

5. The AV of claim 1 , wherein in a case that the recent occupant is not within the AV, the notification modality comprises an external visible notification.

6. The AV of claim 1 , wherein the instructions further include instructions to:
operate the AV to follow the recent occupant.

7. The AV of claim 1 ,
wherein the instructions further comprise instructions to:
determine a value of the object; and

wherein the selecting the notification modality for sending the message is further based on the value of the object.

8. The AV of claim 1 , wherein the instructions further include instructions to:
in response to determining that the recent occupant of the AV is touching a part of the AV and that the object is within the AV, vibrate the part of the AV.

9. An autonomous vehicle (AV) comprising:
a processor configured to execute instructions stored on a non-transitory computer readable medium to:
detect, based on sensor information, an object within the AV;
select, based on a first distance of a recent occupant to the AV, a first notification modality for sending a message to the recent occupant, the message indicating that the object is left by the recent occupant in the AV;
send, using the first notification modality, the message to the recent occupant;
select, based on a second distance of the recent occupant to the AV, a second notification modality for sending the message to the recent occupant;
select, based on a third distance of the recent occupant to the AV, a third notification modality for sending the message to the recent occupant; and
send, using the third notification modality, the message to the recent occupant; and
send, using at least one of the first notification modality, the second notification modality, or the third notification modality the message to the recent occupant.

10. The AV of claim 9 ,
wherein the second notification modality comprises sending a text message to the recent occupant; and
wherein the instructions further include instructions to:
receive, from the recent occupant, a response to the text message.

11. The AV of claim 9 , wherein the first notification modality comprises at least one of haptic feedback, an audio signal, or a visual signal.

12. The AV of claim 9 , wherein the second notification modality comprises at least one of an audio signal or a visual signal.

13. The AV of claim 9 , wherein the instructions further include instructions to:
select the second notification modality and send the message in response to detecting that the object is within the AV after sending, using the first notification modality, the message.

14. The AV of claim 9 ,
wherein the instructions further comprise instructions to:
determine a value of the object; and

wherein to select at least one of the first notification modality, the second notification modality, or the third notification modality is further based on the value of the object.

15. A method for providing a service by an autonomous vehicle (AV) comprising:
detecting, based on sensor information, an object within the AV;
selecting, based on a first distance of a recent occupant to the AV, a first notification modality for sending a first message to the recent occupant, the first message indicating that the object is left by the recent occupant in the AV;
sending, using the first notification modality, the first message to the recent occupant;
selecting, based on a second distance of the recent occupant to the AV, a second notification modality for sending a second message to the recent occupant, wherein the second notification modality comprises sending a text message to the recent occupant;
sending, using the second notification modality, the second message to the recent occupant; and
receiving, from the recent occupant, a response to the text message.

16. The method of claim 15 , wherein the first notification modality comprises at least one of haptic feedback, an audio signal, or a visual signal.

17. The method of claim 15 , wherein at least one of the first notification modality or the second notification modality comprises at least one of an audio signal or a visual signal.

18. The method of claim 15 , further comprising:
selecting the second notification modality and sending the second message in response to detecting that the object is within the AV after sending, using the first notification modality, the first message.

19. The method of claim 15 , further comprising:
selecting, based on a third distance of the recent occupant to the AV, a third notification modality for sending a third message to the recent occupant; and
sending, using the third notification modality, the third message to the recent occupant.

20. An autonomous vehicle (AV) comprising:
a processor configured to execute instructions stored on a non-transitory computer readable medium to:
detect that a recent occupant of the AV has not completed an in-vehicle activity, wherein the in-vehicle activity is one of a set of in-vehicle activities including that an electronic payment of the recent occupant is unfinished; and
in response to detecting that the recent occupant of the AV has not completed the in-vehicle activity:
select, based on a proximity of the recent occupant to the AV, a notification modality for sending a message to the recent occupant regarding the in-vehicle activity; and
send the message using the notification modality.

21. The AV of claim 20 , wherein the set of in-vehicle activities further includes that at least one of a door of the AV or a trunk of the AV is open.

Assignees

• Nissan North America Inc
• Renault Sas

Inventors

• Zhang Jingyi
• Tien Simon
• Cefkin Melissa
• Cesafsky Laura

Classifications

• G08B6/00

  Tactile signalling systems, e.g. tactile personal calling systems · CPC title

• G06V20/593 Primary

  Recognising seat occupancy · CPC title

• G06V20/59

  inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions · CPC title

• G08B21/22

  responsive to presence or absence of persons · CPC title

• G08B21/0225

  Monitoring making use of different thresholds, e.g. for different alarm levels · CPC title