Identifying a customer of an autonomous vehicle

Patent metadata

Field	Value
Publication number	US-11904902-B2
Application number	US-202117511634-A
Country	US
Kind code	B2
Filing date	Oct 27, 2021
Priority date	Oct 29, 2020
Publication date	Feb 20, 2024
Grant date	Feb 20, 2024

Abstract

The technology employs a holistic approach to passenger pickups and other wayfinding situations. This includes identifying where passengers are relative to the vehicle and/or the pickup location. Information synthesis from different sensors, agent behavior prediction models, and real-time situational awareness are employed to identify the likelihood that the passenger to be picked up is at a given location at a particular point in time, with sufficient confidence. The system can provide adaptive navigation by helping passengers understand their distance and direction to the vehicle, for instance using various cues via an app on the person's device. Rider support tools may be provided, which enable a remote agent to interact with a customer via that person's device, such as using the camera on the device to provide wayfinding support to enable the person to find their vehicle. Ride support may also use sensor information from the vehicle when providing wayfinding support.

First claim

The invention claimed is:

1. A method for identifying a customer of a vehicle that is configured to operate in an autonomous driving mode, the method comprising:
receiving, by one or more processors of a computing system of the vehicle, perception information regarding a set of objects in an external environment of the vehicle;
determining, by the one or more processors, that the set of objects in the external environment includes a plurality of pedestrian objects, each pedestrian object corresponding to a person;
obtaining a likelihood that each pedestrian object is the customer, wherein obtaining the likelihood includes:
applying first information from a client device of the customer to a localization model; and
applying second information from a sensor source type other than the client device of the customer to the localization model;

evaluating, by the one or more processors, the likelihood that each pedestrian object is the customer; and
upon determining that the likelihood for a given one of the pedestrian objects indicates that the given pedestrian object is the customer, performing at least one of: (i) sending a notification to the client device of the customer regarding a pickup location to meet the vehicle, (ii) adjusting the pickup location, (iii) causing modification to a driving operation of the vehicle in the autonomous driving mode, or (iv) transmitting information about the given pedestrian object to a remote management system.

2. The method of claim 1 , wherein the sensor source type other than the client device of the customer is selected from the group consisting of vehicle sources that are onboard the vehicle and fleet sources obtained from one or more other vehicles in a fleet of vehicles.

3. The method of claim 2 , wherein the vehicle sources include one or more sensors of a perception system of the vehicle, positioning system sources, or communication system sources.

4. The method of claim 3 , wherein the one or more sensors of the perception system include one or more of lidar, camera, radar or acoustical sensors.

5. The method of claim 1 , wherein:
the first information from the client device of the customer includes geolocation information from the client device of the customer, and
the second information from the sensor source type other than the client device of the customer includes perception information from one or more sensors of a perception system of the vehicle.

6. The method of claim 1 , wherein evaluating the likelihood that each pedestrian object is the customer includes refining a list of pedestrian objects that could be the customer.

7. The method of claim 1 , wherein sending the notification to the client device of the customer includes requesting current or historical geolocation information of the client device.

8. The method of claim 1 , wherein sending the notification to the client device of the customer includes providing updated walking directions or turn-by-turn directions.

9. The method of claim 1 , wherein sending the notification to the client device of the customer includes a request for the customer to perform a selected action.

10. The method of claim 1 , wherein sending the notification to the client device of the customer includes selecting a timing for sending the notification based on a distance of the given pedestrian object to either the pickup location or to the vehicle.

11. The method of claim 1 , wherein sending the notification to the client device of the customer includes selecting a specificity of a message to the customer based on a distance of the given pedestrian object to either the pickup location or to the vehicle.

12. The method of claim 1 , wherein the customer is a rider scheduled to be picked up by the vehicle for a trip.

13. The method of claim 1 , wherein adjusting the pickup location includes moving the pickup location to a position closer to a current location of the customer to reduce a distance the customer would traverse.

14. The method of claim 1 , wherein transmitting information about the given pedestrian object to the remote management system includes a request for rider support from the remote management system.

15. The method of claim 1 , wherein the first information from the client device of the customer includes at least one of:
a gesture of the client device of the customer;
a sound from the client device of the customer; and
activation of a flashlight of the client device of the customer.

16. A vehicle configured to operate in an autonomous driving mode, the vehicle comprising:
a perception system including one or more sensors, the one or more sensors being configured to receive sensor data associated with objects in an external environment of the vehicle;
a driving system including a steering subsystem, an acceleration subsystem and a deceleration subsystem to control driving of the vehicle;
a positioning system configured to determine a current position of the vehicle; and
a control system including one or more processors, the control system operatively coupled to the driving system, the perception system and the positioning system, the control system being configured to:
receive perception information from the perception system regarding a set of objects in the external environment of the vehicle;
determine that the set of objects in the external environment includes a plurality of pedestrian objects, each pedestrian object corresponding to a person;
obtain a likelihood that each pedestrian object is a customer, wherein the control system is configured to obtain the likelihood by being configured to:
apply first information from a client device of the customer to a localization model; and
apply second information from a sensor source type other than the client device of the customer to the localization model;

evaluate the likelihood that each pedestrian object is the customer; and
upon determining that the likelihood for a given one of the pedestrian objects indicates that the given pedestrian object is the customer, perform at least one of: (i) send a notification to the client device of the customer regarding a pickup location to meet the vehicle, (ii) adjust the pickup location, (iii) cause modification to a driving operation to be performed by the driving system in the autonomous driving mode, or (iv) transmit information about the given pedestrian object to a remote management system.

17. The vehicle of claim 16 , wherein the vehicle is configured to deliver a package to the customer at the pickup location.

18. The vehicle of claim 16 , wherein:
the first information from the client device of the customer includes geolocation information from the client device of the customer, and
the second information from the sensor source type other than the client device of the customer includes perception information from the one or more sensors of a perception system of the vehicle.

19. The vehicle of claim 16 , wherein evaluation of the likelihood that each pedestrian object is the customer includes refinement of a list of pedestrian objects that could be the customer.

20. The vehicle of claim 16 , wherein the control system is further configured to issue a request to the client device of the customer for updated location information upon a determination that the customer is within a threshold proximity to the vehicle or to the pickup location.

21. The vehicle of claim 16 , wherein the control system is further configured to issue a request for one or more other vehicles to p

Assignees

• Waymo Llc

Inventors

• Wright Clement
• Schwendimann Lauren
• Moon Maria
• Malta Kevin
• Balachandran Ganesh
• Pedersen Jonathan
• Gutlansky Daniella

Classifications

• G06Q10/0283

  for travel seating · CPC title

• B60W60/00256 Primary

  Delivery operations · CPC title

• B60W2420/42

  Operations & Transport · mapped topic

• B60W2420/52

  Operations & Transport · mapped topic

• B60W2420/54

  Audio sensitive means, e.g. ultrasound · CPC title