Autonomous vehicle and infrastructure aided robotic system for end-to-end package delivery

Patent metadata

Field	Value
Publication number	US-11797014-B2
Application number	US-202117171072-A
Country	US
Kind code	B2
Filing date	Feb 9, 2021
Priority date	Feb 9, 2021
Publication date	Oct 24, 2023
Grant date	Oct 24, 2023

Abstract

A method for controlling a robotic vehicle in a delivery environment includes causing the robotic vehicle to deploy from an autonomous vehicle (AV) at a first AV position in the delivery environment. The method further includes localizing, via a robotic vehicle controller, an initial position within a global reference map using a robot vehicle perception system, receiving, from the AV, a 3-dimensional (3D) augmented map and localizing an updated position in the delivery environment based on the 3D augmented map and the global reference map. The robot vehicle perception system senses obstacle characteristics, and generates a unified 3D augmented map with robot-sensed obstacle characteristics. The method further includes generating a dynamic path plan to a package delivery destination using the unified 3D augmented map, and actuating the robot vehicle to the package delivery destination according to the dynamic path plan.

First claim

That which is claimed is:

1. A method for controlling a robotic vehicle in a delivery environment, the robotic vehicle comprising a sensory system comprising a sensor, the method comprising:
causing the robotic vehicle to deploy from an autonomous vehicle (AV) at a first AV position in the delivery environment,
localizing, via a robotic vehicle controller, an initial position within a global reference map using a robot vehicle perception system;
receiving, from the AV, a 3-dimensional (3D) augmented map;
localizing an updated position in the delivery environment based on the 3D augmented map and the global reference map;
detecting, via the robot vehicle perception system, an obstacle proximate to the robotic vehicle;
generating robot-sensed obstacle characteristic information based on the obstacle that was detected;
generating a first unified 3D augmented map by updating the 3D augmented map with the robot-sensed obstacle characteristics information;
generating a dynamic path plan to a package delivery destination using the first unified 3D augmented map; and
actuating, via the robotic vehicle controller, the robot vehicle to the package delivery destination according to the dynamic path plan;
transmitting the first unified 3D augmented map to the AV;
generating, via a vehicle perception system, AV-sensed obstacle characteristics;
generating a second unified 3D augmented map comprising the robot-sensed obstacle characteristics and AV-sensed obstacle characteristics;
receiving, via the robotic vehicle controller, the second unified 3D augmented map comprising the robot-sensed obstacle characteristics and the AV-sensed obstacle characteristics; and
generating or updating the dynamic path plan to the package delivery destination using the second unified 3D augmented map.

2. The method according to claim 1 , wherein the one or more of the robot-sensed obstacle characteristics and the AV-sensed obstacle characteristics comprises an obstacle localization and an obstacle class.

3. The method according to claim 2 , wherein the obstacle class comprises one of a static obstacle and a dynamic obstacle.

4. The method according to claim 1 , further comprising:
receiving, from the AV, a second AV position indicative of a second AV position in the delivery environment that is different from the first AV position;
generating an updated global reference map with the second AV position;
actuating, via the robotic vehicle controller, the robot vehicle to the package delivery destination according to the dynamic path plan and delivering a package at the package delivery destination; and
actuating the robot vehicle to travel to the second AV position based on the updated global reference map.

5. The method according to claim 4 , wherein the sensor is a camera, wherein the sensory system further comprises a LiDAR system and an ultrasonic sensor, wherein the robot-sensed obstacle characteristics of the first and second unified 3D augmented maps are each generated with sensory data from each of the camera, the LiDAR system, and the ultrasonic sensor, and wherein the robotic vehicle is one of a bipedal robot, a 4-legged robot, and a wheeled robot.

6. The method according to claim 1 , further comprising:
receiving, via the robotic vehicle controller, an infrastructure sensor data feed comprising infrastructure-sensed obstacle characteristics; and
generating the unified 3D augmented map based in part on the infrastructure-sensed obstacle characteristics.

7. The method according to claim 6 , wherein the one or more of the robot-sensed obstacle characteristics and AV-sensed obstacle characteristics comprises an obstacle localization and an obstacle class.

8. The method according to claim 7 , wherein the obstacle class comprises one of a static obstacle and a dynamic obstacle.

9. The method according to claim 1 , wherein the sensor is a camera, wherein the sensory system further comprises a LiDAR system and an ultrasonic sensor, and wherein the robot-sensed obstacle characteristics of the first and second unified 3D augmented maps are each generated with sensory data from each of the camera, the LiDAR system, and the ultrasonic sensor.

10. A robot delivery system for a robot vehicle, the robotic vehicle comprising a sensory system comprising a sensor, the system comprising:
a processor; and
a memory for storing executable instructions, the processor programmed to execute the instructions to:
cause the robotic vehicle to deploy from an autonomous vehicle (AV) at a first AV position in the delivery environment,
localize, via the processor, an initial position within a global reference map using a robot vehicle perception system;
receive, from the AV, a first 3-dimensional (3D) augmented map;
localize an updated position in the delivery environment based on the 3D augmented map and the global reference map;
detect, with a robot vehicle perception system, an obstacle proximate to the robotic vehicle;
generate robot-sensed obstacle characteristic information based on the obstacle that was detected;
generate a first unified 3D augmented map by updating the 3D augmented map with the robot-sensed obstacle characteristics;
generate a dynamic path plan to a package delivery destination using the first unified 3D augmented map; and
actuate, via the processor, the robot vehicle to the package delivery destination according to the dynamic path plan;
transmit the first unified 3D augmented map to the AV;
generate, via a vehicle perception system, AV-sensed obstacle characteristics;
generate a second unified 3D augmented map comprising the robot-sensed obstacle characteristics and AV-sensed obstacle characteristics;
receive, via the processor, the second unified 3D augmented map comprising the robot-sensed obstacle characteristics and the AV-sensed obstacle characteristics; and
generate or update the dynamic path plan to the package delivery destination using the second unified 3D augmented map.

11. The system according to claim 10 , wherein the one or more of the robot-sensed obstacle characteristics and the AV-sensed obstacle characteristics comprises an obstacle localization and an obstacle class; and
wherein the obstacle class comprises one of a static obstacle and a dynamic obstacle.

12. The system according to claim 10 , wherein the processor is further programmed to:
receive, from the AV, coordinates for a second AV position indicative of a second AV position in the delivery environment that is different from the first AV position;
generate an updated global reference map with the second AV position;
actuate the robot vehicle to the package delivery destination according to the dynamic path plan and delivering a package at the package delivery destination; and
actuate the robot vehicle to travel to the second AV position based on the updated global reference map.

13. The system according to claim 10 , wherein the processor is further programmed to:
receive an infrastructure sensor data feed comprising infrastructure-sensed obstacle characteristics; and
generate the unified 3D augmented map based in part on the infrastructure-sensed obstacle characteristics.

14. The system according to claim 13 , wherein the one or more of the robot-sensed obstacle characteristics and AV-sensed obstacle characteristics comprises an obstacle localization and an obstacle class; and
wherein the obstacle class comprises one of a static obstacle and a dynamic obstacle.

15. The system according to claim 10 , wherein the sensor is a camera, wherein the sensory system further comprises a LiDAR system and an ultrason

Assignees

• Ford Global Tech Llc

Inventors

• Pandey Gaurav
• Mcbride James
• Miller Justin
• Lu Jianbo
• Chen Yifan

Classifications

• G05D1/0214 Primary

  in accordance with safety or protection criteria, e.g. avoiding hazardous areas (monitoring the location of vehicles within a certain area, e.g. forbidden or allowed areas, in traffic control systems for road vehicles G08G1/13) · CPC title

• G01C21/3859

  Differential updating map data · CPC title

• G01S17/931 Primary

  of land vehicles · CPC title

• G05D1/0274

  using mapping information stored in a memory device (navigation using map-matching G01C21/30) · CPC title

• B62D63/02 Primary

  Motor vehicles · CPC title