Visually indicating vehicle caution regions

Patent metadata

Field	Value
Publication number	US-10875448-B2
Application number	US-202016862070-A
Country	US
Kind code	B2
Filing date	Apr 29, 2020
Priority date	Dec 27, 2017
Publication date	Dec 29, 2020
Grant date	Dec 29, 2020

Abstract

An example system includes a vehicle, a light projector connected to the vehicle, and a control system. The control system is configured to determine a planned operating region for the vehicle within an environment. The control system is also configured to determine that the planned operating region is within a threshold distance of an object within the environment and, in response, determine a caution region to illuminate with the light projector near the object. The control system is further configured to cause the light projector to project an indication of the caution region near the object. The projected indication remains fixed in relation to the object as the vehicle moves toward the planned operating region.

First claim

What is claimed is:

1. A computer-implemented method comprising:
determining a planned operating region for a robotic device within an environment;
determining that the planned operating region is within a threshold distance of an object within the environment;
in response to determining that the planned operating region is within the threshold distance of the object, determining a caution region to illuminate with a light projector, wherein the light projector is connected to the robotic device and repositionable with respect to the robotic device in one or more degrees of freedom;
causing the robotic device to move towards the planned operating region; and
while the robotic device moves towards the planned operating region, causing the light projector to project an indication of the caution region by: (i) repositioning the light projector with respect to the robotic device to project the indication of the caution region, (ii) determining a keystone correction to apply to the indication of the caution region based on a pose of the light projector with respect to the robotic device and a pose of the robotic device with respect to the caution region, and (ii) projecting the indication of the caution region modified according to the keystone correction.

2. The computer-implemented method of claim 1 , wherein determining the keystone correction to apply to the indication of the caution region comprises:
determining, at a first time point preceding a second time point, the keystone correction for the second time point based on the pose of the light projector with respect to the robotic device planned for the second time point and the pose of the robotic device with respect to the caution region planned for the second time point, wherein the indication of the caution region projected at the second time point is modified according to the keystone correction for the second time point.

3. The computer-implemented method of claim 1 , wherein determining the keystone correction to apply to the indication of the caution region comprises:
updating the keystone correction based on one or more of: (i) a change in the pose of the light projector with respect to the robotic device, or (ii) a change in the pose of the robotic device with respect to the caution region, such that a visual appearance of the indication of the caution region remains visually consistent.

4. The computer-implemented method of claim 1 , wherein determining the keystone correction to apply to the indication of the caution region comprises:
receiving, from a camera, image data representing the indication of the caution region; and
determining the keystone correction to apply to the indication of the caution region further based on the image data.

5. The computer-implemented method of claim 1 , wherein the indication is projected at a fixed position within the environment while the robotic device moves toward the planned operating region.

6. The computer-implemented method of claim 1 , wherein at least part of the indication of the caution region is projected onto a ground surface of the environment.

7. The computer-implemented method of claim 1 , wherein at least part of the indication of the caution region is projected onto the object.

8. The computer-implemented method of claim 1 , wherein the caution region comprises a buffer area within a second threshold distance of at least part of a perimeter of the object.

9. The computer-implemented method of claim 1 , wherein determining the caution region to illuminate with the light projector comprises:
determining an intersection between the planned operating region and a first area, wherein the first area (i) surrounds a perimeter of the object and (ii) is within the threshold distance of the perimeter of the object;
determining a second area between the intersection and the perimeter of the object; and
combining the intersection and the second area to form the caution region.

10. The computer-implemented method of claim 1 , further comprising:
causing the light projector to continue projecting at least part of the indication of the caution region while the planned operating region is occupied by the robotic device.

11. The computer-implemented method of claim 1 , wherein the planned operating region comprises an area to be occupied by the robotic device while interacting with the object, and wherein the caution region comprises an area between the planned operating region and the object.

12. The computer-implemented method of claim 1 , further comprising:
receiving, from one or more sensors connected to the robotic device, sensor data representing an occupant of the environment;
determining, based on the sensor data, that the occupant is predicted to be present in at least one of the planned operating region or the caution region; and
based on determining that the occupant is predicted to be present in the at least one of the planned operating region or the caution region, generating warning signal comprising at least one of: (i) an adjustment to a visual appearance of the indication of the caution region, or (ii) an audible warning.

13. The computer-implemented method of claim 12 , further comprising:
determining a path for the occupant to follow in the environment to avoid the planned operating region and the caution region; and
causing the light projector to project an additional indication of the path.

14. The computer-implemented method of claim 1 , wherein the threshold distance is a first threshold distance, and wherein the method further comprises:
determining, based on the planned operating region, a safety region within the environment planned to be unoccupied by the robotic device, wherein the safety region is beyond a second threshold distance of the planned operating region; and
cause the light projector to project an additional indication of the safety region.

15. The computer-implemented method of claim 1 , further comprising:
determining, based on a position of the planned operating region and a position of the object within the environment, a distance between the planned operating region and the object; and
determining, based on the distance between the planned operating region and the object, a size of the caution region, wherein the indication of the caution region projected by the light projector has the determined size.

16. The computer-implemented method of claim 1 , further comprising:
determining a size of the caution region based on a speed planned for the robotic device within the planned operating region, wherein the indication of the caution region projected by the light projector has the determined size.

17. A system comprising:
a robotic device;
a light projector connected to the robotic device and repositionable with respect to the robotic device in one or more degrees of freedom; and
a control system comprising a processor and a non-transitory computer-readable storage medium having stored therein instructions that, when executed by the processor, cause the processor to perform operations comprising:
determining a planned operating region for a robotic device within an environment;
determining that the planned operating region is within a threshold distance of an object within the environment;
in response to determining that the planned operating region is within the threshold distance of the object, determining a caution region to illuminate with the light projector;
causing the robotic device to move towards the planned operating region; and
while the robotic device moves towards the planned operat

Assignees

• X Dev Llc

Inventors

• Zevenbergen John
• Holmberg Robert
• Fischer Mark

Classifications

• B60Q1/525

  automatically indicating risk of collision between vehicles in traffic or with pedestrians, e.g. after risk assessment using the vehicle sensor data · CPC title

• B60Q1/507

  specific to autonomous vehicles · CPC title

• B60Q1/543

  for indicating other states or conditions of the vehicle (indicating taxis' state of hire or fare G07B13/045) · CPC title

• G01C21/20 Primary

  Instruments for performing navigational calculations (G01C21/24, G01C21/26 take precedence) · CPC title

• B66F17/003

  for fork-lift trucks · CPC title