Time of flight system and method for safety-rated collision avoidance

Patent metadata

Field	Value
Publication number	US-11669092-B2
Application number	US-201916555084-A
Country	US
Kind code	B2
Filing date	Aug 29, 2019
Priority date	Aug 29, 2019
Publication date	Jun 6, 2023
Grant date	Jun 6, 2023

Abstract

A safety system for autonomously mobile machinery (e.g., automated guided vehicles) achieves safety-rated collision avoidance functionality by detecting objects located in the field of view of a three-dimensional (3D) time-of-flight (TOF) vision system or camera. Incorporating a 3D TOF camera into a collision avoidance system allows a large volume to be monitored for object intrusion, improving reliability of object detection. To ensure reliability of the safety system's obstacle detection capabilities, the collision avoidance system also includes self-diagnostic capabilities that verify the accuracy of the TOF camera's distance measurements even in the absence of a test object within the camera's field of view. This is achieved by tilting the TOF camera downward to include the floor within the camera's field of view, allowing the floor to act as a test object that can be leveraged to verify accuracy of the camera's distance measurements.

First claim

What is claimed is:

1. A collision avoidance system, comprising:
a three-dimensional camera configured to mount on a front side of a mobile asset at a downward incline that causes a portion of a ground or floor in front of the mobile asset to be included in a field of view of the three-dimensional camera, the three-dimensional camera comprising:
an emitter component configured to emit light pulses into a space in front of the mobile asset;
a photo-sensor component comprising an array of pixels, wherein respective pixels of the array of pixels are configured to convert a subset of the light pulses received from surfaces within the space to electrical energy proportional to the subset of the light pulses received at the pixels; and
a distance determination component configured to:
determine, based on analysis of the electrical energy, measured distance values d so representing first distances from the respective pixels to corresponding points within the space in front of the mobile asset along optical axes of the pixels, and
determine calculated distance values d vo representing second distances from the points within the space to the mobile asset along lines substantially parallel to a slope of the ground or floor based on the measured distance values d so , field of view angles β of the respective pixels representing angles between the respective pixels' optical axes and an upper boundary of the field of view of the three-dimensional camera, and an inclination angle α of the three-dimensional camera relative to horizontal; and

a control output component configured to generate a control output in response to a determination that at least a subset of the calculated distance values d vo satisfy a defined criterion indicative of a presence of an obstacle within a protective field of the three-dimensional camera, wherein the control output is configured to initiate a safety action that alters operation of the mobile asset to mitigate a collision with the obstacle.

2. The collision avoidance system of claim 1 , wherein the mobile asset is at least one of an automated guided vehicle (AGV), an autonomous mobile robot (AMR), a mobile industrial machine, an automated warehouse, an automated car, an automated amusement park ride, or a non-autonomous vehicle.

3. The collision avoidance system of claim 1 , wherein the defined criterion specifies that the control output is to be generated if a number of pixels having associated calculated distance values d vo that are less than a minimum safe distance exceeds a defined threshold number of pixels.

4. The collision avoidance system of claim 1 , wherein the distance determination component determines the calculated distance value d vo for a pixel, of the pixels, based on a product of the measured distance value d so for the pixel and a cosine of a sum of the field of view angle β for the pixel and the inclination angle α.

5. The collision avoidance system of claim 1 , wherein
the control output component is configured to generate the control output further in response to determining that at least the subset of the calculated distance values d vo that satisfy the defined criterion are associated with pixels that are within a section of the array of pixels corresponding to the protective field of the three-dimensional camera, and
the collision avoidance system further comprises a level component configured to:
measure a direction of inclination and a degree of inclination of the mobile asset, and
initiate an adjustment of the protective field based on the direction of inclination and the degree of inclination.

6. The collision avoidance system of claim 1 , further comprising a diagnostic component configured to initiate a diagnostic sequence that verifies accuracies of the measured distance values d so ,
wherein
the light pulses are first light pulses,
the subset of the light pulses are a first subset of the light pulses,
the electrical energy is first electrical energy,
the emitter component is configured to, during the diagnostic sequence, emit second light pulses into the space in front of the mobile asset while no obstacles are present in the space,
the photo-sensor component is configured to, during the diagnostic sequence, convert a second subset of the light pulses reflected from the ground or floor and received at a subset of the pixels corresponding to the ground or floor to second electrical energy proportional to the second subset of the light pulses,
the distance determination component is configured to, during the diagnostic sequence,
determine measured distance values d sg associated with respective pixels of the subset of the pixels corresponding to the ground or floor based on analysis of the second electrical energy,
compare the measured distance values d sg with expected values of d sg for the respective pixels of the subset of pixels, and
in response to determining that a number of the measured distance values d sg greater than a defined threshold number deviate from their corresponding expected values of d sg in excess of a defined tolerance, instruct the control output component to initiate another control output that places the mobile asset in a safe state.

7. The collision avoidance system of claim 6 , wherein the distance determination component is configured to determine the expected values of d sg corresponding to the respective pixels of the subset of the pixels based on a height h c of the three-dimensional camera from the ground or floor, the field of view angles β of the respective pixels of the subset of the pixels, and the inclination angle α of the three-dimensional camera.

8. The collision avoidance system of claim 1 , wherein the safety action is at least one of a divergence of the mobile asset's path of travel, a slowing of the mobile asset, or a stoppage of the mobile asset.

9. The collision avoidance system of claim 1 , wherein the three-dimensional camera is configured to mount to the mobile asset using a gimbal stabilizer that maintains a consistent average inclination angle α of the three-dimensional camera.

10. A method for automated collision avoidance, comprising:
emitting, by a three-dimensional camera mounted on a front side of an automated moving machine at a downward tilt, light pulses into a monitored area in front of the automated moving machine, wherein the downward tilt is such that a portion of a ground or floor in front of the automated moving machine is within a field of view of the three-dimensional camera;
for respective pixels of a pixel array of the three-dimensional camera, generating, by the three-dimensional camera, electrical energy proportional to a subset of the light pulses received at the respective pixels from surfaces within the monitored area;
determining, by the three-dimensional camera based on analysis of the electrical energy, measured distance values d so representing first distances from the respective pixels to corresponding points within the monitored area in front of the automated moving machine along optical paths of the pixels;
determining, by the three-dimensional camera, calculated distance values d vo representing second distances from the points within the monitored area to the automated moving machine along lines substantially parallel to a slope of the ground or floor based on the measured distance values d so , field of view angles β of the respective pixels defining angles between optical axes of the respective pixels' and an upper boundary of the field of view of the three-dimensional camera, and an inclination angle α of the three-dimensional camera relative to horizontal; and
in response to determining that at least a subset of the calculated distance valu

Assignees

• Rockwell Automation Tech Inc

Inventors

• Boutaud Frederic
• Freeman Linda A

Classifications

• G06T7/75

  involving models · CPC title

• G01S17/931 Primary

  of land vehicles · CPC title

• G01S17/894

  Three-dimensional [3D] imaging with simultaneous measurement of time-of-flight at a two-dimensional [2D] array of receiver pixels, e.g. time-of-flight cameras or flash lidar · CPC title

• G06V20/58

  Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads · CPC title

• G06T2207/10028

  Range image; Depth image; 3D point clouds · CPC title