Optoelectronic sensor and method of detecting objects

Patent metadata

Field	Value
Publication number	US-12105202-B2
Application number	US-202017076187-A
Country	US
Kind code	B2
Filing date	Oct 21, 2020
Priority date	Nov 7, 2019
Publication date	Oct 1, 2024
Grant date	Oct 1, 2024

Abstract

An optoelectronic sensor for detecting objects in a monitored zone is provided, wherein the sensor comprises a laser scanner having a deflection unit rotatable about an axis of rotation for scanning the monitored zone with at least one scanning beam; a first distance measurement unit for determining 3D measurement points of the respective objects impacted by the scanning beam using a time-of-flight method; a panorama camera having a panorama optics and having an image sensor with a plurality of light reception elements for detecting picture elements; and a control and evaluation unit that is configured to fuse the 3D measurement points and the picture elements. In this respect, the optical axis of the panorama camera and the rotation axis coincide.

First claim

The invention claimed is:

1. An optoelectronic sensor for detecting objects in a monitored zone, wherein the optoelectronic sensor comprises:
a laser scanner having a light transmitter, a light receiver, and a deflection unit, wherein the deflection unit is rotatable about an axis of rotation for scanning the monitored zone with at least one scanning beam;
a first distance measurement unit for determining, using a received signal of the light receiver, three-dimensional (3D) measurement points of the respective objects impacted by the scanning beam using a time-of-flight method;
a panorama camera having panorama optics and having an image sensor with a plurality of light reception elements for detecting picture elements; and
a control and evaluation unit that is configured to transform the 3D measurement points of the laser scanner and the picture elements of the panorama camera into a common coordinate system and fuse the transformed 3D measurement points of the laser scanner and the transformed picture elements of the panorama camera in the common coordinate system,
wherein an optical axis of the panorama camera and the axis of rotation coincide.

2. The optoelectronic sensor in accordance with claim 1 ,
wherein the panorama camera is configured as a time-of-flight camera.

3. The optoelectronic sensor in accordance with claim 1 ,
wherein the image sensor is arranged on the axis of rotation.

4. The optoelectronic sensor in accordance with claim 3 ,
wherein the image sensor is oriented perpendicular to the axis of rotation and/or centered about the axis of rotation.

5. The optoelectronic sensor in accordance with claim 1 ,
wherein the control and evaluation unit is configured to associate at least one angular position of the deflection unit with light reception elements on a radial line of the image sensor.

6. The optoelectronic sensor in accordance with claim 1 ,
wherein the control and evaluation unit is configured to associate light reception elements at a radial distance from the optical axis with a scanning beam.

7. The optoelectronic sensor in accordance with claim 1 ,
wherein the laser scanner and the panorama camera are arranged directly above one another on the axis of rotation.

8. The optoelectronic sensor in accordance with claim 1 ,
wherein the control and evaluation unit is configured to recognize a light spot generated on the image sensor by a scanning beam and to associate 3D measurement points and picture elements with one another on the basis of said light spot.

9. The optoelectronic sensor in accordance with claim 1 ,
wherein the image sensor is an event-based image sensor.

10. The optoelectronic sensor in accordance with claim 9 ,
wherein the light reception elements are configured to determine when the intensity detected by the respective light reception element changes and to exactly then deliver a piece of image information in an event-based manner.

11. The optoelectronic sensor in accordance with claim 10 ,
wherein a respective light reception element delivers, as image information, a piece of differential information as to whether the intensity has decreased or increased and/or an integrated intensity in a time window defined by a change of the intensity.

12. The optoelectronic sensor in accordance with claim 1 ,
wherein at least one of the laser scanner and the panorama camera is configured as a module that can be removed from the optoelectronic sensor or added to the optoelectronic sensor.

13. The optoelectronic sensor in accordance with claim 12 ,
wherein the control and evaluation unit is configured to calibrate the laser scanner and the panorama camera with respect to one another on a subsequent addition of a module.

14. The optoelectronic sensor in accordance with claim 1 ,
wherein the laser scanner and the panorama camera are arranged with a mutual offset.

15. The optoelectronic sensor in accordance with claim 1 , wherein the control and evaluation unit that is further configured to perform calibration on the laser scanner and the panorama camera to establish the common coordinate system.

16. A method of detecting objects in a monitored zone, comprising:
scanning the monitored zone with at least one scanning beam generated by a laser scanner, the laser scanner having a light transmitter, a light receiver, and a deflection unit, wherein the deflection unit is rotatable about an axis of rotation;
determining three-dimensional (3D) measurement points of the respective objects impacted by the scanning beam using a received signal of the light receiver and a time-of-flight method;
detecting the monitored zone using a panorama camera having panorama optics and having an image sensor with a plurality of light reception elements in order to obtain picture elements;
transforming the 3D measurement points of the laser scanner and the picture elements of the panorama camera into a common coordinate system; and
fusing the transformed 3D measurement points of the laser scanner and the transformed picture elements of the panorama camera in the common coordinate system,
wherein an optical axis of the panorama camera and the axis of rotation coincide.

17. The method in accordance with claim 16 ,
wherein the fusing associates a scanning angle of the scanning unit with an azimuth angle on the image sensor and associates an elevation angle of the scanning beam with a radial distance from the axis of rotation on the image sensor.

18. The method in accordance with claim 17 , wherein the fusing further comprises imaging of scanning planes of the scanning unit onto concentric circular rings about the axis of rotation on the image sensor.

19. The method in accordance with claim 16 ,
wherein the laser scanner and the panorama camera are arranged with a mutual offset.

20. The method in accordance with claim 16 , further comprising performing calibration on the laser scanner and the panorama camera to establish the common coordinate system.

Assignees

• Sick Ag

Inventors

• Russ Thomas
• Tschuch Sebastian
• Müller Romain

Classifications

• H04N23/698

  for achieving an enlarged field of view, e.g. panoramic image capture · CPC title

• H04N23/695

  Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects · CPC title

• G01S17/08

  for measuring distance only (indirect measurement G01S17/46; active triangulation systems G01S17/48) · CPC title

• G01S7/497

  Means for monitoring or calibrating · CPC title

• G01S7/4817

  relating to scanning · CPC title