Method and apparatus for projective volume monitoring

Patent metadata

Field	Value
Publication number	US-9501692-B2
Application number	US-201213650461-A
Country	US
Kind code	B2
Filing date	Oct 12, 2012
Priority date	Oct 14, 2011
Publication date	Nov 22, 2016
Grant date	Nov 22, 2016

Abstract

According to one aspect of the teachings presented herein, a projective volume monitoring apparatus is configured to detect objects intruding into a monitoring zone. The projective volume monitoring apparatus is configured to detect the intrusion of objects of a minimum object size relative to a protection boundary, based on an advantageous processing technique that represents range pixels obtained from stereo correlation processing in spherical coordinates and maps those range pixels to a two-dimensional histogram that is defined over the projective coordinate space associated with capturing the stereo images used in correlation processing. The histogram quantizes the horizontal and vertical solid angle ranges of the projective coordinate space into a grid of cells. The apparatus flags range pixels that are within the protection boundary and accumulates them into corresponding cells of the histogram, and then performs clustering on the histogram cells to detect object intrusions.

First claim

What is claimed is:

1. A method of detecting objects intruding into a monitoring zone, said method performed by a projective volume monitoring apparatus and comprising:
capturing a stereo image from a pair of image sensors;
correlating the stereo image to obtain a depth map comprising range pixels represented in three-dimensional Cartesian coordinates;
converting the range pixels into spherical coordinates, so that each range pixel is represented as a radial distance along a respective pixel ray and a corresponding pair of solid angle values within the horizontal and vertical fields of view associated with capturing the stereo image;
obtaining a set of flagged pixels by flagging those range pixels that fall within a protection boundary defined for the monitoring zone;
accumulating the flagged pixels into corresponding cells of a two-dimensional histogram that quantizes the solid angle ranges of the horizontal and vertical fields of view; and
clustering cells in the histogram to detect intrusions of objects within the protection boundary that meet a minimum object size threshold.

2. The method of claim 1 , wherein clustering cells in the histogram comprises identifying qualified cells as those cells that accumulated at least a minimum number of the flagged pixels, and identifying qualified clusters as those clusters of qualified cells that meet a minimum cluster size corresponding to the minimum object size threshold.

3. The method of claim 2 , wherein clustering cells in the histogram further comprises determining whether any qualified clusters persist for a defined window of time and, if so, determining that an object intrusion has occurred.

4. The method of claim 2 , wherein qualified clusters are identified for the stereo image captured in each of a number of successive image frames, and further comprising filtering the qualified clusters over time by determining whether a same qualified cluster persists over a defined number of image frames and, if so, determining that an object intrusion has occurred.

5. The method of claim 4 , wherein filtering the qualified clusters over time further includes determining whether equivalent qualified clusters persist over the defined number of image frames and, if so, determining that an object intrusion has occurred, wherein equivalent qualified clusters are those qualified clusters that split or merge within a same region of the histogram over the defined number of image frames.

6. The method of claim 1 , further comprising generating one or more sets of mitigation pixels from the stereo image, wherein the mitigation pixels represent pixels in the stereo image that are flagged as being faulty or unreliable, and integrating the one or more sets of mitigation pixels into the set of flagged pixels for accumulation in corresponding cells of the histogram, along with the range pixels flagged for inclusion in the set of flagged pixels, so that any given cluster includes cells that accumulated range pixels, mitigation pixels, or a mix of both.

7. The method of claim 6 , wherein generating the one or more sets of mitigation pixels comprises detecting at least one of: pixels in the stereo image that are outside of a usable dynamic range; pixels in the stereo image that are stuck; pixels in the stereo image that have reduced sensitivity; pixels in the stereo image that are noisy; and pixels in the stereo image that are identified as corresponding to shadowed regions of the monitoring zone.

8. The method of claim 1 , wherein capturing the stereo image comprises capturing corresponding high exposure and low exposure stereo images and forming a high dynamic range stereo image from the corresponding high and low exposure stereo images, and further comprising obtaining a rectified high dynamic range stereo image by rectifying the high dynamic range stereo image, so that correlation processing is performed on the rectified high dynamic range stereo image.

9. The method of claim 8 , further comprising flagging pixels in the rectified high dynamic range stereo image as mitigation pixels that are deemed bad or unreliable for range detection purposes, and further comprising adding the mitigation pixels to the set of flagged pixels, such that clustering considers both the range pixels and the mitigation pixels in the set of flagged pixels.

10. The method of claim 8 , further comprising capturing the stereo image in each one of a number of image frames, and correspondingly obtaining a rectified high dynamic range stereo image in each such image frame, and controlling exposure timing used for capturing the low and high exposure stereo images in each image frame as a function of intensity statistics determined from the high dynamic range stereo images or from the rectified high dynamic range stereo images.

11. The method of claim 1 , wherein capturing the stereo image comprises capturing a first stereo image in one or more image frames using a first pair of image sensors operating with a first baseline, and further comprising capturing a redundant second stereo image in the one or more image frames using a second pair of image sensors operating with a second baseline, and wherein said method steps of correlating, converting, obtaining, accumulating and clustering are performed independently for the first and second stereo images, and wherein the method further includes deciding that an object intrusion has occurred if there is a threshold level of correspondence between object intrusions detected with respect to the first stereo images and object intrusions detected with respect to the second stereo images.

12. The method of claim 1 , further comprising suppressing false detection of objection intrusions by re-computing depth information at a higher reliability for the ranged pixels in the depth map that were flagged for inclusion in the set of flagged pixels and evaluating the recomputed depth information to verify that each range pixel in the set of flagged pixels falls within the protection boundary.

13. The method of claim 1 , further comprising suppressing epipolar errors in the clusters to obtain error-corrected clusters.

14. The method of claim 1 , where the monitoring zone comprises two overlapping protection zones defined by respective protection boundaries, and further comprising monitoring a first one of the overlapping protection zones using detection parameters configured for detecting persons walking or running and monitoring a second one of the overlapping protection zones using detection parameters configured for detecting crawling and prone persons.

15. The method of claim 1 , wherein clustering cells in the histogram to detect intrusions of objects within the protection boundary that meet a minimum object size threshold includes temporally filtering detected clusters to distinguish between clusters arising from noise and clusters arising from object intrusions.

16. A projective volume monitoring apparatus configured to detect objects intruding into a monitoring zone, said projective volume monitoring apparatus comprising:
image sensors configured to capture a stereo image; and
image processing circuits operatively associated with the image sensors and configured to:
correlate the stereo image to obtain a depth map comprising pixels represented in three-dimensional Cartesian coordinates;
convert the range pixels into spherical coordinates, so that each range pixel is represented as a radial distance along a respective pixel ray and a corresponding pair of solid angle values within the horizontal and vertical fields of view associated with capturing the stereo image;
obtai

Assignees

• Omron Tateisi Electronics Co

Inventors

• Tyagi Ambrish
• Drinkard John
• Tani Yoshiharu
• Kinoshita Koichi

Classifications

• F16P3/144 Primary

  using light grids · CPC title

• G06V40/103

  Static body considered as a whole, e.g. static pedestrian or occupant recognition · CPC title

• G06K9/00369 Primary

  Physics · mapped topic

• G06T7/0022

  Physics · mapped topic

• G06V40/25

  Recognition of walking or running movements, e.g. gait recognition · CPC title