Vision-based multi-camera factory monitoring with dynamic integrity scoring

Patent metadata

Field	Value
Publication number	US-9251598-B2
Application number	US-201414249386-A
Country	US
Kind code	B2
Filing date	Apr 10, 2014
Priority date	Apr 10, 2014
Publication date	Feb 2, 2016
Grant date	Feb 2, 2016

Abstract

A human monitoring system includes a plurality of cameras and a visual processor. The plurality of cameras are disposed about a workspace area, where each camera is configured to capture a video feed that includes a plurality of image frames, and the plurality of image frames are time-synchronized between the respective cameras. The visual processor is configured to receive the plurality of image frames from the plurality of vision-based imaging devices and determine an integrity score for each respective image frame. The processor may then isolate a foreground section from two or more of the views, determine a principle body axis for each respective foreground section, and determine a location point according to a weighted least squares function amongst the various principle body axes.

First claim

The invention claimed is:

1. A method of identifying a location of an object within a workspace, the method comprising:
obtaining a plurality of 2D images of the workspace, each respective 2D image being acquired from a camera disposed at a different location within the workspace;
determining, via a processor, an integrity score for each respective 2D image, the integrity score relating to at least one of an absolute pixel difference, a global or a local histogram difference, and an absolute edge difference between the respective 2D image, and a prior 2D image from the same camera;
identifying a foreground portion within at least three of the plurality of 2D images;
determining a principle body axis for each identified foreground portion, the principle body axis being a mean centerline of the respective foreground portion and aligned with a vanishing point of the respective image;
mapping the determined principle body axis from each respective 2D image into a common ground plane that is coincident with a floor of the workspace;
determining a location point within the ground plane, wherein the location point minimizes a least squares function among each mapped principle body axis, wherein the least squares function is weighted according to the integrity score for each of the respective 2D images, and wherein the location point represents a point location of the object within the workspace;
projecting the foreground portion from each respective 2D image to each of a plurality of parallel space planes;
identifying an area within each of the plurality of planes where a plurality of projected foreground portions overlap;
combining the identified area from each of the plurality of planes to form a 3D bounding envelope of the object; and
wherein the bounding envelope is a 3D probabilistic representation of the location of the object within the workspace.

2. The method of claim 1 , further comprising performing a control action if the bounding envelope overlaps with a predefined volume.

3. The method of claim 1 , further comprising recording the coordinates of the location point if the location point is within the bounding envelope.

4. The method of claim 3 , further comprising assembling a motion track, wherein the motion track represents the position of the location point over a period of time; and
identifying a portion of the period of time where the location point is in motion within the workspace, and a portion of the period of time where the location point is stationary within the workspace.

5. The method of claim 4 , further comprising determining an action performed by the object during the portion of the period of time where the location point is stationary within the workspace.

6. The method of claim 1 , further comprising:
determining a primary axis of the bounding envelope, wherein the primary axis of the bounding envelope intersects the ground plane to define a second location point; and
fusing the determined location point within the ground plane with the second location point to form a refined location point.

7. The method of claim 1 , further comprising fusing the bounding envelope with a voxel representation of the workspace to create a refined object primitive.

8. The method of claim 7 , further comprising determining at least one of a velocity and an acceleration of a portion of the refined object primitive.

9. The method of claim 8 , further comprising altering the behavior of an automated device based on the at least one of velocity and acceleration.

10. The method of claim 1 , wherein the plurality of parallel spaced planes includes at least three planes; and
wherein one of the at least three planes includes a ground plane.

11. A system comprising:
a plurality of cameras disposed at different locations within the workspace, and each configured to 2D image the workspace from a different perspective, wherein each respective camera of the plurality of cameras is configured to capture a 2D image of the workspace;
a processor in communication with each of the plurality of cameras and configured to receive the captured 2D image from each of the plurality of cameras, the processor further configured to:
identify a foreground portion within at least two of the plurality of 2D images;
determine an integrity score for each respective 2D image, the integrity score relating to at least one of an absolute pixel difference, a global or a local histogram difference, and an absolute edge difference between the respective 2D image, and a prior 2D image from the same camera;
determine a principle body axis for each identified foreground portion, the principle body axis being a mean centerline of the respective foreground portion and aligned with a vanishing point of image;
map each detected principle body axis into a ground plane that is coincident with a floor of the workspace;
determine a location point within the ground plane, wherein the location point minimizes a least squares function among each mapped principle body axis, wherein the least squares function is weighted according to the integrity score for each of the respective 2D images, and wherein the location point represents a point location of an object within the workspace;
project the foreground portion from each respective 2D image to each of a plurality of parallel spaced planes;
identify an area within each of the plurality of planes where a plurality of projected foreground portions overlap;
combine the identified area from each of the plurality of planes to form a 3D bounding envelope of the object; and
wherein the bounding envelope is a 3D probabilistic representation of the location of the object within the workspace.

12. The system of claim 11 , wherein the processor is further configured to record the coordinates of the location point if the location point is within the bounding envelope.

13. The system of claim 12 , wherein the processor is further configured to:
assemble a motion track, wherein the motion track represents the position of the location point over a period of time; and
identify a portion of the period of time where the location point is in motion within the workspace, and a portion of the period of time where the location point is stationary within the workspace.

14. The system of claim 13 , wherein the processor is further configured to determine an action performed by the object during the portion of the period of time where the location point is stationary within the workspace.

15. The method of claim 11 , wherein the processor is further configured to fuse the ground plane with the plurality of planes to form a planar probability map.

16. The system of claim 11 , wherein the processor is further configured to:
determine a primary axis of the bounding envelope, wherein the primary axis of the bounding envelope intersects the ground plane to define a second location point; and
fuse the determined location point within the ground plane with the second location point to form a refined location point.

Assignees

• Gm Global Tech Operations Inc

Inventors

• Wells James W
• Kim Kyungnam
• Medasani Swarup
• Owechko Yuri

Classifications

• G06T7/2093 Primary

  Physics · mapped topic

• G06T2207/10004

  Still image; Photographic image · CPC title

• G06T2207/30196

  Human being; Person · CPC title

• H04N7/181 Primary

  for receiving images from a plurality of remote sources · CPC title

• G06V2201/12

  Acquisition of 3D measurements of objects · CPC title