Multi-view human detection using semi-exhaustive search

Patent metadata

Field	Value
Publication number	US-9524426-B2
Application number	US-201414219109-A
Country	US
Kind code	B2
Filing date	Mar 19, 2014
Priority date	Mar 19, 2014
Publication date	Dec 20, 2016
Grant date	Dec 20, 2016

Abstract

A human monitoring system includes a plurality of cameras and a visual processor. The plurality of cameras are disposed about a workspace area, where each camera is configured to capture a video feed that includes a plurality of image frames, and the plurality of image frames are time-synchronized between the respective cameras. The visual processor is configured to receive the plurality of image frames from the plurality of vision-based imaging devices and detect the presence of a human from at least one of the plurality of image frames using pattern matching performed on an input image. The input image to the pattern matching is a sliding window portion of the image frame that is aligned with a rectified coordinate system such that a vertical axis in the workspace area is aligned with a vertical axis of the input image.

First claim

The invention claimed is:

1. A human monitoring system for monitoring a workspace area having automated moveable equipment, the system comprising:
a plurality of cameras disposed about the workspace area, each camera configured to capture a video feed that includes a plurality of image frames;
a visual processor configured to:
receive the plurality of image frames from the plurality of cameras;
detect the presence of a human from at least one of the plurality of image frames using pattern matching performed on an input image;
wherein the input image to the pattern matching is a sliding window portion of the at least one image frame;
wherein the sliding window portion is aligned with a rectified coordinate system such that a vertical axis in the workspace area is aligned with a vertical axis of the input image; and

provide an alert if the detected human is proximate the automated moveable equipment.

2. The system of claim 1 , wherein the rectified coordinate system is established according to at least one of a perspective of the workspace area within the at least one image frame, and a vanishing point of the at least one image frame.

3. The system of claim 1 , wherein the pattern matching includes at least one of a support vector machine and a neural network.

4. The system of claim 1 , wherein the pattern matching can further detect the pose of the human, and wherein the pose includes at least one of standing, walking, reaching, and crouching.

5. The system of claim 1 , wherein the visual processor is further configured to select the sliding window from a region of interest within the at least one image frame;
wherein the region of interest is a subset of the at least one image frame that is a possible area for a human to be located; and
wherein the region of interest includes a portion of the floor of the workspace area.

6. The system of claim 5 , wherein the sliding window traverses the entire region of interest in a first image frame prior to detecting the presence of a human a subsequent image frame.

7. The system of claim 1 , wherein the visual processor uses a position and a velocity of a detected human in a first frame to prioritize the detection in a subsequent frame.

8. The system of claim 1 , wherein the visual processor is further configured to fuse the time-synchronized image frames from the plurality of views into a common coordinate system; and
wherein visual processor is configured to identify the location of the detected human in the common coordinate system by mapping a representation of the human from the plurality of views into the common coordinate system, and by determining a point of intersection of the mapped representations.

9. The system of claim 8 , wherein the visual processor is further configured to assemble a motion track of the detected human according to the identified location across a plurality of successive image frames.

10. The system of claim 9 , wherein the visual processor is further configured to:
compare the motion track to a predetermined, expected motion track; and
provide an alert if the motion track is not similar to the expected motion track.

11. A human monitoring system for monitoring a workspace area having automated moveable equipment, the system comprising:
a plurality of cameras disposed about the workspace area, each camera configured to capture a video feed that includes a plurality of image frames;
a visual processor configured to:
receive the plurality of image frames from the plurality of cameras;
detect the presence of a human from at least one of the plurality of image frames using a support vector machine executed using an input image;
wherein the input image to the support vector machine is a sliding window portion of the at least one image frame;
wherein the sliding window portion is aligned with a rectified coordinate system such that a vertical axis in the workspace area is aligned with a vertical axis of the input image;
wherein the rectified coordinate system is established according to at least one of a perspective of the workspace area within the at least one image frame, and a vanishing point of the at least one image frame; and

provide an alert if the detected human is proximate the automated moveable equipment.

12. The system of claim 11 , wherein the support vector machine can further detect the pose of the human, and wherein the pose includes at least one of standing, walking, reaching, and crouching.

13. The system of claim 11 , wherein the visual processor is further configured to select the sliding window from a region of interest within the at least one image frame;
wherein the region of interest is a subset of the at least one image frame that is a possible area for a human to be located; and
wherein the region of interest includes a portion of the floor of the workspace area.

14. The system of claim 13 , wherein the sliding window traverses the entire region of interest in a first image frame prior to detecting the presence of a human a subsequent image frame.

15. The system of claim 11 , wherein the visual processor uses a position and a velocity of a detected human in a first frame to prioritize the detection in a subsequent frame.

16. The system of claim 11 , wherein the visual processor is further configured to fuse the time-synchronized image frames from the plurality of views into a common coordinate system; and
wherein visual processor is configured to identify the location of the detected human in the common coordinate system by mapping a representation of the human from the plurality of views into the common coordinate system, and by determining a point of intersection of the mapped representations.

17. The system of claim 16 , wherein the visual processor is further configured to assemble a motion track of the detected human according to the identified location across a plurality of successive image frames.

18. The system of claim 17 , wherein the visual processor is further configured to:
compare the motion track to a predetermined, expected motion track; and
provide an alert if the motion track is not similar to the expected motion track.

Assignees

• Gm Global Tech Operations Llc

Inventors

• Kim Kyungnam
• Jeong Changsoo

Classifications

• G06V10/764 Primary

  using classification, e.g. of video objects · CPC title

• G06V40/103

  Static body considered as a whole, e.g. static pedestrian or occupant recognition · CPC title

• G06F18/2411

  based on the proximity to a decision surface, e.g. support vector machines · CPC title

• H04N7/181

  for receiving images from a plurality of remote sources · CPC title

• G06V10/462

  Salient features, e.g. scale invariant feature transforms [SIFT] · CPC title