Automatic scene calibration method for video analytics

Patent metadata

Field	Value
Publication number	US-10372970-B2
Application number	US-201615266747-A
Country	US
Kind code	B2
Filing date	Sep 15, 2016
Priority date	Sep 15, 2016
Publication date	Aug 6, 2019
Grant date	Aug 6, 2019

Abstract

To determine real-world information about objects moving in a scene, the camera capturing the scene is typically calibrated to the scene. Automatic scene calibration can be accomplished using people that are found moving about in the scene. During a calibration period, a video content analysis system processing video frames from a camera can identify blobs that are associated with people. Using an estimated height of a typical person, the video content analysis system can use the location of the person's head and feet to determine a mapping between the person's location in the 2-D video frame and the person's location in the 3-D real world. This mapping can be used to determine a cost for estimated extrinsic parameters for the camera. Using a hierarchical global estimation mechanism, the video content analysis system can determine the estimated extrinsic parameters with the lowest cost.

First claim

What is claimed is:

1. A method for automated scene calibration, comprising:
determining a blob from a current video frame;
identifying the blob as associated with an object, the blob including pixels that represent at least a portion of the object;
determining, using the blob, a ground plane for the current video frame, wherein the ground plane represents a surface upon which the object is positioned;
selecting approximate three-dimensional points on the ground plane;
estimating extrinsic parameters for a camera model;
determining, using the camera model and the estimated extrinsic parameters, two-dimensional coordinates within the current video frame that correspond to the approximate three-dimensional points; and
determining, using the two-dimensional coordinates and the ground plane, values for a homographic matrix, wherein a homographic transformation using the values for the homographic matrix provides a mapping from the two-dimensional coordinates in the current video frame to three-dimensional real-world points.

2. The method of claim 1 , wherein determining the two-dimensional coordinates includes using virtual intrinsic parameters, wherein the virtual intrinsic parameters include at least a focal length and an optical center.

3. The method of claim 1 , wherein at least the portion of the object is less than a whole of the object, wherein the object includes a person, wherein the pixels included in the blob include at least an upper body of the person, and wherein determining the ground plane includes using an estimated height of the person to locate an approximate position of one or both feet of the person.

4. The method of claim 1 , wherein at least the portion of the object is less than a whole of the object, wherein the object includes a person, wherein the pixels included in the blob include at least a face of the person, and wherein determining the ground plane includes using an estimated distance between eyes of the person and an estimated height of the person to locate an approximate position of one or both feet of the person.

5. The method of claim 1 , further comprising:
using random sample consensus to modify the estimated extrinsic parameters.

6. The method of claim 1 , further comprising:
determining, using a cost function, a cost value for the estimated extrinsic parameters, wherein determining the cost value includes:
determining an estimated height of an object in the current video frame using the estimated extrinsic parameters;
determining a detected height of the object using coordinates of the object within the current video frame; and
comparing the estimated height and the detected height using the cost function.

7. The method of claim 6 , wherein determining the estimated height includes:
determining, using the homographic matrix, a three-dimensional point for two-dimensional coordinates of a bottom the object, wherein the two-dimensional coordinates are within the current video frame; and
determining two-dimensional coordinates of a top of the object using the camera model and an estimated real-world height of the object.

8. The method of claim 6 , further comprising:
determining a plurality cost values for a plurality of extrinsic parameters, the plurality of cost values including the cost value; and
identifying from the plurality of cost values a set of extrinsic parameters with a lowest cost value.

9. The method of claim 6 , wherein the cost function is a size-pose-based cost function.

10. An apparatus, comprising:
a memory configured to store video data; and
a processor configured to:
determine a blob from a current video frame;
identify the blob as associated with an object, the blob including pixels that represent at least a portion of the object;
determine, using the blob, a ground plane for the current video frame, wherein the ground plane represents a surface upon which the object is positioned;
select approximate three-dimensional points on the ground plane;
estimate extrinsic parameters for a camera model;
determine, using the camera model and the estimated extrinsic parameters, two-dimensional coordinates within the current video frame that correspond to the approximate three-dimensional points; and
determine, using the two-dimensional coordinates and the ground plane, values for a homographic matrix, wherein a homographic transformation using the values for the homographic matrix provides a mapping from the two-dimensional coordinates in the current video frame to three-dimensional real-world points.

11. The apparatus of claim 10 , wherein the camera model provides a mapping from three-dimensional real-world points to two-dimensional coordinates in the current video frame.

12. The apparatus of claim 10 , wherein homographic transformation provides a mapping from one coordinate system to another coordinate system.

13. The apparatus of claim 10 , wherein extrinsic parameters include at least three rotational parameters and two translational parameters.

14. The apparatus of claim 10 , wherein the processor is configured to determine the two-dimensional coordinates using virtual intrinsic parameters, wherein the virtual intrinsic parameters include at least a focal length and an optical center.

15. The apparatus of claim 10 , wherein at least the portion of the object is less than a whole of the object, wherein the object includes a person, wherein the pixels included in the blob include at least an upper body of the person, and wherein the processor is configured to determine the ground plane using an estimated height of the person to locate an approximate position of one or both feet of the person.

16. The apparatus of claim 10 , wherein at least the portion of the object is less than a whole of the object, wherein the object includes a person, wherein the pixels included in the blob include at least a face of the person, and wherein the processor is configured to determine the ground plane using an estimated distance between eyes of the person and an estimated height of the person to locate an approximate position of one or both feet of the person.

17. The apparatus of claim 10 , wherein the processor is further configured to:
use random sample consensus to modify the estimated extrinsic parameters.

18. The apparatus of claim 10 , wherein the processor is further configured to:
determine, using a cost function, a cost value for the estimated extrinsic parameters, wherein determining the cost value includes:
determining an estimated height of an object in the current video frame using the estimated extrinsic parameters;
determining a detected height of the object using coordinates of the object within the current video frame; and
comparing the estimated height and the detected height using the cost function.

19. The apparatus of claim 18 , wherein the processor is configured to determine the estimated height by:
determining, using the homographic matrix, a three-dimensional point for two-dimensional coordinates of a bottom the object, wherein the two-dimensional coordinates are within the current video frame; and
determining two-dimensional coordinates of a top of the object using the camera model and an estimated real-world height of the object.

20. The apparatus of claim 18 , wherein the processor is further configured to:
determine a plurality cost values for a plurality of extrinsic parameters, the plurality of cost values including the cost value; and
identify from the plurality of cost values a set of extrinsic parameter

Assignees

• Qualcomm Inc

Inventors

• Wang Lei
• Gao Dashan
• Ma Lei
• Chiu Chinchuan

Classifications

• G06K9/00281

  Physics · mapped topic

• G06K9/00248 Primary

  Physics · mapped topic

• G06T2207/30196

  Human being; Person · CPC title

• G06T2207/30232

  Surveillance · CPC title

• G06T7/246

  using feature-based methods, e.g. the tracking of corners or segments · CPC title