Motion stabilization and detection of articulated objects

Patent metadata

Field	Value
Publication number	US-8977060-B2
Application number	US-201313895964-A
Country	US
Kind code	B2
Filing date	May 16, 2013
Priority date	May 16, 2013
Publication date	Mar 10, 2015
Grant date	Mar 10, 2015

Abstract

The techniques and systems described herein are directed to isolating part-centric motion in a visual scene and stabilizing (e.g., removing) motion in the visual scene that is associated with camera-centric motion and/or object-centric motion. By removing the motion that is associated with the camera-centric motion and/or the object-centric motion, the techniques are able to focus motion feature extraction mechanisms (e.g., temporal differencing) on the isolated part-centric motion. The extracted motion features may then be used to recognize and/or detect the particular type of object and/or estimate a pose or position of a particular type of object.

First claim

What is claimed is:

1. A method comprising:
receiving a sequence of images associated with a visual scene that contains a plurality of objects;
computing, by one or more processors and based at least in part on a predetermined level of coarseness, a coarse optical flow field for the sequence of images, the coarse optical flow field representing movement of groups of pixels within the visual scene;
applying a window to one or more regions of one or more images that contain the coarse optical flow field;
isolating motion in the one or more images that satisfies a threshold amount of movement within the window;
extracting motion features associated with the motion; and
determining that the motion features characterize a particular type of articulated object.

2. The method as recited in claim 1 , wherein the motion is part-centric motion that characterizes typical movement of one or more parts of the particular type of articulated object and that is useful in distinguishing the particular type of articulated object from other types of objects contained in the visual scene.

3. The method as recited in claim 1 , further comprising removing non-useful motion from the one or more images that fails to satisfy the threshold amount of movement within the window, the non-useful motion being associated with camera-centric motion or object-centric motion.

4. The method as recited in claim 1 , wherein the motion is periodic motion and the sequence of images spans a minimum amount of time in which at least one period occurs.

5. The method as recited in claim 1 , wherein the particular type of articulated object is a human and the motion is associated with movement of arms, legs, or a combination of both.

6. The method as recited in claim 1 , wherein the predetermined level of coarseness is set by a motion detection device configured to detect the particular type of articulated object and is based on the particular type of articulated object.

7. The method as recited in claim 1 , wherein a size of the window is based on the particular type of articulated object so that the window completely covers regions of the images that include the motion reflecting movement of one or more parts of the particular type of articulated object.

8. The method as recited in claim 1 , wherein the sequence of images are received by an image capture unit that is moving.

9. One or more computer-readable storage media having computer-executable instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:
receiving a sequence of images associated with a visual scene that contains a plurality of objects;
computing, based at least in part on a predetermined level of coarseness, a coarse optical flow field for the sequence of images, the coarse optical flow field representing motion for groups of pixels within the visual scene;
determining whether the motion for the groups of pixels within a detection window satisfies a threshold amount of movement; and
isolating motion that satisfies the threshold amount of movement.

10. The one or more computer-readable media as recited in claim 9 , wherein the operations further comprise:
extracting motion features associated with the motion; and
determining that the motion features characterize an articulated object.

11. The one or more computer-readable storage media as recited in claim 10 , wherein the motion features extracted are temporal difference features.

12. The one or more computer-readable storage media as recited in claim 9 , wherein the motion is part-centric motion that characterizes typical movement of one or more parts of the articulated object and that is useful in distinguishing the articulated object from other objects contained in the visual scene.

13. The one or more computer-readable storage media as recited in claim 9 , wherein the operations further comprise removing non-useful motion from the sequence of images that fails to satisfy the threshold amount of movement, the non-useful motion being associated with camera-centric motion or object-centric motion.

14. The one or more computer-readable storage media as recited in claim 9 , wherein the motion is frequent motion and the sequence of images spans a minimum amount of time in which at least one instance of the frequent motion occurs.

15. The one or more computer-readable storage media as recited in claim 9 , wherein a size of the detection window is based on a type of the articulated object so that the detection window completely covers at least part of the motion that reflects movement of one or more parts of the articulated object.

16. One or more devices comprising:
one or more processors;
one or more computer-readable storage media storing computer-executable instructions;
a motion stabilizing module, maintained in the one or more computer-readable storage media and executed by the one or more processors, to isolate part-centric motion in a sequence of images; and
an articulated object detection module, maintained in the one or more computer-readable storage media and executed by the one or more processors, to detect that the sequence of images contains an articulated object characterized by the part-centric motion.

17. The one or more devices as recited in claim 16 , further comprising a motion feature extracting module, maintained in the one or more computer-readable storage media and executed by the one or more processors, to extract motion features from the part-centric motion, wherein the motion features are used by the articulated object detection module to detect that the sequence of images contains the articulated object.

18. The one or more devices as recited in claim 16 , wherein the motion stabilizing module further removes camera-centric motion and object-centric motion in the sequence of images.

19. The one or more devices as recited in claim 16 , wherein the part-centric motion characterizes typical movement of one or more parts of the articulated object and is useful in distinguishing the articulated object from other objects.

20. The one or more devices as recited in claim 19 , wherein the articulated object is a human and the one or more parts include one or more arms, one or more legs, or a combination of both.

Assignees

• Microsoft Corp
• Microsoft Technology Licensing Llc

Inventors

• Dollar Piotr
• Zitnick Iii Charles Lawrence
• Park Dennis I

Classifications

• G06T7/00 Primary

  Image analysis · CPC title

• G06V20/40 Primary

  in video content (extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44) · CPC title

• G06V40/23 Primary

  Recognition of whole body movements, e.g. for sport training · CPC title

• G06T7/246

  using feature-based methods, e.g. the tracking of corners or segments · CPC title