Background model for user recognition

Patent metadata

Field	Value
Publication number	US-9639166-B2
Application number	US-201514645220-A
Country	US
Kind code	B2
Filing date	Mar 11, 2015
Priority date	Mar 11, 2015
Publication date	May 2, 2017
Grant date	May 2, 2017

Abstract

A computer system comprises a hardware interface, a computer-memory device, an update engine, and a posture-recognition engine. The hardware interface is configured to receive depth video of an environment from a camera. The computer-memory device stores a background model of the environment preservable over a reboot of the computer system, the background model including a plurality of trusted coordinates derived from the depth video. The update engine is configured to update the background model, including moving a trusted coordinate to greater depth if an observed pixel is behind the trusted coordinate over a first duration, but retaining the trusted coordinate if the observed pixel depth is in front of the trusted coordinate over the first duration. The posture-recognition engine is configured to recognize posture of a user in front of a background portion of the video, which is bounded by the trusted coordinates of the background model.

First claim

The invention claimed is:

1. A computer system comprising:
a hardware interface configured to receive depth video of an environment from a camera;
a computer-memory device storing a background model of the environment preservable over a reboot of the computer system, the background model including a plurality of trusted coordinates derived from the depth video;
an update engine configured to update the background model, including moving a trusted coordinate to greater depth if an observed pixel depth is behind the trusted coordinate over a first duration, but retaining the trusted coordinate if the observed pixel depth is in front of the trusted coordinate over the first duration, and moving the trusted coordinate to lesser depth if the observed pixel depth is in front of the trusted coordinate over a second, longer duration; and
a posture-recognition engine configured to ignore a background portion of the depth video but to recognize posture of a user in front of the background portion, the background portion bounded by the plurality of trusted coordinates of the background model.

2. The computer system of claim 1 , wherein the depth video includes a time-resolved series of depth images and a time-resolved series of infrared brightness images.

3. The computer system of claim 1 , wherein the camera is a time-of-flight depth camera.

4. The computer system of claim 1 , wherein the trusted coordinates are encoded in a trusted matrix of pixels spanning a field of view of the camera, the trusted matrix representing average depth and infrared brightness for each pixel.

5. The computer system of claim 4 , wherein the background model includes an accumulator matrix of pixels spanning the field of view of the camera, the accumulator matrix representing average depth, infrared brightness, and standard deviations of the average depth and infrared brightness for each pixel.

6. The computer system of claim 1 , wherein the second duration is reduced if the observed pixel and the trusted coordinate have similar infrared brightness, and wherein the second duration is prolonged if the observed pixel depth and the trusted coordinate have different infrared brightness.

7. The computer system of claim 1 , further comprising a background-removal engine configured to exclude the background portion from the depth video furnished to the posture-recognition engine.

8. The computer system of claim 7 , wherein the background-removal engine is configured to mask the depth video at and behind the background portion.

9. The computer system of claim 1 , wherein the posture-recognition engine is further configured to recognize a posture of the user and to furnish the posture as input to an operating system or application of the computer system.

10. In a computer system operatively coupled to a camera system via a hardware interface, a method to provide user input to the computer system, the method comprising:
receiving depth video of an environment from a camera via the hardware interface;
storing a background model of the environment preservable over a reboot of the computer system, the background model including a plurality of trusted coordinates extracted from the depth video;
updating the background model, including moving a trusted coordinate to greater depth if an observed pixel depth is behind the trusted coordinate over a first duration, but retaining the trusted coordinate if the observed pixel depth is in front of the trusted coordinate over the first duration, and moving the trusted coordinate to lesser depth if the observed pixel depth is in front of the trusted coordinate over a second, longer duration; and
ignoring a background portion of the depth video but recognizing posture of a user in front of the background portion, the background portion bounded by the plurality of trusted coordinates of the background model.

11. The method of claim 10 , wherein the background model is updated for each of a series of consecutive frames of the depth video and for each pixel of each frame.

12. The method of claim 10 , wherein ignoring the background portion includes masking the observed pixel depth if the observed pixel depth is within the threshold number of standard deviations from the trusted coordinate.

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Meekhof Casey
• Craig Robert M
• Peeper Craig
• Cook Patrick O
• Dalal Ketan
• Tankovich Vladimir
• Rakovchuk Anton

Classifications

• G06V10/34

  Smoothing or thinning of the pattern; Morphological operations; Skeletonisation · CPC title

• G06V40/103

  Static body considered as a whole, e.g. static pedestrian or occupant recognition · CPC title

• G06F3/005

  Input arrangements through a video camera · CPC title

• G09G5/18

  Timing circuits for raster scan displays (specially adapted for television H04N {; synchronisation between the display unit and other display units, videodisc player G09G5/12}) · CPC title

• G06F3/017 Primary

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title