Systems and methods for removing a background of an image

Patent metadata

Field	Value
Publication number	US-10147194-B2
Application number	US-201715589827-A
Country	US
Kind code	B2
Filing date	May 8, 2017
Priority date	Oct 7, 2009
Publication date	Dec 4, 2018
Grant date	Dec 4, 2018

Abstract

An image such as a depth image of a scene may be received, observed, or captured by a device. A grid of voxels may then be generated based on the depth image such that the depth image may be downsampled. A background included in the grid of voxels may then be discarded to isolate one or more voxels associated with a foreground object such as a human target and the isolated voxels associated with the foreground object may be processed.

First claim

What is claimed:

1. A computer-readable storage device having stored thereon computer executable instructions for processing depth information of a scene, the computer executable instructions comprising instructions for:
receiving a depth image of the scene, wherein the depth image comprises one or more objects;
comparing a value associated with voxels in one or more objects in the depth image with a corresponding value of a reference voxel;
identifying voxels as moving when a difference between the value and the corresponding value exceeds a motion threshold;
identifying other voxels as non-moving;
calculating a foreground score based on a percentage of moving voxels associated with one object; and
identifying the one object as moving when the foreground score exceeds a score threshold.

2. The computer-readable storage device of claim 1 , the computer executable instructions further comprising instructions for determining whether to merge an object in the one or more objects with a nearby object in the one or more objects.

3. The computer-readable storage device of claim 2 , wherein determining whether to merge the object in the one or more objects with the nearby object in the one or more objects comprises: comparing X-values, Y-values, or depth values associated with voxels in the object with X-values, Y-values, or depth values associated with voxels of the nearby object; and merging the object with the nearby object if, based on the comparison, the X-values, Y-values, or the depth values associated with the voxels in the object overlap the X-values, Y-values, or depth values associated with the voxels of the nearby object.

4. The computer-readable storage device of claim 1 , the computer executable instructions further comprising instructions for:
isolating the one object associated with the voxels identified to be moving; and
processing the one object isolated.

5. The computer-readable storage device of claim 1 , the computer executable instructions further comprising instructions for determining that at least one object of the one or more objects is non-moving.

6. The computer-readable storage device of claim 5 , the computer executable instructions further comprising instructions for discarding the at least one non-moving object of the one or more objects.

7. A system for processing depth information of a scene, the system comprising:
a capture device comprising an image camera component configured to capture a depth image of the scene; and
a computing device communicatively coupled to the capture device, wherein the computing device comprises a processor and memory storing thereon computer-readable instructions that, when executed by the processor, cause the computing device to perform operations comprising:
receiving a depth image of the scene, wherein the depth image comprises one or more objects;
comparing a value associated with voxels in one or more objects in the depth image with a corresponding value of a reference voxel;
identifying voxels as moving when a difference between the value and the corresponding value exceeds a motion threshold;
identifying other voxels as non-moving;
calculating a foreground score based on a percentage of moving voxels associated with one object; and
identifying the one object as moving when the foreground score exceeds a score threshold.

8. The system of claim 7 , further comprising computer-readable instructions that, when executed by the processor, cause the computing device to perform operations comprising determining whether to merge an object in the one or more objects with a nearby object in the one or more objects.

9. The system of claim 8 , further comprising computer-readable instructions that, when executed by the processor, cause the computing device to perform operations comprising:
determining whether merge an object in the one or more objects with a nearby object in the one or more objects: comparing X-values or depth values associated with voxels in one of the object with X-values or depth values associated with voxels of a nearby object; and
merging the object with the nearby object if, based on the comparison, the X-values or the depth values associated with the voxels in the object overlap the X-values or depth values associated with the voxels of the nearby object.

10. The system of claim 7 , further comprising computer-readable instructions that, when executed by the processor, cause the computing device to perform operations comprising:
isolating the one object associated with the voxels identified to be moving; and
processing the one object isolated.

11. The system of claim 7 , further comprising computer-readable instructions that, when executed by the processor, cause the computing device to perform operations comprising discarding at least one non-moving object of the one or more objects.

12. A method for processing depth information of a scene, the method comprising:
receiving a depth image of the scene, wherein the depth image comprises one or more objects;
comparing a value associated with voxels in one or more objects in the depth image with a corresponding value of a reference voxel;
identifying voxels as moving when a difference between the value and the corresponding value exceeds a motion threshold;
identifying other voxels as non-moving;
calculating a foreground score based on a percentage of moving voxels associated with at least one object; and
identifying the at least one object as moving when the foreground score exceeds a score threshold.

13. The method of claim 12 , further comprising determining whether to merge an object in the one or more objects with a nearby object in the one or more objects.

14. The method of claim 13 , wherein determining whether to merge the object in the one or more objects with the nearby object in the one or more objects comprises: comparing X-values, Y-values, or depth values associated with voxels in the object with X-values, Y-values, or depth values associated with voxels of the nearby object; and merging the object with the nearby object if, based on the comparison, the X-values, Y-values, or the depth values associated with the voxels in the object overlap the X-values, Y-values, or depth values associated with the voxels of the nearby object.

15. The method of claim 12 , further comprising:
isolating the at least one object associated with the voxels identified to be moving; and
processing the at least one object isolated.

16. The method of claim 12 , further comprising determining that at least one object of the one or more objects is non-moving.

17. The method of claim 16 , further comprising discarding at least one non-moving object of the one or more objects.

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Peeper Craig
• Lee Johnny
• Leyvand Tommer
• Stachniak Szymon

Classifications

• G06T7/215

  Motion-based segmentation · CPC title

• G06T15/00

  Three-dimensional [3D] image rendering · CPC title

• G06T7/11

  Region-based segmentation · CPC title

• G06T7/194 Primary

  involving foreground-background segmentation · CPC title

• G06T7/254

  involving subtraction of images · CPC title