Generation of depth images based upon light falloff

Patent metadata

Field	Value
Publication number	US-9513768-B2
Application number	US-201213490457-A
Country	US
Kind code	B2
Filing date	Jun 7, 2012
Priority date	Mar 5, 2012
Publication date	Dec 6, 2016
Grant date	Dec 6, 2016

Abstract

Technologies pertaining to computing depth images of a scene that includes a mobile object based upon the principle of light falloff are described herein. An infrared image of a scene that includes a mobile object is captured, wherein the infrared image has a plurality of pixels having a respective plurality of intensity values. A depth image for the scene is computed based at least in part upon square roots of respective intensity values in the infrared image.

First claim

What is claimed is:

1. A system comprising:
at least one processor; and
memory storing instructions that, when executed by the at least one processor, cause the at least one processor to perform acts comprising:
receiving an infrared image of a scene irradiated by an infrared emitter, the infrared image captured by an infrared camera, the infrared image comprises pixels, each pixel in the pixels having an intensity value assigned thereto;
generating a depth image of the scene based upon the infrared image of the scene, wherein the depth image comprises depth pixels, each depth pixel having a depth value assigned thereto that is indicative of a distance between the infrared emitter and a respective object in the scene, wherein generating the depth image of the scene comprises applying the principle of light falloff with respect to intensity values of the pixels of the infrared image to compute the depth values of the depth image, and further wherein when the principle of light falloff is applied to an intensity value of a pixel in the infrared image, a depth value of a corresponding depth pixel in the depth image is computed by the at least one processor based upon a square root of the intensity value of the pixel in the infrared image; and

rendering graphical data on a display based upon the depth image.

2. The system of claim 1 comprised by a mobile computing device.

3. The system of claim 1 , further comprising a housing, the housing comprising the at least one processor, the memory, the infrared camera, and the infrared emitter.

4. The system of claim 1 , wherein when the principle of light falloff is applied to the pixel in the infrared image, the at least one processor computes the depth value of the corresponding depth pixel in the depth image further based upon a constant.

5. The system of claim 1 , wherein generating the depth image further comprises:
generating the depth image based upon a first image captured by a first RGB camera; and
a second image captured by a second RGB camera, wherein the first RGB camera and the second RGB camera are stereoscopically arranged.

6. The system of claim 5 , the acts further comprising:
calibrating depth images that have been generated based upon binocular vision technologies; and
calibrating depth images that have been generated based upon images captured by the infrared camera to learn calibration parameters, the calibration parameters are indicative of a mapping between:
pixels of depth images that have been generated based upon binocular vision technologies; and
pixels of depth images that have been generated based upon images captured by the infrared camera.

7. The system of claim 6 , wherein generating the depth image further comprises:
generating an initial depth image of the scene based upon the first image captured by the first RGB camera and the second image captured by the second RGB camera; and
generating the depth image based upon the initial depth image.

8. The system of claim 7 , wherein generating the depth image further comprises:
segmenting the initial depth image into a plurality of segments, each segment comprising pixels in the initial depth image with depth values falling into a respective range;
mapping the plurality of segments to respective portions of the infrared image based upon the calibration parameters; and
generating the depth image based upon mapping of the plurality of segments to the respective portions of the infrared image.

9. The system of claim 8 , the initial depth image has a first resolution, the depth image has a second resolution, and the second resolution is higher than the first resolution.

10. The system of claim 1 , wherein the scene, when captured in the infrared image, comprises a human hand, the acts further comprising:
recognizing a gesture being made by the human hand relative to a graphical object being displayed on the display, wherein the gesture is recognized based upon the depth image; and
rendering the graphical data on the display screen of the computing device based upon the gesture.

11. The system of claim 10 , the graphical object is a lock screen, the gesture is an upward pivoting of the human hand about a wrist, and the graphical data is a visual transition to a home screen, the home screen comprising a plurality of selectable tiles for a respective plurality of executable applications.

12. The system of claim 10 , the graphical object is a graphical user interface that comprises a first plurality of selectable tiles for a first plurality of applications that are executable by a processor, the gesture is a leftwardly or rightwardly pivoting of the human hand about a wrist, and the graphical data is a visual transition to another graphical user interface that comprises a second plurality of selectable tiles for a respective second plurality of applications that are executable by the processor, wherein a direction of the visual transition is one of leftward or rightward and corresponds to a direction of the pivoting of the human hand.

13. The system of claim 10 , the graphical object is a graphical user interface that comprises a plurality of selectable tiles for a respective plurality of applications that are executable by a processor, the gesture comprises an extension of a pointer finger of the human hand and transitioning of the pointer finger toward the display, and the graphical data is a visual indicator that a particular tile from amongst the plurality of selectable tiles has been selected.

14. A method executed by at least one computer processor, the method comprising:
receiving, from an infrared camera, a sequence of infrared images of a scene that comprises a mobile object, the scene irradiated with infrared light by an infrared emitter, wherein each infrared image in the sequence of infrared images comprises a plurality of pixels, each pixel in each infrared image in the sequence of infrared images having an intensity value assigned thereto;
generating a sequence of depth images of the scene based upon the sequence of infrared images of the scene, wherein each depth image in the sequence of depth images comprises a plurality of depth pixels, each depth pixel in each depth image in the sequence of depth images having depth values assigned thereto, wherein a subset of the depth values are indicative of a distance between the mobile object and the infrared emitter, wherein generating the sequence of depth images based upon the sequence of infrared images comprises:
for an infrared image in the sequence of infrared images, determining, based upon application of the principle of light falloff to intensity values of pixels of the infrared image, depth values for corresponding depth pixels in a depth image, wherein when the at least one computer processor applies the principle of light falloff to an intensity value of a pixel in the infrared image, the at least one computer processor computes a depth value of a corresponding depth pixel in the depth image based upon a square root of the intensity value of the pixel in the infrared image; and

rendering graphical data on a display based upon the subset of depth values in the depth images that are indicative of the distance between the mobile object and the infrared image.

15. The method of claim 14 , wherein the mobile object is a human hand, the method further comprising:
recognizing a gesture being made by the human hand relative to a graphical object on the display based upon the subset of depth values; and
rendering the graphical data on the display based upon the gesture being made by the human hand.

16.

Assignees

• Zhao Chunshui
• Yan Jing
• Gu Jiawei
• Hsu Feng-Hsiung
• Song Shaopeng
• Wang Mingjie
• Li Jie
• Microsoft Technology Licensing Llc

Inventors

• Zhao Chunshui
• Yan Jing
• Gu Jiawei
• Hsu Feng-Hsiung
• Song Shaopeng
• Wang Mingjie
• Li Jie

Classifications

• G06T2207/10048

  Infrared image · CPC title

• H04N13/254

  in combination with electromagnetic radiation sources for illuminating objects · CPC title

• G06T2207/10021

  Stereoscopic video; Stereoscopic image sequence · CPC title

• G06T7/55

  from multiple images · CPC title

• G06F3/0485

  Scrolling or panning · CPC title