Opportunistic structured light

Patent metadata

Field	Value
Publication number	US-9696145-B2
Application number	US-201414185562-A
Country	US
Kind code	B2
Filing date	Feb 20, 2014
Priority date	May 13, 2013
Publication date	Jul 4, 2017
Grant date	Jul 4, 2017

Abstract

A method for using structured light in a handheld projection device is provided that includes projecting a structured light pattern in at least one portion of a frame being projected by the handheld projection device, wherein the at least one portion of the frame is a subset of the frame, capturing an image of the projected frame, computing scene depth information based on the structured light pattern in the captured image, and using the scene depth information in processing of a subsequent frame of the video stream.

First claim

What is claimed is:

1. A method for using structured light in a handheld projection device, the method comprising:
replacing only a portion of a frame in a video stream with a structured light pattern, wherein the portion of the frame is less than all of the frame;
projecting the frame including the structured light pattern for the portion of the frame and video content for other pixels of the frame with the handheld projection device;
capturing an image of the projected frame;
computing scene depth information based on the structured light pattern in the captured image; and
using the scene depth information in processing of a subsequent frame of the video stream.

2. The method of claim 1 , wherein using the scene depth information includes using the scene depth information to perform keystone correction on a next projected frame.

3. The method of claim 1 , wherein the portion of the frame is a boundary of the frame.

4. The method of claim 3 , wherein the portion of the frame includes boundary pixels determined based on a location of keystone distortion in the frame.

5. The method of claim 1 , further comprising determining a location of the portion of the frame based on distortion in the frame caused by encoding the frame in an encoded video stream.

6. The method of claim 5 , wherein determining a location includes estimating distortion in one or more contiguous blocks of the frame based on quantization parameters of the blocks signaled in the encoded video stream.

7. The method of claim 5 , wherein determining a location includes estimating distortion in one or more contiguous blocks of the frame based on boundary strength of a deblocking filter.

8. The method of claim 5 , wherein determining a location includes estimating distortion in one or more contiguous blocks of the frame based on differences between motion vectors of neighboring blocks.

9. The method of claim 5 , wherein determining a location includes decoding indicators of regions of high distortion signaled in the encoded video stream.

10. The method of claim 1 , wherein determining a location includes determining a location of the portion of the frame based on luminance.

11. A handheld projection device configured to use structured light, the device comprising:
a projector;
a camera;
one or more processors coupled to the projector and camera, the one or more processors configured to:
replace only a portion of a frame in the video stream with a structured light pattern, wherein the portion of the frame is less than all of the frame;
cause the projector to project the frame including the structured light pattern for the portion of the frame and video content for other pixels of the frame;
cause the camera to capture an image of the projected frame;
compute scene depth information based on the structured light pattern in the captured image; and
use the scene depth information in processing of a subsequent frame of the video stream.

12. The handheld projection device of claim 11 , wherein the one or more processors are further configured to use the scene depth information to perform keystone correction on a next projected frame.

13. The handheld projection device of claim 11 , wherein the portion of the frame is a boundary of the frame.

14. The handheld projection device of claim 13 , wherein the portion of the frame includes boundary pixels determined based on a location of keystone distortion in the frame.

15. The handheld projection device of claim 11 , wherein the one or more processors are further configured to determine a location of the portion of the frame based on distortion in the frame caused by encoding the frame in an encoded video stream.

16. The handheld projection device of claim 15 , wherein the one or more processors are further configured to estimate distortion in one or more contiguous blocks of the frame based on quantization parameters of the blocks signaled in the encoded video stream.

17. The handheld projection device of claim 15 , wherein the one or more processors are further configured to estimate distortion in one or more contiguous blocks of the frame based on boundary strength of a deblocking filter.

18. The handheld projection device of claim 15 , wherein the one or more processors are further configured to estimate distortion in one or more contiguous blocks of the frame based on differences between motion vectors of neighboring blocks.

19. The handheld projection device of claim 15 , wherein the one or more processors are further configured to decode indicators of regions of high distortion signaled in the encoded video stream.

20. The handheld projection device of claim 11 , wherein the one or more processors are further configured to determine a location of the portion of the frame based on luminance.

Assignees

• Texas Instruments Inc

Inventors

• Sze Vivienne
• Dedeoglu Goksel
• Sharma Vinay

Classifications

• G01B11/25 Primary

  by projecting a pattern, e.g. {one or more lines,} moiré fringes on the object (G01B11/255 takes precedence {; image analysis for depth or shape recovery G06T7/50}) · CPC title