Projection-based user interface

Patent metadata

Field	Value
Publication number	US-2018088674-A1
Application number	US-201615279954-A
Country	US
Kind code	A1
Filing date	Sep 29, 2016
Priority date	Sep 29, 2016
Publication date	Mar 29, 2018
Grant date	—

Abstract

Techniques for providing a virtual touch screen are described. An example of a computing device with a virtual touch screen includes a projector to project a user interface image onto a touch surface, and a depth camera to generate a depth image representing objects in a vicinity of the user interface image, and a touch mask generator. The computing device also includes a touch detection module to analyze the touch mask to detect touch events. The touch detection module is configured to identify a finger in the touch mask, identify a centroid region of the finger and compute a distance of the centroid region from a touch surface, and compare the distance to a threshold distance to identify a touch event.

First claim

What is claimed is:

1 . A computing device with a virtual touch screen, comprising:
a projector to project a user interface image onto a touch surface; a depth camera to generate a depth image representing objects in a vicinity of the user interface image; a touch mask generator to receive the depth image from the depth camera and generate a touch mask; and a touch detection module to analyze the touch mask to detect touch events, wherein the touch detection module is configured to: identify a finger in the touch mask; identify a centroid region of the finger and compute a distance of the centroid region from a touch surface; and compare the distance to a threshold distance to identify a touch event.

2 . The computing device of claim 1 , wherein the touch detection module is configured to determine an angle of the finger from the touch mask and adjust the threshold distance based on the angle.

3 . The computing device of claim 1 , wherein the touch detection module is configured to track movement of the centroid region of the finger between successive frames to stabilize a position of the centroid region.

4 . The computing device of claim 1 , wherein the touch detection module is configured to identify a hand in the touch mask and analyze a portion of the touch mask corresponding to the hand to identify the finger.

5 . The computing device of claim 4 , wherein to identify the hand, the touch detection module is configured to:
generate a binary mask from the touch mask; identify a contour region corresponding to an object in the binary touch mask; compute a histogram of depth values within the contour region; and compare the histogram of depth values to a predetermined depth model for a typical hand object.

6 . The computing device of claim 5 , wherein to identify the finger, the touch detection module is configured to:
identify the contour region as a corresponding to a hand; analyze the contour region to identify a potential fingertip based on contour angles of the contour region; determine a geometric feature of the potential fingertip from the touch mask; and compare the geometric feature to a model of a typical fingertip.

7 . The computing device of claim 1 , wherein to generate the touch mask, the touch mask generator is configured to implement a two-pass process for eliminating global noise variations and local noise variations from depth value data of the depth image received from the depth camera, wherein global noise variations are eliminated in a first pass of the two-pass process to generate a residual depth map, and local noise variations are eliminated in a second pass of the two-pass process.

8 . The computing device of claim 7 , wherein during the first pass, the touch mask generator is to:
remove outliers from the depth value data of the depth image to generate outlier-removed depth data; perform a two-dimensional linear regression on the outlier-removed depth data in a central area of the depth image to generate a principle plane of the touch surface; and subtract the principle plane from the depth value data to generate the residual depth map.

9 . The computing device of claim 7 , wherein during the second pass, the touch mask generator is to:
continuously update a Gaussian distribution at each pixel of the residual depth map across successive frames of the depth value data; compute an average pixel depth for each pixel based on the Gaussian distribution; and for each pixel, subtract the average pixel depth from a corresponding pixel of the residual depth map to generate normalized residual depth values.

10 . The computing device of claim 9 , wherein during the second pass, the touch mask generator is to collect all normalized residual depth values with a standard deviation of two sigma away from the touch surface for inclusion within the touch mask.

11 . A method of operating a virtual touch screen device, comprising:
projecting a user interface image onto a touch surface; generating a depth image representing objects in a vicinity of the user interface image; generating a touch mask from the depth image; identifying a finger in the touch mask; identifying a centroid region of the finger and computing a distance of the centroid region from a touch surface; and comparing the distance to a threshold distance to identify a touch event.

12 . The method of claim 11 , comprising determining an angle of the finger from the touch mask and adjusting the threshold distance based on the angle.

13 . The method of claim 11 , comprising tracking movement of the centroid region of the finger between successive frames to stabilize a position of the centroid region.

14 . The method of claim 11 , comprising identifying a hand in the touch mask and analyzing a portion of the touch mask corresponding to the hand to identify the finger.

15 . The method of claim 14 , wherein identifying the hand comprises:
generating a binary mask from the touch mask; identifying a contour region corresponding to an object in the binary touch mask; computing a histogram of depth values within the contour region; and comparing the histogram of depth values to a predetermined depth model for a typical hand object.

16 . The method of claim 15 , wherein identifying the finger comprises:
identifying the contour region as a corresponding to a hand; analyzing the contour region to identify a potential fingertip based on contour angles of the contour region; determining a geometric feature of the potential fingertip from the touch mask; and comparing the geometric feature to a model of a typical fingertip.

17 . The method of claim 11 , wherein generating the touch mask comprises eliminating global noise variations to generate a residual depth map, and eliminating local noise variations from the residual depth map to generate the touch mask.

18 . The method of claim 17 , wherein eliminating global noise variations comprises:
removing outliers from the depth image to generate outlier-removed depth data; performing a two-dimensional linear regression on the outlier-removed depth data in a central area of the depth image to generate a principle plane of the touch surface; and subtracting the principle plane from the depth image to generate the residual depth map.

19 . The method of claim 17 , wherein eliminating local noise variations comprises:
continuously updating a Gaussian distribution at each pixel of the residual depth map across successive frames; computing an average pixel depth for each pixel based on the Gaussian distribution; and for each pixel, subtracting he average pixel depth from a corresponding pixel of the residual depth map to generate normalized residual depth values.

20 . The method of claim 19 , wherein eliminating local noise variations comprises collecting all normalized residual depth values with a standard deviation of two sigma away from the touch surface for inclusion within the touch mask.

21 . A tangible, non-transitory, computer-readable medium comprising instructions that, when executed by a processor, direct the processor to operate a virtual touch screen device, the instructions to direct the processor to:
project a user interface image onto a touch surface; generate a depth image that represents objects in a vicinity of the user interface image; generate a touch mask from the depth image; identify a finger in the touch mask; identify a centroid region of the finger and compute a d

Assignees

• Intel Corp

Inventors

• Chen Fan
• Dongre Kedar A

Classifications

• G06F3/0304

  Detection arrangements using opto-electronic means (constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042) · CPC title

• G06F3/017 Primary

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title

• G06F3/0425

  using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected (tracking a projected light spot to determine a position on a display surface G06F3/0386) · CPC title

• G06F3/005

  Input arrangements through a video camera · CPC title