Interaction detection using structured light images

Patent metadata

Field	Value
Publication number	US-9524059-B2
Application number	US-201313834807-A
Country	US
Kind code	B2
Filing date	Mar 15, 2013
Priority date	Mar 15, 2013
Publication date	Dec 20, 2016
Grant date	Dec 20, 2016

Abstract

An apparatus and method are provided to determine the occurrence and location of an interaction with an interface, particularly with an image of a user interface that may be projected or otherwise produced on a surface. The apparatus uses one or more single-element sensors (such as a photodiode) to sense and capture light readings of a scene, the readings corresponding to a plurality of structured light images injected within the presentation of the interface. The readings are compared to a baseline set of readings to determine the occurrence and location of an interaction event by an obstacle (i.e., a finger or a stylus) such as a touch event or a movement of the obstacle.

First claim

What is claimed is:

1. A method comprising:
sensing, using at least one photodiode, a plurality of time-varying structured light images individually projected on a surface in individual time slots to generate a plurality of sensor readings, in which individual ones of the sensor readings correspond to individual ones of the plurality of time-varying structured light images, each of the plurality of time-varying structured light images including at least one high-contrast boundary internal to an image area of the respective time-varying structured light image, the plurality of time-varying structured light images differing one from another in location of the at least one high-contrast boundary;
comparing, using at least one processing device, at least a portion of the plurality of sensor readings with at least a portion of a plurality of baseline sensor readings, the at least a portion of the plurality of sensor readings and the at least a portion of the plurality of baseline sensor readings corresponding to a same set of the plurality of time-varying structured light images that are projected on the surface; and
determining, using the at least one processing device, an occurrence of and an approximate location of an object being inserted into a projection path of the plurality of time-varying structured light images projected on to the surface based on the comparing and based on a location of a high-contrast boundary in at least one of the time-varying structured light images.

2. The method of claim 1 in which the method further includes:
sensing, using the at least one photodiode, the plurality of time-varying structured light images projected on the surface in individual time slots in the absence of the object being inserted into the projection path to generate the plurality of baseline sensor readings, individual ones of the plurality of baseline sensor readings corresponding to individual ones of the plurality of time-varying structured light images; and
storing the plurality of baseline sensor readings in at least one memory device.

3. The method of claim 1 in which the at least one high-contrast boundary includes at least one straight line defining at least two regions including a first region including a uniform bright area and a second region including a uniform dark area.

4. The method of claim 1 in which the at least one high-contrast boundary includes at least one of a vertical straight line or a horizontal straight line spanning at least a majority of the image area.

5. The method of claim 1 in which the method further includes:
sensing at a second time, using the at least one photodiode, a second plurality of time-varying structured light images to generate a second plurality of sensor readings in response to determining the approximate location of the object, individual ones of the second plurality of sensor readings corresponding to individual ones of the second plurality of time-varying structured light images, in which the second plurality of time-varying structured light images includes at least a plurality of individual structured light images each including at least one high-contrast boundary proximate to the approximate location of the object;
comparing, using the at least one processing device, the second plurality of sensor readings with a second plurality of baseline sensor readings, individual ones of the second plurality of baseline sensor readings corresponding to individual ones of the second plurality of sensor readings; and
determining, using the at least one processing device, additional information regarding the object based on the second comparing.

6. The method of claim 1 in which determining the occurrence of and approximate location of the object being inserted into the projection path further includes analyzing the comparison between at least one individual sensor reading of the at least a portion of the plurality of sensor readings and at least one individual baseline sensor reading of the at least a portion of the plurality of the plurality of baseline sensor readings, in which the at least one individual sensor reading and the at least one individual baseline sensor reading each correspond to the same at least one individual time-varying structured light image, the analyzing including determining that a difference between the at least one individual sensor reading and the at least one individual baseline sensor reading exceeds a threshold.

7. The method of claim 1 in which determining the occurrence of and approximate location of the object being inserted into the projection path includes:
determining that a difference between at least one individual sensor reading and at least one individual baseline sensor reading exceeds a threshold, the at least one individual sensor reading and the at least one individual baseline sensor reading corresponding to the same at least one individual structured light image; and
determining the location of the high-contrast boundary in the at least one individual structured light image; and
determining the approximate location of the object based on the location of the high-contrast boundary.

8. The method of claim 1 in which the plurality of time-varying structured light images are individually projected on the surface in individual time slots during a projection of a non-structured light image, the plurality of time-varying structured light images being projected by a same projector utilized to project the non-structured light image.

9. An apparatus comprising:
at least one processing device configured to:
effect projection of a plurality of time-varying structured light images on a surface in individual time slots inserted during projection of a non-structured light image via a same projector used to project the non-structured light image, each of the plurality of time-varying structured light images including at least one high-contrast boundary internal to an image area of the respective time-varying structured light image, the plurality of time-varying structured light images differing one from another in location of the at least one high-contrast boundary;
compare a plurality of sensor readings from at least one photodiode with a corresponding plurality of baseline sensor readings, individual ones of the plurality of sensor readings and the plurality of baseline sensor readings each corresponding to individual ones of the plurality of time-varying structured light images that are projected on the surface; and
determine an occurrence of an object being inserted into a projection path at an approximate location based on the comparing and based on a location of a high-contrast boundary in at least one of the time-varying structured light images.

10. The apparatus of claim 9 in which the at least one processing device is further configured to:
effect projection of the plurality of time-varying structured light images on the surface in the absence of the object being inserted into the projection path;
generate the plurality of baseline sensor readings based on readings taken via the at least one photodiode of the projection of the plurality of time-varying structured light images in the absence of the object being inserted into the projection path, individual ones of the plurality of baseline sensor readings corresponding to individual ones of the plurality of time-varying structured light images; and
effect storage of the plurality of baseline sensor readings into at least one memory device.

11. The apparatus of claim 9 in which the apparatus further includes the projector operatively connected to the at least one processing device and the at least one photodiode operatively connected to the at least one p

Assignees

• Texas Instruments Inc

Inventors

• Capps Marshall Charles
• Dedeoglu Goksel

Classifications

• G06F3/0418 Primary

  for error correction or compensation, e.g. based on parallax, calibration or alignment · CPC title

• G06F3/03542

  Light pens for emitting or receiving light · CPC title

• G06F3/0386

  for light pen · CPC title

• G06F3/042 Primary

  by opto-electronic means · CPC title

• G06F3/0426

  tracking fingers with respect to a virtual keyboard projected or printed on the surface (virtual keyboards on touch screens G06F3/04886) · CPC title