Fusing depth and pressure imaging to provide object identification for multi-touch surfaces

Patent metadata

Field	Value
Publication number	US-11809659-B2
Application number	US-202217668364-A
Country	US
Kind code	B2
Filing date	Feb 9, 2022
Priority date	Oct 12, 2010
Publication date	Nov 7, 2023
Grant date	Nov 7, 2023

Abstract

An apparatus for inputting information into a computer includes a 3 d sensor that senses 3 d information and produces a 3 d output. The apparatus includes a 2 d sensor that senses 2 d information and produces a 2 d output The apparatus includes a processing unit which receives the 2 d and 3 d output and produces a combined output that is a function of the 2 d and 3 d output. A method for inputting information into a computer. The method includes the steps of producing a 3 d output with a 3 d sensor that senses 3 d information. There is the step of producing a 2 d output with a 2 d sensor that senses 2 d information. There is the step of receiving the 2 d and 3 d output at a processing unit. There is the step of producing a combined output with the processing unit that is a function of the 2 d and 3 d output.

First claim

The invention claimed is:

1. An apparatus for inputting information into a computer comprising:
a 3D sensor that senses 3D information and produces a 3D output;
a 2D sensor that senses 2D information and produces a 2D output, including a contact point having a contact radius that is sensed by the 2d sensor; and
a processing unit which receives the 2D and 3D output and produces a combined output that is a function of the 2D and 3D output, the combined output includes the contact point in a set of detected touches until the contact point is not active, at which time the processing unit removes the contact point from the set of detected touches.

2. The apparatus of claim 1 wherein objects are identified and tracked in 3D and 2D by the 3D and 2D sensors.

3. The apparatus of claim 2 wherein fingers, hands, feet, people, pens or other objects are identified and tracked in 3D and 2D.

4. The apparatus of claim 3 including a memory and wherein the identity of each object is maintained over time.

5. The apparatus of claim 4 wherein the identity of objects from the 3D sensor are paired with objects from the 2D sensor by the processing unit.

6. The apparatus of claim 5 wherein the 2D sensor has a surface and the 2D sensor senses contact on the surface.

7. The apparatus of claim 6 wherein the 2D sensor senses imposed force on the surface.

8. The apparatus of claim 7 wherein the 2D sensor includes a pressure imaging sensor.

9. The apparatus of claim 8 wherein the 3D sensor includes a range imaging camera.

10. The apparatus of claim 8 wherein the 3D sensor includes an IR depth camera.

11. The apparatus of claim 8 wherein the 3D sensor includes an RGB camera.

12. A method for inputting information into a computer comprising:
producing a 3D output with a 3d sensor that senses 3d information;
producing a 2D output with a 2d sensor that senses 2d information, including a contact point having a contact radius that is sensed by the 2d sensor; and
producing a combined output with the processing unit that is a function of the 2D and 3D output, the combined output includes the contact point in a set of detected touches until the contact point is not active, at which time the processing unit removes the contact point from the set of detected touches.

13. The method of claim 12 including the step of identifying and tracking objects in 3D and 2D by the 3D and 2D sensors.

14. The method of claim 13 including the step of identifying and tracking fingers, hands, feet, people, pens or other objects in 3D and 2D.

15. The method of claim 14 including the step of maintaining in a memory the identity of each object over time.

16. The method of claim 15 including the step of pairing with the processing unit the identity of objects from the 3D sensor with objects from the 2D sensor.

17. The method of claim 16 including the step of the 2D sensor senses contact on its surface.

18. The method of claim 17 including the step of the 2D sensor senses imposed force on its surface.

19. The method of claim 18 wherein the 2D sensor includes a pressure imaging sensor.

20. An apparatus for inputting information into a computer comprising:
a 2D sensor that senses 2D information and produces a 2D output, including a contact point having a contact radius that is sensed by the 2d sensor, the 2D sensor having a surface and the 2D sensor senses imposed force on the surface; and
a processing unit which receives the 2D output and produces an output that is a function of the 2D output, the output includes the contact point in a set of detected touches until the contact point is not active, at which time the processing unit removes the contact point from the set of detected touches.

Assignees

• Perlin Kenneth
• Hendee Charles
• Grau Alex
• Seidman Gerald
• Univ New York
• Tatonic Tech Llc

Inventors

• Perlin Kenneth
• Hendee Charles
• Grau Alex
• Seidman Gerald

Classifications

• G06F3/04166 Primary

  Details of scanning methods, e.g. sampling time, grouping of sub areas or time sharing with display driving (Synchronisation with the driving of the display or the backlighting unit to avoid interferences generated internally G06F3/04184) · CPC title

• G06F3/005

  Input arrangements through a video camera · CPC title

• G06F3/011

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title

• G06F3/0304

  Detection arrangements using opto-electronic means (constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042) · CPC title

• G06F3/045

  using resistive elements, e.g. a single continuous surface or two parallel surfaces put in contact · CPC title