System and method for predicting a touch position of a pointer on a touch-enabled unit or determining a pointing direction in 3d space

Patent metadata

Field	Value
Publication number	US-2019004667-A1
Application number	US-201816003257-A
Country	US
Kind code	A1
Filing date	Jun 8, 2018
Priority date	Jul 3, 2017
Publication date	Jan 3, 2019
Grant date	—

Abstract

A system for predicting a touch position of a pointer on a touch-enabled unit includes a 3D imaging unit, a processing unit coupled to the 3D imaging unit, and a touch-enabled unit coupled to the processing unit. The 3D imaging unit is configured to monitor an interaction zone in front of the touch-enabled unit. The processing unit includes a prediction module that is configured to predict where a pointer that approaches the touch-enabled unit and is monitored by the 3D imaging unit will touch the touch-enabled unit, and a calibration module that is configured to generate at least one calibration parameter by comparing the predicted touch position to the actual touch position of the pointer detected by the touch-enabled unit and to transfer the at least one calibration parameter to the prediction module in order to calibrate the prediction module.

First claim

We claim:

1 . A system ( 10 ) for predicting a touch position of a pointer ( 21 ) on a touch-enabled unit ( 13 ), said system comprising:
a 3D imaging unit ( 11 ); a processing unit ( 12 ) coupled to the 3D imaging unit ( 11 ); a touch-enabled unit ( 13 ) coupled to the processing unit ( 12 ), wherein the 3D imaging unit ( 11 ) is configured to monitor an interaction zone in front of the touch-enabled unit ( 13 ), and the processing unit ( 12 ) comprises a prediction module ( 14 ) that is configured to predict where a pointer ( 21 ) that approaches the touch-enabled unit ( 13 ) and is monitored by the 3D imaging unit ( 11 ) will touch the touch-enabled unit ( 11 ); and a calibration module ( 15 ) that is configured to generate at least one calibration parameter ( 31 ) by comparing the predicted touch position ( 28 ) to the actual touch position ( 29 ) of the pointer ( 21 ) detected by the touch-enabled unit ( 13 ) and to transfer the at least one calibration parameter ( 31 ) to the prediction module ( 14 ) to calibrate the prediction module ( 14 ).

2 . The system ( 10 ) as claimed in claim 1 , wherein the prediction module ( 14 ) is configured to provide coordinates of the predicted touch position ( 28 ) in a coordinate system attached to the 3D imaging unit ( 11 ) and the touch-enabled unit ( 13 ) is configured to provide coordinates of the actual touch position ( 29 ) in a coordinate system attached to the touch-enabled unit ( 13 ), and wherein the at least one calibration parameter ( 31 ) comprises translating and/or rotating and/or scaling parameters for at least one of the coordinate systems attached to the 3D imaging unit ( 11 ) and attached to the touch-enabled unit ( 13 ) such that a deviation between the coordinates of the predicted touch position ( 28 ) in the coordinate system attached to the 3D imaging unit ( 11 ) and the coordinates of the actual touch position ( 29 ) in the coordinate system attached to the touch-enabled unit ( 13 ) is minimized.

3 . The system ( 10 ) as claimed in claim 1 , wherein a 3D image of a scene captured by the 3D image unit ( 11 ) comprises a 2D intensity image of the scene and a depth map of the scene, and wherein the at least one calibration parameter ( 31 ) comprises an offset for the values of the depth map.

4 . The system ( 10 ) as claimed in one of the preceding claims, wherein the calibration module ( 15 ) is configured to use a plurality of pairs for the generation of the at least one calibration parameter ( 31 ), wherein each of the pairs contains a predicted touch position ( 28 ) and a corresponding actual touch position ( 29 ).

5 . The system ( 10 ) as claimed in claim 4 , wherein the calibration module ( 15 ) is configured to use only those pairs of the plurality of pairs for the generation of the at least one calibration parameter ( 31 ) that contain a predicted touch position ( 28 ) that was predicted when the distance of the pointer ( 21 ) from the touch-enabled unit ( 13 ) was in a predetermined distance range, and/or
wherein the calibration module ( 15 ) is configured to use only those pairs of the plurality of pairs for the generation of the at least one calibration parameter ( 31 ) that contain a predicted touch position ( 28 ) that was predicted during a predetermined time interval before the pointer ( 21 ) touched the touch-enabled unit ( 13 ).

6 . The system ( 10 ) as claimed in claim 4 , wherein the calibration module ( 15 ) is configured to weight each pair with a respective uncertainty factor before using the pairs for the generation of the at least one calibration parameter ( 31 ), wherein the uncertainty factor depends on the distance of the pointer ( 21 ) from the touch-enabled unit ( 13 ) when the prediction module ( 14 ) predicted the touch position of the respective pair.

7 . The system ( 10 ) as claimed in claim 1 , wherein the calibration module ( 15 ) is configured to request a user to touch at least one predetermined position on the touch-enabled unit ( 13 ) with the pointer ( 21 ), wherein the prediction module ( 14 ) is configured to predict where the pointer ( 21 ) that approaches the touch-enabled unit ( 13 ) to touch the at least one predetermined position will touch the touch-enabled unit ( 13 ), and wherein the calibration module ( 15 ) is configured to generate the at least one calibration parameter ( 31 ) by comparing the predicted touch position ( 28 ) to the at least one predetermined position.

8 . The system ( 10 ) as claimed in claim 1 , said system further comprising a user identification module ( 32 ) that is configured to identify a user using the system ( 10 ), wherein the calibration module ( 15 ) is configured to generate the at least one calibration parameter ( 31 ) for a respective user and to store the at least one calibration parameter ( 31 ) in a memory, wherein the at least one calibration parameter is transferred to the prediction module ( 14 ) from the memory if the user identification module ( 32 ) identifies the respective user using the system ( 10 ).

9 . A method for predicting a touch position of a pointer ( 21 ) on a touch-enabled unit ( 13 ), said method comprising:
monitoring an interaction zone in front of the touch-enabled unit ( 13 ) by using a 3D imaging unit ( 11 ); predicting by using a prediction method where a pointer ( 21 ) that approaches the touch-enabled unit ( 13 ) and is monitored by the 3D imaging unit ( 11 ) will touch the touch-enabled unit ( 13 ); and generating at least one calibration parameter ( 31 ) by comparing the predicted touch position ( 28 ) to the actual touch position ( 29 ) of the pointer ( 21 ) detected by the touch-enabled unit ( 13 ), and using the at least one calibration parameter ( 31 ) to calibrate the prediction method.

10 . A system ( 50 ) for determining a pointing direction of a pointer ( 21 ) in 3D space, said system comprising:
a 3D imaging unit ( 11 ), and a processing unit ( 51 ) coupled to the 3D imaging unit ( 11 ), wherein the processing unit ( 51 ) comprises a determination module ( 52 ) that is configured to determine a pointing direction ( 55 ) of a pointer ( 21 ) of a user that is monitored by the 3D imaging unit ( 11 ) when the user has the intention to point to an object ( 54 ), and a calibration module ( 53 ) that is configured to generate at least one calibration parameter ( 56 ) by comparing the determined pointing direction ( 55 ) to the position of the object ( 54 ) and to transfer the at least one calibration parameter ( 56 ) to the determination module ( 52 ) in order to calibrate the determination module ( 52 ).

11 . The system ( 50 ) as claimed in claim 10 , further comprising an input unit ( 57 ) for inputting a command by the user, wherein the calibration module ( 53 ) is configured to assume that the user has the intention to point to the object ( 54 ) if the command input by the user in the input unit ( 57 ) is related to the object ( 54 ).

12 . The system ( 50 ) as claimed in claim 10 , wherein the calibration module ( 53 ) is configured to request the user to point with the pointer ( 21 ) to a predetermined object ( 54 ), wherein the determination module ( 52 ) is configured to determine the pointing direction ( 55 ) of the pointer ( 21 ) when pointing to the predetermined object ( 54 ), and wherein the calibration module ( 53 ) is configured to generate the at least one calibration parameter ( 56 ) by comparing the determined pointing direction ( 55 ) to the position of the predetermined object ( 54 ).

13 . The system ( 50 ) as claimed in claim 10 , wherein the determination module ( 52 ) is configured to determine the pointing direction ( 55 ) of the pointer ( 2

Assignees

• Delphi Tech Llc

Inventors

• Barth Alexander

Classifications

• G06F3/0425 Primary

  using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected (tracking a projected light spot to determine a position on a display surface G06F3/0386) · CPC title

• G06F2203/04101

  2.5D-digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface and also measures the distance of the input means within a short range in the Z direction, possibly with a separate measurement setup · CPC title

• G06F2203/04108

  Touchless 2D- digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface without distance measurement in the Z direction · CPC title

• G06F3/0412 Primary

  Digitisers structurally integrated in a display · CPC title

• G06F3/0418

  for error correction or compensation, e.g. based on parallax, calibration or alignment · CPC title