Method and apparatus for classifying finger touch events

Patent metadata

Field	Value
Publication number	US-11262864-B2
Application number	US-201715832080-A
Country	US
Kind code	B2
Filing date	Dec 5, 2017
Priority date	Mar 25, 2013
Publication date	Mar 1, 2022
Grant date	Mar 1, 2022

Abstract

A system for classifying touch events includes a touch screen configured to display an interactive element, one or more acoustic sensors coupled to the touch screen, a touch event detector configured to monitor the one or more acoustic sensors and to save acoustic signals sensed by the one or more acoustic sensors, wherein the touch event detector is further configured to detect touch events in which the interactive element is touched by a first or a second finger part of a user, and wherein the touch events result in generating the acoustic signals, and an acoustic classifier configured to classify the acoustic signals.

First claim

What is claimed is:

1. A system comprising:
a touch screen for displaying an interactive element generated by one or more applications; one or more acoustic sensors associated with the touch screen;
a touch event detector configured to monitor the one or more acoustic sensors and to save acoustic signals sensed by the one or more acoustic sensors and generated for a touch events from the interactive element being touched by a first or a second finger part of a user; and
an acoustic classifier configured to:
segment the acoustic signals into a digital representation of the acoustic signals,
convert the digital representation of the acoustic signals into electrical signals,
derive a series of features from the electrical signals, and
use the series of features to distinguish whether a finger part used for the touch event is the first finger part or the second finger part,

the one or more applications being configured to scroll a display window presented on the touch screen if the first finger part is used for the touch event or display a context menu having a plurality of selectable options if the second finger part is used for the touch event.

2. The system of claim 1 , wherein the touch event detector is further configured to detect touch events in which a second interactive element of the touch screen is touched by a third or a fourth finger part of a user, resulting in generating second acoustic signals, the acoustic classifier is further configured to repeat the operations to segment, convert, derive, and use for the second acoustic signals from the third or fourth finger part, and the one or more applications are configured to scroll a display window presented on the touch screen if the third finger part is used for the touch events or highlight a text or image portion in the display window if the fourth finger is dragged over such text or image portion.

3. The system of claim 1 , wherein the touch event detector is further configured to detect touch events in which a second interactive element displayed by the touch screen is touched by a third or a fourth finger part of a user, resulting in generating second acoustic signals , the acoustic classifier is further configured to repeat the operations to segment, convert, derive, and use for the second acoustic signals from the third or fourth finger part, and the one or more applications are configured to scroll a display window presented on the touch screen if the third finger part is used for the touch events or select a text or image portion in the display if the fourth finger is dragged over such text or image portion.

4. The system of claim 1 , wherein the touch event detector is further configured to detect touch events in which a second interactive element displayed by the touch screen is touched by a third, fourth, or fifth finger part of a user, resulting in generating second acoustic signals, the acoustic classifier is further configured to repeat the operations to segment, convert, derive, and use for the second acoustic signals from the third, fourth, or fifth finger part to distinguish between the third, fourth, or fifth finger part, and the one or more applications include a drawing application that is configured to display a thick brush, a thin marker, or an eraser for drawing on the touch screen if the third, fourth, or fifth finger part, respectively, is used for the touch events.

5. The system of claim 1 , wherein the touch event detector is further configured to detect touch events in which a second interactive element displayed by the touch screen is touched by a third or a fourth finger part of a user, resulting in generating second acoustic signals, the acoustic classifier is further configured to repeat the operations to segment, convert, derive, and use for the second acoustic signals from the third or fourth finger part, and the one or more applications include a gaming application that is configured to move a character on the touch screen if the third finger part is used for the touch events or move the character in a crouched position if the fourth finger is used for the touch events.

6. The system of claim 1 , wherein the touch event detector is further configured to detect touch events in which a second interactive element displayed by the touch screen is touched by a third or a fourth finger part of a user, resulting in generating second acoustic signals, the acoustic classifier is further configured to repeat the operations to segment, convert, derive, and use for the second acoustic signals from the third or fourth finger part, and the one or more applications include a gaming application that is configured to display and use a primary weapon on the touch screen if the third finger part is used for the touch events or display and use an auxiliary weapon on the touch screen if the fourth finger is used for the touch events.

7. The system of claim 1 , wherein the touch event detector is further configured to detect touch events in which a second interactive element displayed by the touch screen is touched by a third or a fourth finger part of a user, resulting in generating second acoustic signals, the acoustic classifier is further configured to repeat the operations to segment, convert, derive, and use for the second acoustic signals from the third or fourth finger part, and the one or more applications include a gaming application that is configured to move a character on the touch screen if the third finger part is used for the touch events or move the character in a fast mode if the fourth finger is used for the touch events.

8. The system of claim 1 , wherein the acoustic signals saved by the touch event detector are transmitted to the acoustic classifier to enable the acoustic classifier to classify the acoustic signals, and the acoustic classifier identifies whether the first finger part or the second finger part of the user is used to touch the interactive element based on classifying the acoustic signals.

9. The system of claim 8 , wherein the touch events result in the generation of the acoustic signals from a surface of the touch screen.

10. The system of claim 9 , wherein the touch event detector is arranged at a rear side of the touch screen.

11. The system of claim 1 , wherein segmenting the acoustic signals includes sampling the acoustic signals using a sliding window of samples of the acoustic signals.

12. The system of claim 11 , wherein converting the digital representation includes performing a Fourier Transform on the sampled acoustic signals having time domain representation to generate the electrical signals having frequency domain representation, and wherein deriving the series of features is based on the frequency domain representation of the electrical signals.

13. A method comprising:
displaying an interactive element of a touch screen;
monitoring one or more acoustic sensors coupled to the touch screen to save acoustic signals sensed by the one or more acoustic sensors;
detecting touch events in which the interactive element is touched by a first or a second finger part of a user, wherein the touch events result in generating the acoustic signals;
classifying the acoustic signals by segmenting the acoustic signals into a digital representation of the acoustic signals, converting the digital representation of the acoustic signals into electrical signals, and deriving a series of features from the electrical signal;
determining whether the first finger part or the second finger part is used to touch the interactive element on the touch screen based on classified acoustic signals, and
scrolling a display window of the touch screen if the first finger par

Assignees

• Qeexo Co

Inventors

• Harrison Christopher
• Schwarz Julia
• Xiao Robert

Classifications

• G06F3/0488

  using a touch-screen or digitiser, e.g. input of commands through traced gestures · CPC title

• G06F3/04842

  Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence) · CPC title

• G06F3/0433

  in which the acoustic waves are either generated by a movable member and propagated within a surface layer or propagated within a surface layer and captured by a movable member · CPC title

• G06F3/0416 Primary

  Control or interface arrangements specially adapted for digitisers · CPC title

• G06F3/0482

  Interaction with lists of selectable items, e.g. menus · CPC title