Method and apparatus for classifying finger touch events on a touchscreen

Patent metadata

Field	Value
Publication number	US-9864454-B2
Application number	US-201514612089-A
Country	US
Kind code	B2
Filing date	Feb 2, 2015
Priority date	Mar 25, 2013
Publication date	Jan 9, 2018
Grant date	Jan 9, 2018

Abstract

A system for classifying touch events includes a touch screen configured to display an interactive element, one or more vibro-acoustic sensors coupled to the touch screen, a touch event detector configured to monitor the one or more vibro-acoustic sensors and to save vibro-acoustic signals sensed by the one or more vibro acoustic sensors, wherein the touch event detector is further configured to detect touch events in which the interactive element is touched by a first or a second finger part of a user, and wherein the touch events result in generating the vibro-acoustic signals, and a vibro-acoustic classifier configured to classify the vibro-acoustic signals.

First claim

What is claimed is:

1. A system for classifying finger touch events, the system comprising:
a touch screen configured to display an interactive element;
one or more vibro acoustic sensors coupled to the touch screen;
a touch event detector configured to monitor the one or more vibro acoustic sensors and to save vibro-acoustic signals sensed by the one or more vibro acoustic sensors, wherein the touch event detector is further configured to detect touch events in which the interactive element is touched by a first or a second finger part from a plurality of finger parts of a user including a finger tip, a knuckle and a fingernail, and wherein the touch events result in generating the vibro-acoustic signals; and
a vibro-acoustic classifier configured to classify the vibro-acoustic signals, wherein a first function or a second function related to the interactive element is selectively activated based on classification of the vibro-acoustic signals by the vibro-acoustic classifier.

2. The system of claim 1 , wherein the touch event detector is triggered by the touch screen.

3. The system of claim 1 , wherein the touch event detector is triggered by onset of the vibro-acoustic signal.

4. The system of claim 1 , wherein a vibro-acoustic sensor is an impact sensor, a vibration sensor, an accelerometer, a strain gauge, or an acoustic sensor.

5. The system of claim 4 , wherein the acoustic sensor is a condenser microphone, a piezoelectric microphone or a MEMS microphone.

6. The system of claim 1 , wherein the vibro-acoustic signals saved by the touch event detector is transmitted to the vibro-acoustic classifier to enable the vibro-acoustic classifier to classify the vibro-acoustic signals.

7. The system of claim 6 , wherein the vibro-acoustic classifier identifies whether the first finger part or the second finger part of the user is used to touch the interactive element based on classifying the vibro-acoustic signals.

8. The system of claim 7 , wherein the touch events result in the generation of the vibro-acoustic signal from a surface of the touch screen.

9. The system of claim 8 , wherein the touch event detector is arranged at a rear side of the touch screen.

10. The system of claim 6 , wherein the vibro-acoustic classifier comprises:
a segmentation unit configured to segment the vibro-acoustic signals into a digital representation of the vibro-acoustic signals;
a conversion unit coupled to the segmentation unit and configured to convert the digital representation of the vibro-acoustic signals into electrical signals;
a feature extraction unit coupled to the conversion unit and configured to derive a series of features from the electrical signals; and
a classification unit coupled to the feature extraction unit and configured to use the series of features to distinguish whether a finger part used for the touch event is the first finger part or the second finger part.

11. The system of claim 10 , wherein the segmentation unit is configured to sample the vibro-acoustic signals using a sliding window of samples of the vibro-acoustic signals.

12. The system of claim 11 , wherein the conversion unit is configured to perform a Fourier Transform on the sampled vibro-acoustic signals having time domain representation to generate the electrical signals having frequency domain representation, and wherein the feature extraction unit is configured to calculate a series of features from the frequency domain representation of the electrical signals.

13. A method for classifying touch events, the method comprising:
displaying an interactive element of a touch screen;
monitoring one or more vibro acoustic sensors coupled to the touch screen to save vibro-acoustic signals sensed by the one or more vibro acoustic sensors;
detecting touch events in which the interactive element is touched by a first or a second finger part of a user, wherein the touch events result in generating the vibro-acoustic signals;
classifying the vibro-acoustic signals;
determining whether the first finger part or the second finger part from a plurality of finger parts is used to touch the interactive element on the touch screen based on classified vibro-acoustic signals, wherein the plurality of finger parts include a fingertip, a knuckle and a finger nail, and
activating a first function or a second function related to the interactive element based on determining whether the first finger part or the second finger part is used to touch the interactive element on the touch screen.

14. The method of claim 13 , wherein classifying the vibro-acoustic signals comprises:
segmenting the vibro-acoustic signals into a digital representation of the vibro-acoustic signals;
converting the digital representation of the vibro-acoustic signals into electrical signals; and
deriving a series of features from the electrical signals.

15. The method of claim 14 , wherein segmenting the vibro-acoustic signals comprise sampling the vibro-acoustic signals using a sliding window of samples of the vibro-acoustic signals.

16. The method of claim 15 , wherein converting the digital representation of the vibro-acoustic signals comprises performing a Fourier Transform on sampled vibro-acoustic signals having time domain representation to generate the electrical signals having frequency domain representation.

17. The method of claim 16 , wherein deriving the series of features comprise calculating a series of features from the frequency domain representation of the electrical signals, and distinguishing whether a finger part used for the touch event is the first finger part or the second finger part based on the series of features.

18. A non-transitory computer readable medium containing instructions for classifying touch events, wherein execution of the program instructions by a processor causes the processor to carry out the steps of:
displaying an interactive element of a touch screen;
monitoring one or more vibro acoustic sensors coupled to the touch screen to save vibro-acoustic signals sensed by the one or more vibro acoustic sensors;
detecting touch events in which the interactive element is touched by a first or a second finger part of a user from a plurality of finger parts including a fingertip, a knuckle and a finger nail, wherein the touch events result in generating the vibro-acoustic signals;
classifying the vibro-acoustic signals;
determining whether the first finger part or the second finger part is used to touch the interactive element on the touch screen based on classified vibro-acoustic signals, and
activating a first function or a second function related to the interactive element based on determining whether the first finger part or the second finger part is used to touch the interactive element on the touch screen.

Assignees

• Qeexo Co

Inventors

• Harrison Christopher
• Schwarz Julia
• Xiao Robert

Classifications

• G06F3/043

  using propagating acoustic waves · CPC title

• G06F3/0482

  Interaction with lists of selectable items, e.g. menus · CPC title

• G06F3/04842

  Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence) · CPC title

• G06F3/0416 Primary

  Control or interface arrangements specially adapted for digitisers · CPC title

• G06F3/0433

  in which the acoustic waves are either generated by a movable member and propagated within a surface layer or propagated within a surface layer and captured by a movable member · CPC title