Electronic device for recognizing gesture of user from sensor signal of user and method for recognizing gesture using the same

Patent metadata

Field	Value
Publication number	US-11460928-B2
Application number	US-202017116074-A
Country	US
Kind code	B2
Filing date	Dec 9, 2020
Priority date	Dec 18, 2019
Publication date	Oct 4, 2022
Grant date	Oct 4, 2022

Abstract

Aspects of the disclosure include an electronic device comprising: a sensor device including at least one biometric sensor; memory storing a table of a plurality of gestures, wherein for each gesture, a corresponding plurality of features are stored in the memory; and at least one processor operatively connected to the sensor device and the memory, wherein the at least one processor is configured to: obtain a bio-signal of a user from the at least one biometric sensor; select a section of the bio-signal that includes one feature of the corresponding plurality of features for the plurality of gestures; determine a specific one of the plurality of gestures based on the one feature included in the section of the bio-signal and the corresponding plurality features for the plurality of gestures.

First claim

What is claimed is:

1. An electronic device comprising:
a sensor device including at least one biometric sensor;
memory storing a table of a plurality of gestures, wherein for each gesture, a corresponding plurality of features are stored in the memory; and
at least one processor operatively connected to the sensor device and the memory,
wherein the at least one processor is configured to:
obtain a bio-signal of a user from the at least one biometric sensor;
identify a crossing point between the bio-signal and a specified reference line and a transition point of the bio-signal;
segment the bio-signal into a plurality of signal sections based on at least one of the crossing point and the transition point;
extract a feature value from each of the plurality of signal sections;
determine a specific one of the plurality of gestures based on the extracted feature value and the corresponding plurality features for the plurality of gestures; and
when a number of plurality of signal sections is less than a preset number, change a value of the reference line such that the number of plurality of signal sections is equal to or greater than the preset number.

2. The electronic device of claim 1 , wherein the specified reference line is a line connecting points where a raw value becomes 0 with each other when a value of the bio-signal is the raw value.

3. The electronic device of claim 1 , wherein the specified reference line is a line connecting points where a value of the bio-signal normalized by the processor becomes 0 with each other when the value of the bio-signal is not a raw value.

4. The electronic device of claim 1 , wherein the processor is configured to:
segment the bio-signal into a plurality of signal sections using the crossing point;
detect a signal section including a plurality of transition points among the plurality of signal sections; and
segment the detected signal section such that the plurality of transition points are respectively included in different signal sections.

5. The electronic device of claim 4 , wherein the plurality of transition points include a first transition point and a second transition point set at a position adjacent to the first transition point,
wherein the processor is configured to segment the detected signal section such that the first transition point and the second transition point are included in one signal section when at least one of an amount of change in an x value and an amount of change in a y value between the first transition point and the second transition point is less than a preset amount of change.

6. The electronic device of claim 4 , wherein the plurality of transition points include a first transition point and a second transition point set at a position adjacent to the first transition point,
wherein the processor is configured to segment the detected signal section such that the first transition point and the second transition point are included in one signal section when a magnitude of the first transition point is equal to or greater than a preset magnitude, and a magnitude of the second transition point is less than the preset magnitude.

7. The electronic device of claim 4 , wherein the processor is configured to segment a first signal section using a preset distance as a reference when, in the first signal section including a first transition point among the plurality of signal sections, a distance between the first transition point and a start point of the first signal section or between the first transition point and an end point of the first signal section exceeds the preset distance.

8. The electronic device of claim 4 , wherein the processor is configured to:
determine whether at least one transition point is included in each of the plurality of signal sections; and
set signal sections including the at least one transition point among the plurality of signal sections as a plurality of candidate sections for recognizing the gesture of the user.

9. The electronic device of claim 8 , wherein the processor is configured to apply each weighted value of gesture recognition to each of the plurality of candidate sections, wherein each weighted value is inversely proportional to an intensity of the bio-signal for each candidate section.

10. The electronic device of claim 4 , wherein the processor is configured to set signal sections where an intensity of the bio-signal is equal to or greater than a preset intensity among the plurality of signal sections as a plurality of candidate sections.

11. The electronic device of claim 4 , wherein the processor is configured to set signal sections where an integral value of the bio-signal is less than a preset integral value among the plurality of signal sections as a plurality of candidate sections.

12. The electronic device of claim 4 , wherein the processor is configured to:
segment the bio-signal into the plurality of signal sections using the crossing point; and
set a preset reference line such that at least one transition point is included in each of the plurality of signal sections.

13. The electronic device of claim 1 , wherein the sensor device includes at least one of an acceleration sensor, a photoplethysmography (PPG) sensor, a gyro sensor, a geomagnetic sensor, and an electrocardiogram sensor.

14. The electronic device of claim 1 , wherein the processor is configured to:
define a form of each of the segmented signal sections using the transition point included in each of the segmented signal sections;
set a sequence of the segmented signal sections based on the form of each of the segmented signal sections; and
extract the sequence of the segmented signal sections as the feature value.

15. The electronic device of claim 1 , wherein the processor is configured to independently extract the feature value from each of the segmented signal sections.

16. The electronic device of claim 1 , wherein the processor is configured to extract the feature value using information of a second signal section including a gesture signal of the segmented signal sections and information of a third signal section not including the gesture signal when the bio-signal includes a signal of a predetermined pattern and the gesture signal.

17. A method for recognizing a gesture performed by an electronic device, the method comprising:
sensing a sensor signal from a user;
storing a table of a plurality of gestures, wherein for each gesture, a corresponding plurality of features are stored;
setting a first crossing point between the sensor signal and a preset reference line and a transition point of the sensor signal;
segmenting the sensor signal using at least one of the first crossing point and the transition point;
selecting a section of the sensor signal that includes one feature of the corresponding plurality of features for the plurality of gestures; and
determining a specific one of the plurality of gestures based on the one feature included in the segment and the corresponding plurality features for the plurality of gestures,
wherein the method further comprises:
when a number of the segmented section is less than a preset number, changing a value of the preset reference line;
setting a second crossing point between the sensor signal and the preset reference line that changed the value; and
additionally segmenting the sensor signal using the second crossing point such that the number of segmented section is equal to or greater than the preset number.

Assignees

• Samsung Electronics Co Ltd

Inventors

• Park Sungjin
• Kim Cheolo
• Kim Taeyoon
• Ahn Jinyoup
• Lim Chaiman
• Cho Wonjoon

Classifications

• G06F3/017 Primary

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title

• G06F3/0346

  with detection of the device orientation or free movement in a three-dimensional [3D] space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors · CPC title

• G06F3/015

  Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection · CPC title