Device and method for detecting gestures on the skin

Patent metadata

Field	Value
Publication number	US-9921658-B2
Application number	US-201414374276-A
Country	US
Kind code	B2
Filing date	Feb 6, 2014
Priority date	Feb 6, 2014
Publication date	Mar 20, 2018
Grant date	Mar 20, 2018

Abstract

A device and method in accordance with the present disclosure provide an input to an electronic device via a gesture performed by a user. A distortion of the user's skin surface due to a gesture performed on the user's skin surface is detected, and a characteristic of the detected distortion is determined. Based on the determined characteristic, the gesture performed on the user's skin surface is determined.

First claim

What is claimed is:

1. A method of determining a gesture performed by a user, the method comprising:
using a camera to obtain a sequence of images of a skin surface of the user;
comparing the sequence of images to each other to detect a distortion of the skin surface in a lateral direction relative to the camera, the distortion due to a gesture performed on the user's skin surface, wherein:
the distortion of the skin surface comprises at least one of a wrinkling of the skin surface, a flattening of the skin surface, or a lateral movement of the skin surface relative to the camera;
determining the distortion of the skin includes determining a characteristic of the detected distortion based on a determined movement of one or more points on the skin surface;
determining the characteristic comprises:
for each of the sequence of images, identifying the one or more points on the skin surface relative to the camera; and
determining the movement of the one or more points by comparing a lateral position of each of the one or more points relative to the camera in the sequence of images, wherein the determined movement comprises at least one of lateral movement of the one or more points relative to the camera, compression of the one or more points on the skin surface relative to the camera, flattening of the one or more points on the skin surface relative to the camera, or stretching of the one or more points on the skin surface relative to the camera; and

based on the determined characteristic, determining the gesture performed on the user's skin surface.

2. The method according to claim 1 , wherein determining the gesture includes comparing the characteristic of the detected distortion of the skin surface to a plurality entries stored in a memory, each entry including skin distortion data corresponding to distortion of the skin surface due to a predetermined gesture.

3. The method according to claim 1 , wherein the gesture is performed on a forearm of the user, and detecting the distortion of the skin is performed at a wrist of the user.

4. The method according to claim 1 , wherein detecting the distortion of the skin surface comprises using an optical device to detect the distortion.

5. The method according to claim 1 , wherein the gesture comprises at least one of a single flick of the user's skin surface, a double flick of the user's skin surface in the same direction, a double flick of the user's skin surface in opposing directions, or dragging the user's skin surface.

6. The method according to claim 1 , wherein detecting distortion of the skin surface comprises detecting movement of the skin surface relative to a reference point.

7. The method according to claim 6 , further comprising:
concluding there is skin movement when there is a difference between one or more sequence of images.

8. A gesture detection device, comprising:
a processor and memory;
a camera operatively coupled to the processor, the camera configured to provide to the processor image data corresponding to skin distortion; and
logic stored in the memory and executable by the processor, the logic configured to:
cause the camera to capture a sequence of images of a skin surface of a user;
compare the sequence of images to each other to detect a distortion of the skin surface in a lateral direction relative to the camera, the distortion due to a gesture performed on the skin surface, wherein the distortion of the skin surface comprises at least one of a wrinkling of the skin surface, a flattening of the skin surface, or a lateral movement of the skin surface relative to the camera;
determine a characteristic of the detected distortion based on a determined movement of one or more points on the skin surface, the determining comprising:
for each of the sequence of images, identify the one or more points on the skin surface relative to the camera; and
determine the movement of the one or more points by comparing a lateral position of each of the one or more points relative to the camera in the sequence of images, wherein the determined movement comprises at least one of lateral movement of the one or more points relative to the camera, compression of the one or more points on the skin surface relative to the camera, flattening of the one or more points on the skin surface relative to the camera, or stretching of the one or more points on the skin surface relative to the camera; and

determine a gesture performed on the skin surface based on the determined characteristic.

9. The detection device according to claim 8 , wherein the logic is configured to determine the gesture by comparing the characteristic of the detected distortion to a plurality entries stored in the memory, each entry including skin distortion data having a characteristic corresponding to skin distortion due to a predetermined gesture.

10. The device according to claim 9 , further comprising an attachment device configured for attachment to a wrist, wherein the camera is attached to the attachment device.

11. The device according to claim 9 , wherein the logic configured to determine the characteristic of detected distortion based on image data provided by the camera includes logic configured to determine skin distortion corresponding to at least one of a single flick of the user's skin surface, a double flick of the user's skin surface in the same direction, a double flick of the user's skin surface in opposing directions, or dragging the user's skin surface.

12. The device according to claim 9 , wherein the logic configured to detect distortion of the user's skin surface comprises logic configured to detect movement of the user's skin surface relative to the camera.

13. The device according to claim 12 , further comprising:
logic configured to conclude there is skin movement when there is a difference between one or more sequence of images.

14. The device according to claim 9 , wherein the logic is further configured to:
determine an orientation of an arm of the user; and
logic that assigns an input command based on i) the determined gesture and ii) the determined orientation of the arm of the user, wherein:
when the arm of the user is determined to be in a first orientation, the assigned input command corresponds to a first command, and
when the arm of the user is determined to be in a second orientation different from the first orientation, the assigned input command corresponds to a second command different from the first command.

15. The device according to claim 14 , further comprising:
assigning the assigned input command to one electronic device of a plurality of electronic devices based on the determined orientation of the user's arm.

16. The device according to claim 14 , further comprising:
performing the detection step only when the determined orientation corresponds to a predetermined orientation.

17. The device according to claim 14 , wherein the logic configured to determine the orientation of the user's arm comprises logic that uses at least one of data obtained from an orientation sensor to determine the orientation.

18. The device according to claim 14 , further comprising:
logic configured to perform the compare step only when the determined orientation corresponds to a predetermined orientation.

Assignees

• Sony Corp
• Sony Mobile Communications Inc

Inventors

• Midholt Magnus
• Thorn Ola
• De Leon David

Classifications

• G06F3/011

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title

• G06F3/014 Primary

  Hand-worn input/output arrangements, e.g. data gloves · CPC title

• G06F3/017 Primary

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title