Motion and gesture input from a wearable device

Patent metadata

Field	Value
Publication number	US-10488936-B2
Application number	US-201514616573-A
Country	US
Kind code	B2
Filing date	Feb 6, 2015
Priority date	Sep 30, 2014
Publication date	Nov 26, 2019
Grant date	Nov 26, 2019

Abstract

This relates to a device that detects a user's motion and gesture input through the movement of one or more of the user's hand, arm, wrist, and fingers, for example, to provide commands to the device or to other devices. The device can be attached to, resting on, or touching the user's wrist, ankle or other body part. One or more optical sensors, inertial sensors, mechanical contact sensors, and myoelectric sensors can detect movements of the user's body. Based on the detected movements, a user gesture can be determined. The device can interpret the gesture as an input command, and the device can perform an operation based on the input command. By detecting movements of the user's body and associating the movements with input commands, the device can receive user input commands through another means in addition to, or instead of, voice and touch input, for example.

First claim

What is claimed is:

1. A portable electronic device comprising:
a touch screen included in a housing of the device;
a plurality of first light emitters included in the housing of the device, each first light emitter configured to emit first light at a user's body part;
a plurality of first optical sensors included in the housing of the device, each first optical sensor optically coupled to one of the plurality of first light emitters to form one of a plurality of first emitter-sensor pairs and configured to detect a first reflectance, the first reflectance being a reflectance of the emitted first light,
wherein each first emitter-sensor pair is associated with a different tendon located in the body part than others of the plurality of first emitter-sensor pairs;
a plurality of second light emitters included in a strap of the device, each second light emitter configured to emit second light at the user's body part, wherein the strap is separate and distinct from the housing of the device, the strap is attached to the housing of the device; and
a plurality of second optical sensors included in the strap of the device, each second optical sensor optically coupled to one of the plurality of second light emitters to form one of a plurality of second emitter-sensor pairs and configured to detect a second reflectance, the second reflectance being a reflectance of the emitted second light, wherein each second emitter-sensor pair is associated with a tendon located in the body part; and
logic included in the housing of the device configured to:
determine a gesture from the first and second reflectances detected by the first and second optical sensors, respectively, and
associate a command with the determined gesture.

2. A method of determining a gesture comprising:
detecting one or more touch inputs using a touch screen located in a housing of a device;
emitting a plurality of first light at a user's body part using a plurality of first light emitters located in the housing of the device, each first light emitted at a different tendon located in the user's body part than others of the emitted first light;
for each emitted first light from the plurality of first light emitters, detecting a change in first reflectance using a plurality of first optical sensors located in the housing of the device, the change in first reflectance indicative of movement of the associated tendon;
emitting a plurality of second light at a plurality of tendons included in the user's body part using a plurality of second light emitters located in a strap of the device, the strap separate and distinct from the housing of the device, the strap attached to the housing of the device;
for each emitted second light from the plurality of second light emitters, detecting a change in second reflectance using a plurality of second optical sensors located in the strap of the device;
determining a gesture from the detected changes in first and second reflectances using logic located in the housing of the device; and
associating a command with the determined gesture.

3. The device of claim 1 , wherein at least two of the plurality of first light emitters and at least two of the plurality of first optical sensors emit and detect light at different wavelengths.

4. The device of claim 3 , wherein the different wavelengths are selected from a group comprising infrared, blue, and green wavelengths.

5. The device of claim 1 , wherein the first optical sensors are multi-functional sensors configured to detect a photoplethysmography signal.

6. The device of claim 1 , further comprising an inertial sensor configured to determine transitional and rotational motion, the inertial sensor including an accelerator, and a gyroscope, wherein the gesture is further determined from the transitional and rotational motion.

7. The device of claim 1 , further comprising a mechanical contact sensor configured to detect and conform to movements associated with the user's body part, and wherein the gesture is further determined from the detected movements.

8. The device of claim 1 , further comprising a myoelectric sensor configured to detect electrical signals associated with the user's body part, and wherein the gesture is further determined from the detected electrical signals.

9. A portable electronic device comprising:
a touch screen included in a housing of the device;
a strap attached to the housing of the device, wherein the strap comprises an inner band and an outer band,
the inner band includes: a plurality of optical features, and
the outer band includes:
a light emitter configured to emit light at the plurality of optical features, and
an optical sensor configured to detect a reflectance of the emitted light, wherein the reflectance is associated with movement of at least one tendon located in a user's body part; and

logic configured to:
measure a change in characteristics associated with movement of the inner band in response to movement of the at least one tendon located in the user's body part,
determine a gesture based on the change in the characteristics, and
associate a command with the gesture.

10. The device of claim 9 , wherein the inner band comprises a plurality of regions,
the plurality of regions configured to stretch or compress in response to the movement of the at least one tendon,
wherein the characteristics is a resistance due to a change in the stretch or compression in at least one of the plurality of regions.

11. The device of claim 9 , further comprising an inertial sensor configured to determine transitional and rotational motion, the inertial sensor including an accelerator and a gyroscope, wherein the gesture is further determined from the transitional and rotational motion.

12. The device of claim 9 , wherein the inner band is made of a flexible material, and the outer band is made of a rigid material.

13. The device of claim 9 , wherein the plurality of optical features form a pattern from spatially separated optical features, and the pattern is disposed around a circumference of the strap.

14. The device of claim 9 , further comprising a myoelectric sensor configured to detect electrical signals associated with the user's body part, and wherein the gesture is further determined from the detected electrical signals.

15. The method of claim 2 , further comprising:
recording the association of the command with the determined gesture.

16. The method of claim 2 , wherein each emitted first and second light includes a different wavelength than others of the emitted first and second light.

17. The method of claim 2 further comprising:
measuring a photoplethysmography signal using the plurality of first light emitters and the plurality of first optical sensors.

18. The method of claim 2 , wherein the determining the gesture includes receiving a second gesture or movement information from another device, and the associated command is further based on the second gesture.

19. The method of claim 2 , further comprising:
determining translational and rotational motion using an accelerometer and a gyroscope,
wherein determining the gesture further includes determining the gesture from the determined translational and rotational motion.

Assignees

• Apple Inc

Inventors

• Baranski Andrzej
• Shedletsky Anna-Katrina
• Lonkar Kuldeep P
• Isikman Serhan
• Lynch Stephen Brian
• Ely Colin M
• Werner Christopher
• De Jong Erik
• Weiss Samuel B

Classifications

• G06F1/163

  Wearable computers, e.g. on a belt · CPC title

• A61B2562/0261

  Strain gauges · CPC title

• A61B5/0059

  using light, e.g. diagnosis by transillumination, diascopy, fluorescence (photoacoustic A61B5/0093; optical measurement of heart rate A61B5/02416; optical measurement of blood flow A61B5/0261; optical measurement of analytes A61B5/1455) · CPC title

• A61B5/11

  Measuring movement of the entire body or parts thereof, e.g. head or hand tremor or mobility of a limb {(A61B5/1038 takes precedence; motion detection to correct for motion artifacts in physiological signals A61B5/721)} · CPC title

• A61B5/1125

  Grasping motions of hands · CPC title