Electronic controller with finger motion sensing

Patent metadata

Field	Value
Publication number	US-10874939-B2
Application number	US-201816010385-A
Country	US
Kind code	B2
Filing date	Jun 15, 2018
Priority date	Jun 16, 2017
Publication date	Dec 29, 2020
Grant date	Dec 29, 2020

Abstract

A controller includes a body having a handle, and an array of proximity sensors spatially distributed on, in, beneath, or near the outer surface of the handle, responsive to a proximity of a user's fingers to that outer surface. A finger tracker converts the output of the array of proximity sensors to a set of joint angles corresponding to a plurality of the user's fingers. The controller may include a renderer for processing the joint angles to deform a hand mesh that is rendered for display. Values may be calculated to facilitate normalization of the output of the proximity sensor array and thereby generate a set of normalized finger detection data. This data may be processed through curl logic to produce a linear estimate of gross finger curl with respect to the user and thereby generate a set of estimates for a plurality of finger joint angles for the user.

First claim

We claim:

1. A system comprising:
a handheld controller comprising:
a controller body having a handle; and
a proximity sensor coupled to the handle and responsive to a proximity of a finger of a user to the handle;

one or more processors; and
one or more computer-readable media storing computer-executable instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising:
receiving, from the proximity sensor, proximity data representing the proximity of the finger of the user to the handle;
determining an unfiltered finger curl value based at least in part on the proximity data;
passing the unfiltered finger curl value through one or more filters to obtain a filtered finger curl value; and
determining an estimated joint angle of the finger based at least in part on the filtered finger curl value.

2. The system of claim 1 , the operations further comprising:
processing the estimated joint angle to deform a hand mesh that is rendered for display.

3. The system of claim 1 , wherein the determining of the unfiltered finger curl value based at least in part on the proximity data comprises:
calculating a set of values based at least in part on the proximity data;
generating, based at least in part on the set of values, a set of normalized finger detection data; and
processing the set of normalized finger detection data to determine the unfiltered finger curl value.

4. The system of claim 1 , wherein the handheld controller is for use in conjunction with a virtual reality system.

5. The system of claim 1 , wherein the handheld controller includes a tracking arc that is fixed to the controller body.

6. The system of claim 5 , wherein the tracking arc includes a first plurality of tracking sensors, the first plurality of tracking sensors being responsive to electromagnetic radiation emitted by an electronic device of the system.

7. The system of claim 1 , wherein the handheld controller includes a hand retainer configured to physically bias a palm of the user against an outside surface of the handle.

8. The system of claim 1 , wherein the controller body further comprises a distal head that extends from the handle, the distal head including:
a distal end, and
at least one thumb-operated control disposed between the handle and the distal end.

9. The system of claim 1 , wherein the proximity sensor comprises a plurality of capacitive sensors embedded under an outer surface of the handle, the outer surface comprising an electrically insulative material.

10. A method comprising:
receiving first data from a proximity sensor that represents a proximity of a finger of a user relative to a handle of a controller;
determining an unfiltered finger curl value based at least in part on the first data;
passing the unfiltered finger curl value through one or more filters to obtain a filtered finger curl value; and
determining an estimate of a joint angle of the finger based at least in part on the filtered finger curl value.

11. The method according to claim 10 , further comprising:
generating a hand mesh based at least in part on the estimate of the joint angle of the finger; and
rendering the hand mesh for display.

12. The method according to claim 10 , further comprising:
receiving second data from the proximity sensor prior to the receiving of the first data;
determining that the second data corresponds to a predetermined hand gesture of the user; and
initializing finger tracking in response to the determining that the second data corresponds to the predetermined hand gesture.

13. The method according to claim 12 , wherein the predetermined hand gesture comprises an ungrasp gesture.

14. The system of claim 1 , wherein the proximity sensor is incorporated into one or more flexible printed circuits, and wherein each of the one or more flexible printed circuits is attached to an inner surface of the handle.

15. The system of claim 1 , wherein the proximity sensor comprises an array comprising a plurality of capacitive sensors.

16. The method according to claim 10 , further comprising:
calculating a plurality of values based at least in part on the first data to facilitate normalization of the proximity sensor with respect to the user; and
generating a set of normalized finger detection data based at least in part on the plurality of values.

17. The method according to claim 16 , further comprising:
processing the set of normalized finger detection data through a set of curl logic to produce an estimate of finger curl with respect to the user; and
calculating the estimate of the joint angle based at least in part on the estimate of the finger curl.

18. The method according to claim 17 , wherein the estimate of the joint angle is a linear estimate.

19. The system of claim 1 , wherein the estimated joint angle of the finger comprises a first estimated joint angle of the finger, the operations further comprising:
determining a joint model associated with the finger of the user; and
determining, based at least in part on the joint model and the first estimated joint angle, a second estimated joint angle of the finger.

20. The system of claim 1 , wherein the finger of the user comprises a first finger of the user, the handheld controller further comprising:
an additional proximity sensor coupled to the handle and responsive to a proximity of a second finger of the user to the handle;
and
wherein the operations further comprise:
receiving, from the additional proximity sensor, additional proximity data representing the proximity of the second finger of the user to the handle, the additional proximity data being used to determine the estimated joint angle of the first finger.

21. The system of claim 1 , wherein the finger of the user comprises a first finger of the user, the handheld controller further comprising:
a force-sensing resistor (FSR) coupled to the handle and responsive to at least one of the first finger or a second finger of the user grasping the handle; and
wherein the operations further comprise:
receiving, from the FSR, force data representing a force of the at least one of the first finger or the second finger of the user grasping the handle, the force data being used to determine the estimated joint angle of the first finger.

22. The method according to claim 10 , wherein the finger of the user comprises a first finger of the user, the method further comprising:
receiving second data from an additional proximity sensor that represents a proximity of a second finger of the user relative to the handle, wherein the second data is used to determine the estimate of the joint angle of the first finger.

23. The method according to claim 10 , wherein the finger of the user comprises a first finger of the user, the method further comprising:
receiving second data from a force-sensing resistor (FSR) that represents a force of at least one of the first finger or a second finger of the user grasping the handle, wherein the second data is used to determine the estimate of the joint angle of the first finger.

24. A device comprising:
one or more processors; and
one or more computer-readable media storing computer-executable instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising:
receiving proximity data generated by a proximity sensor, the proximity data

Assignees

• Valve Corp

Inventors

• Nietfeld Scott
• Leinbaugh Jeffrey George
• Hope Eric James
• Bryan Philip
• Britt Scott Richard
• Mucha Jeffrey Walter
• Gottschalk Claire Michelle
• Slocum Jeremy

Classifications

• G06F3/014

  Hand-worn input/output arrangements, e.g. data gloves · CPC title

• A63F2300/8082

  Virtual reality · CPC title

• A63F13/42

  by mapping the input signals into game commands, e.g. mapping the displacement of a stylus on a touch screen to the steering angle of a virtual vehicle · CPC title

• A63F13/245 Primary

  specially adapted to a particular type of game, e.g. steering wheels · CPC title

• A63F13/214

  for locating contacts on a surface, e.g. floor mats or touch pads · CPC title