Methods, devices, and systems for modifying perceived haptic stimulations on a user

Patent metadata

Field	Value
Publication number	US-11422628-B2
Application number	US-202016856956-A
Country	US
Kind code	B2
Filing date	Apr 23, 2020
Priority date	Jan 8, 2018
Publication date	Aug 23, 2022
Grant date	Aug 23, 2022

Abstract

A method creates haptic stimulations on a user of an artificial reality system. The system includes a head-mounted display (HMD) and a wearable device. The HMD includes a display and speakers. The wearable device includes a plurality of transducers that can each generate waves to provide haptic feedback to a user. The system displays media content on the display and, in accordance with the displayed media content, determines a virtual object location in the displayed media content corresponding to a physical object location. The system provides, to the user, audio directed to the virtual object location. The system activates one or more transducers to provide haptic feedback at a target location on the user, distinct from the physical object location, to produce haptic feedback whose perceptual interpretation is at the physical object location based on a combination of the displayed media, the provided audio, and the haptic feedback.

First claim

What is claimed is:

1. A method, performed at an artificial reality system having:
a head-mounted display having a display to display media content, one or more processors, and one or more speakers; and
a wearable device, the wearable device including a plurality of transducers that can each generate one or more waves to provide haptic feedback to a user of the wearable device;
wherein the method comprises concurrently:
displaying media content on the display of the head-mounted display;
in accordance with the displayed media content, determining a virtual object location in the displayed media content;
providing, to the user of the artificial reality system via the one or more speakers, audio directed to the virtual object location;
activating one or more first transducers to generate waves at a first frequency within a first frequency range; and
activating one or more second transducers to generate waves at a second frequency within a second frequency range, distinct from the first frequency range, wherein the waves generated by the one or more first transducers and the one or more second transducers provide haptic feedback at a chosen location on the user's finger, wrist, forearm, ankle, calf, bicep, ribs, thigh, ankle, or chest, the chosen location being distinct from the wearable device location, and a combination of the displayed media, the provided audio, and the haptic feedback creates a perceptual interpretation at the virtual object location.

2. The method of claim 1 , wherein the virtual object location corresponds to a location at a virtual representation of an appendage of the user.

3. The method of claim 1 , wherein activating the one or more first transducers comprises utilizing anatomical information of the user of the wearable device determined earlier based on waves received by at least one transducer of the one or more first transducers.

4. The method of claim 1 , wherein generating waves via the one or more second transducers comprises:
activating a first transducer of the one or more second transducers at a first time; and
activating a second transducer of the one or more second transducers at a second time after the first time.

5. The method of claim 1 , wherein:
the wearable device further includes a band configured to be secured around a wrist of the user; and
each of the plurality of transducers is coupled to the band.

6. The method of claim 1 , wherein the virtual object location is separated from the wearable device by a positive distance.

7. The method of claim 3 , wherein the artificial reality system further comprises a radio and the method further comprises sending the anatomical information to a device, wherein:
the device generates a representation of a body part of the user of the wearable device from the anatomical information; and
the artificial reality system includes the representation in the displayed media content.

8. The method of claim 7 , wherein the device is a remote device that is distinct from the head-mounted display and the wearable device.

9. A artificial reality system, including:
a head-mounted display having a display to display media content, one or more processors, and one or more speakers; and
a wearable device, the wearable device including a plurality of transducers that can each generate one or more waves to provide haptic feedback to a user of the wearable device;
wherein the artificial reality system is configured to concurrently:
display media content on the display of the head-mounted display;
in accordance with the displayed media content, determine a virtual object location in the displayed media content;
provide, to the user of the artificial reality system via the one or more speakers, audio directed to the virtual object location;
activate one or more first transducers to generate waves at a first frequency within a first frequency range; and
activate one or more second transducers to generate waves at a second frequency within a second frequency range, distinct from the first frequency range, wherein the waves generated by the one or more first transducers and the one or more second transducers provide haptic feedback at a chosen location on the user's finger, wrist, forearm, ankle, calf, bicep, ribs, thigh, ankle, or chest, the chosen location being distinct from the wearable device location, and a combination of the displayed media, the provided audio, and the haptic feedback creates a perceptual interpretation at the virtual object location.

10. The artificial reality system of claim 9 , wherein activating the one or more first transducers comprises utilizing anatomical information of the user of the wearable device determined earlier based on waves received by at least one transducer of the one or more first transducers.

11. The artificial reality system of claim 9 , wherein:
the wearable device further includes a band configured to be secured around a wrist of the user; and
each of the plurality of transducers is coupled to the band.

12. The artificial reality system of claim 9 , wherein the virtual object location is separated from the wearable device by a positive distance.

13. The artificial reality system of claim 10 , further comprising a radio, wherein:
a device receiving the anatomical information generates a representation of a body part of the user of the wearable device from the anatomical information; and
the artificial reality system includes the representation in the displayed media content.

14. The artificial reality system of claim 13 , wherein the device is a remote device that is distinct from the head-mounted display and the wearable device.

15. A non-transitory computer-readable storage medium storing one or more programs configured for execution by one or more processors of an artificial reality system having a wearable device including a plurality of transducers, each transducer being configured to generate one or more waves and a head-mounted display having a display to display media content, the one or more programs including instructions, which when executed by the one or more processors cause the artificial reality system to concurrently:
display media content on the display of the head-mounted display;
in accordance with the displayed media content, determine a virtual object location in the displayed media content;
provide, to the user of the artificial reality system via the one or more speakers, audio directed to the virtual object location;
activate one or more first transducers to generate waves at a first frequency within a first frequency range; and
activate one or more second transducers to generate waves at a second frequency within a second frequency range, distinct from the first frequency range, wherein the waves generated by the one or more first transducers and the one or more second transducers provide haptic feedback at a chosen location on the user's finger, wrist, forearm, ankle, calf, bicep, ribs, thigh, ankle, or chest, the chosen location being distinct from the wearable device location, and a combination of the displayed media, the provided audio, and the haptic feedback creates a perceptual interpretation at the virtual object location.

16. The non-transitory computer-readable storage medium of claim 15 , wherein the instructions to activate the one or more first transducers comprise instructions to utilize anatomical information of the user of the wearable device determined earlier based on waves received by at least one transducer of the one or more first transducers.

17. The non-transitory computer-readable storag

Assignees

• Facebook Tech Llc
• Univ California
• Meta Platforms Tech Llc

Inventors

• Keller Sean Jason
• Trutna Tristan Thomas
• Benko Hrvoje
• King Raymond
• Stanley Andrew Arthur
• Di Luca Massimiliano
• Visell Yon
• Shao Yitian
• Dandu Bharat

Classifications

• G06F3/011

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title

• G06F3/012

  Head tracking input arrangements · CPC title

• G06F3/0426

  tracking fingers with respect to a virtual keyboard projected or printed on the surface (virtual keyboards on touch screens G06F3/04886) · CPC title

• G06F3/014

  Hand-worn input/output arrangements, e.g. data gloves · CPC title

• G06T19/006

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title