Systems and methods for providing haptic feedback for remote interactions

Patent metadata

Field	Value
Publication number	US-10613627-B2
Application number	US-201414275216-A
Country	US
Kind code	B2
Filing date	May 12, 2014
Priority date	May 12, 2014
Publication date	Apr 7, 2020
Grant date	Apr 7, 2020

Abstract

A system includes a first electronic device and a second electronic device. The first electronic device includes a sensor configured to sense or property experienced by the first electronic device, and a transmitter configured to transmit a signal based on output from the sensor. The second electronic device is in signal communication with the first electronic device. The second electronic device includes a receiver configured to receive the transmitted signal, a detector configured to determine an object that a user of the second device is focusing on, a processor configured to generate a haptic signal representative of the transmitted signal if it is determined that the object the user is focusing on corresponds with a location of the first electronic device, and a haptic output device configured to receive the haptic signal and generate a haptic effect to the user.

First claim

What is claimed is:

1. A system comprising:
an electronic device comprising:
an eye-gaze sensor;
a wireless receiver;
one or more processors configured to:
receive one or more eye-gaze sensor signals from the eye-gaze sensor;
determine a focus object based on the one or more eye-gaze sensor signals, the focus object being a real-world object that a user of the electronic device is determined to be focused on;
receive, using the wireless receiver, one or more wireless signals from a source electronic device indicating a sensed condition or a sensed property of the focus object, the source electronic device corresponding to the focus object and configured to sense the sensed condition or the sensed property of the focus object;
generate a haptic signal based on the condition or property of the focus object; and

a haptic output device configured to receive the haptic signal and generate a haptic effect to the user.

2. The system according to claim 1 , wherein the eye-gaze sensor comprises a camera configured to capture an image of an eye of the user, and an image processor configured to determine a direction of an eye gaze of the user based on the image.

3. The system according to claim 1 , wherein the source electronic device is configured to sense vibrations or deformations of the focus object and wherein the sensed condition or the sensed property indicate the sensed vibrations or deformations.

4. The system according to claim 3 , wherein the haptic effect simulates the sensed vibrations or deformations.

5. The system according to claim 1 , wherein the source electronic device further comprises a camera configured to output a video stream, the video stream being part of the signal, and wherein the electronic device further comprises a display configured to display the video stream.

6. The system according to claim 1 , wherein the electronic device is a head-mounted display.

7. The system according to claim 6 , wherein the eye-gaze sensor comprises a camera mounted on the head-mounted display, the camera having a field of view aligned with a direction the head-mounted display is pointing.

8. A method comprising:
receiving, by an electronic device and from an eye-gaze sensor, one or more eye-gaze sensor signals from the eye-gaze sensor;
determining, by the electronic device, a focus object based on the one or more eye-gaze sensor signals, the focus object being a real-world object that a user of the electronic device is determined to be focused on;
receiving, by the electronic device and using a wireless receiver, one or more wireless signals indicating a sensed condition or sensed property experienced by the focus object from a source electronic device, the source electronic device corresponding to the focus object and configured to sense the sensed condition or the sensed property of the focus object and to transmit the one or more wireless signals;
generating, by the electronic device, a haptic signal based on the sensed condition or the sensed property of the focus object; and
transmitting the haptic signal to a haptic output device to output a haptic effect.

9. The method according to claim 8 , wherein the eye-gaze sensor comprises a camera and wherein determining the object that the user of the electronic device is focusing on comprises:
capturing, by the camera, an image of an eye of the user; and
determining a direction of an eye gaze of the user based on the image.

10. The method according to claim 8 , wherein the source electronic device is configured to sense vibrations or deformations of the focus object, and wherein the sensed condition or the sensed property indicate the sensed vibrations or deformations.

11. The method according to claim 10 , wherein the haptic effect simulates the sensed vibrations or deformations.

12. The method according to claim 8 , further comprising:
receiving, from a camera of the source electronic device, a video stream, wherein the one or more wireless signals include the video stream; and
displaying, by a display of the electronic device, the video stream.

13. The method according to claim 8 , wherein the electronic device is a head-mounted display.

14. The method according to claim 13 , wherein the eye-gaze sensor comprises a camera mounted on the head-mounted display, the camera having a field of view aligned with a direction the head-mounted display is pointing.

15. A non-transitory computer-readable medium storing program code executable by a processor, the program code configured to cause the processor to:
receive, by an electronic device and from an eye-gaze sensor, one or more eye-gaze sensor signals from the eye-gaze sensor;
determine a focus object based on the one or more eye-gaze sensor signals, the focus object being a real-world object that a user of the electronic device is determined to be focused on;
receive, by the electronic device and using a wireless receiver, one or more wireless signals indicating a sensed condition or sensed property experienced by the focus object from a source electronic device, the source electronic device corresponding to the focus object and configured to sense the sensed condition or the sensed property of the focus object and to transmit the one or more wireless signals;
generate a haptic signal based on the sensed condition or the sensed property of the focus object; and
transmit the haptic signal to a haptic output device to output a haptic effect.

16. The non-transitory computer-readable medium according to claim 15 , wherein the eye-gaze sensor comprises a camera and wherein the program code is further configured to cause the processor to:
capture, using the camera, an image of an eye of the user; and
determine a direction of an eye gaze of the user based on the image.

17. The non-transitory computer-readable medium according to claim 15 , wherein the source electronic device is configured to sense vibrations or deformations of the focus object, and wherein the sensed condition or the sensed property indicate the sensed vibrations or deformations.

18. The non-transitory computer-readable medium according to claim 17 , wherein the haptic effect is configured to simulate the sensed vibrations or deformations.

19. The non-transitory computer-readable medium according to claim 15 , wherein the program code is further configured to cause the processor to:
receive a video stream from the source electronic device, wherein the one or more wireless signals include the video stream; and
cause a display of the video stream.

20. The non-transitory computer-readable medium according to claim 15 , wherein the electronic device is a head-mounted display.

21. The non-transitory computer-readable medium according to claim 20 , wherein the eye-gaze sensor comprises a camera mounted on the head-mounted display, the camera having a field of view aligned with a direction the head-mounted display is pointing.

Assignees

• Immersion Corp

Inventors

• Levesque Vincent
• Modarres Ali
• Grant Danny A
• Dionne Jean Francois
• Birnbaum David M

Classifications

• G06F3/012

  Head tracking input arrangements · CPC title

• G06F3/016 Primary

  Input arrangements with force or tactile feedback as computer generated output to the user · CPC title

• G06F3/013

  Eye tracking input arrangements (G06F3/015 takes precedence) · CPC title

• G06T19/006

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• G02B27/0172

  characterised by optical features · CPC title