Systems and methods for recording haptic data for use with multi-media data

Patent metadata

Field	Value
Publication number	US-9812165-B2
Application number	US-201414576589-A
Country	US
Kind code	B2
Filing date	Dec 19, 2014
Priority date	Dec 19, 2014
Publication date	Nov 7, 2017
Grant date	Nov 7, 2017

Abstract

A system includes a recorder configured to record audio and/or video of a subject of interest and output a recording of the subject of interest and a non-contact sensor associated with the recorder. The non-contact sensor is constructed and arranged to measure movement and/or vibration of the subject of interest from substantially the same perspective and at the same time as the recorder. The system includes a controller configured to transform the measured movement and/or vibration of the subject of interest measured by the non-contact sensor into a tactile data stream for sending to a haptic display device for playback with the recording of the subject of interest by the recorder and providing haptic effects corresponding to the measured movement and/or vibration to a user of the haptic display device in synchronization with the recording.

First claim

What is claimed is:

1. A system comprising:
a controller configured to control a first servomechanism and a second servomechanism;
a recorder configured to move by the first servomechanism, record an audio or a video of a subject of interest from a point-of-view of the recorder, and output a recording of the subject of interest; and
a non-contact sensor associated with the recorder, the non-contact sensor configured to move by the second servomechanism to maintain a same point-of-view as the point-of-view of the recorder and measure a movement or a vibration of the subject of interest from the same point-of-view as the point-of-view of the recorder and at a same time as the recorder recording the audio or the video,
wherein the controller is further configured to transform the movement or the vibration of the subject of interest measured by the non-contact sensor into a tactile data stream for sending to a haptic display device for playback with the recording of the subject of interest by the recorder and providing haptic effects corresponding to the movement or the vibration to the haptic display device in synchronization with the recording.

2. The system according to claim 1 , wherein the non-contact sensor comprises a laser vibrometer.

3. The system according to claim 1 , wherein the non-contact sensor comprises a tracking device.

4. The system according to claim 1 , wherein the non-contact sensor comprises video stream analysis algorithms.

5. The system according to claim 1 , wherein the recorder and the non-contact sensor are co-located on a same device.

6. The system according to claim 1 , wherein the second servomechanism is configured to move the non-contact sensor to a position or an orientation having the same point-of-view as the point-of-view of the recorder.

7. The system according to claim 6 , wherein the first servomechanism is synchronized with the second servomechanism so that a movement of the non-contact sensor is synchronized with a movement of the recorder.

8. The system according to claim 1 , further comprising a second sensor configured to be in contact with the subject of interest being recorded by the recorder and to sense a second movement or a second vibration of the subject of interest, wherein the controller is further configured to combine data received from the non-contact sensor and the second sensor to obtain combined data and transform the combined data into the tactile data stream.

9. The system according to claim 8 , wherein the controller comprises a transformer configured to mix sensor signals having different sampling rates or sensitivity to obtain the combined data and output the combined data.

10. A method comprising:
recording an audio or a video of a subject of interest with a recorder from a point-of-view of the recorder, wherein the recorder is configured to move by a first servomechanism that is controlled by a controller;
measuring a movement or a vibration of the subject of interest with a non-contact sensor associated with the recorder and having a same point-of-view as the point-of-view of the recorder and at a same time as the recording, wherein the non-contact sensor is configured to move by a second servomechanism that is controlled by the controller;
moving the non-contact sensor by the second servomechanism to maintain the same point-of-view as the point-of-view of the recorder; and
transforming, with the controller, the movement or the vibration into a tactile data stream for sending to a haptic display device for playback with the audio or the video of the subject of interest and providing haptic effects corresponding to the movement or the vibration to the haptic display device in synchronization with the audio or the video.

11. The method according to claim 10 , wherein the moving the non-contact sensor, with the second servomechanism controlled by the controller, includes moving the non-contact sensor to a position having the same point-of-view as the point-of-view of the recorder as the recorder moves and changes its point-of-view.

12. The method according to claim 11 , further comprising:
moving the recorder,
wherein the first servomechanism is synchronized with the second servomechanism so that a movement of the non-contact sensor is synchronized with a movement of the recorder.

13. The method according to claim 10 , further comprising:
sensing a second movement or a second vibration of the subject of interest with a second sensor in contact with the subject of interest;
combining data received from the non-contact sensor and the second sensor to obtain combined data; and
transforming the combined data into the tactile data stream.

14. The method according to claim 13 , further comprising:
transforming sensor signals having different sampling rates or sensitivity to obtain the combined data; and
outputting the combined data.

15. The method according to claim 10 , wherein the recorder and the non-contact sensor are co-located on a same device.

16. A non-transitory computer readable medium having instructions stored thereon that, when executed by a processor, cause the processor to:
record, by a recorder, an audio or a video of a subject of interest from a point-of-view and output a recording of the subject of interest, wherein the recorder is configured to move by a first servomechanism that is controlled by a controller;
measure, by a non-contact sensor associated with the recorder, a movement or a vibration of the subject of interest from a same point-of-view as the point-of-view of the recorder and at a same time as the recorder recording the audio or the video, wherein the non-contact sensor is configured to move by a second servomechanism that is controlled by the controller;
move the non-contact sensor by the second servomechanism to maintain the same point-of-view as the point-of-view of the recorder; and
transform, by the controller, the movement or the vibration of the subject of interest measured by the non-contact sensor into a tactile data stream for sending to a haptic display device for playback with the recording of the subject of interest by the recorder and providing haptic effects corresponding to the movement or the vibration to the haptic display device in synchronization with the recording.

17. The computer readable medium according to claim 16 , wherein the recorder and the non-contact sensor are co-located on a same device.

18. The computer readable medium according to claim 16 , wherein the non-contact sensor is moved, by the second servomechanism, to a position or an orientation having the same point-of-view as the point-of-view of the recorder.

19. The computer readable medium according to claim 18 ,
wherein the first servomechanism is synchronized with the second servomechanism so that a movement of the non-contact sensor is synchronized with a movement of the recorder.

20. The computer readable medium according to claim 16 , wherein the instructions, when executed by the processor, further cause the processor to:
sense a second movement or a second vibration of the subject of interest by a second sensor configured to be in contact with the subject of interest being recorded by the recorder;
combine data received from the non-contact sensor and the second sensor, by the controller, to obtain combined data; and
transform the combined data, by the controller, into the tactile data stream.

Assignees

• Immersion Corp

Inventors

• Ullrich Christopher J
• Lacroix Robert

Classifications

• G06F3/016

  Input arrangements with force or tactile feedback as computer generated output to the user · CPC title

• H04N21/42201

  biosensors, e.g. heat sensor for presence detection, EEG sensors or any limb activity sensors worn by the user (input arrangements for interaction with the human body based on nervous system activity detection G06F3/015) · CPC title

• G11B27/10

  Indexing; Addressing; Timing or synchronising; Measuring tape travel · CPC title

• G06F3/03

  Arrangements for converting the position or the displacement of a member into a coded form · CPC title

• G11B20/10527 Primary

  Audio or video recording; Data buffering arrangements (G11B20/12 - G11B20/18 take precedence) · CPC title