Haptic effect generation for space-dependent content

Patent metadata

Field	Value
Publication number	US-10147460-B2
Application number	US-201615392744-A
Country	US
Kind code	B2
Filing date	Dec 28, 2016
Priority date	Dec 28, 2016
Publication date	Dec 4, 2018
Grant date	Dec 4, 2018

Abstract

Systems and methods for authoring and encoding haptic effects are provided for space-dependent content, such as 360-degree videos, three-dimensional videos, or virtual or augmented reality contents. The systems and methods can generate one or more haptic layers for encoding or modifying haptic effects for the content.

First claim

What is claimed is:

1. A method of authoring and rendering haptic effects, the method comprising: receiving a haptic track and a video texture map, the haptic track having haptic data, the haptic data defining operation of a haptic actuator, the video texture map representative of at least one frame of a video, the haptic track associated with the at least one frame of the video, and the video texture map having a plurality of video pixels; and generating haptic data modifiers associated with haptic pixels, each haptic pixel associated with at least one of the video pixels, the haptic data modifiers for modifying at least a portion of the haptic data and modifying operation of the haptic actuator; controlling the haptic actuator based on the haptic track and the haptic pixels; and generating, by the haptic actuator, the haptic effects for a user.

2. The method of claim 1 , further comprising:
generating a haptic texture map including the haptic pixels and the haptic data modifiers associated with the haptic pixels.

3. The method of claim 2 , wherein the haptic texture map has a different spatial resolution than a spatial resolution of the video texture map.

4. The method of claim 2 , where the haptic texture map has a different temporal resolution than a temporal resolution of the video texture map.

5. The method of claim 2 , further comprising:
displaying the haptic texture map;
receiving a user input of at least one haptic effect parameter; and
displaying a modified haptic texture map representative of the user input of the at least one haptic effect parameter.

6. The method of claim 2 , further comprising:
displaying the haptic texture map on a display screen in a computing device.

7. The method of claim 6 , wherein the haptic texture map includes a monochromatic image representative of the haptic data modifiers.

8. The method of claim 7 , wherein at least some of the haptic pixels of the monochromatic image have different values representative of different haptic data modifiers.

9. The method of claim 1 , wherein the haptic data modifiers include attenuation data for adjusting magnitudes of haptic effects defined by the haptic data in the haptic track.

10. The method of claim 1 , wherein the haptic data modifiers include attenuation data for adjusting at least one of frequency, waveform, sharpness, and actuator selection of haptic effects defined by the haptic data in the haptic track.

11. The method of claim 1 , wherein the video is selected from the group consisting of a 360-degree video, a virtual reality environment, a three-dimensional video, a three-dimensional video game, a three-dimensional modeling environment, and combinations thereof.

12. The method of claim 1 , wherein the video includes a plurality of frames, and wherein the steps of receiving the haptic track and the video texture map and generating the haptic data modifiers are repeated for at least some of a plurality of frames in the video.

13. The method of claim 1 , wherein each of the haptic pixels is associated with two or more of the video pixels.

14. The method of claim 1 , wherein two or more of the haptic pixels are associated with each of the video pixels.

15. The method of claim 1 , wherein receiving a haptic track and a video texture map includes:
receiving a plurality of haptic tracks and a video texture map, each haptic track having haptic data, the video texture map representative of a frame of a video, the plurality of haptic tracks associated with the frame of the video, and the video texture map having a plurality of video pixels;
wherein the haptic data modifiers are configured to modify at least a portion of the haptic data of the plurality of haptic tracks.

16. The method of claim 1 , further comprising:
modifying at least a portion of the haptic data based on the haptic data modifiers.

17. A system for authoring and rendering haptic effects, the system comprising: a haptic effect authoring device, coupled to a network, including a processor configured to: receive a haptic track and a video texture map, the haptic track having haptic data, the haptic data defining operation of a haptic actuator, the video texture map representative of at least one frame of a video, the haptic track associated with the frame of the video, and the video texture map having a plurality of video pixels, and generate haptic data modifiers associated with haptic pixels, each haptic pixel associated with at least one of the video pixels, the haptic data modifiers for modifying at least a portion of the haptic data and modifying operation of the haptic actuator; and a haptic enabled device, coupled to the network, including: the haptic actuator, and a processor configured to control the haptic actuator based on the haptic track and the haptic pixels, wherein the haptic actuator generates the haptic effects for a user.

18. The system of claim 17 , wherein the haptic effect authoring device includes: an input device configured to receive a user input; and a display device; wherein the software instructions further cause the apparatus to display a haptic texture map using the display device, the haptic texture map including the haptic pixels and the haptic data modifiers associated with the haptic pixels.

19. The system of claim 18 , wherein the haptic texture map has a different spatial resolution than a spatial resolution of the video texture map.

20. The system of claim 18 , where the haptic texture map has a different temporal resolution than a temporal resolution of the video texture map.

21. The system of claim 18 , wherein the processor of the haptic effect authoring device is further configured to: receive a user input of at least one haptic effect parameter via the input device; and display a modified haptic texture map via the display device, the modified haptic texture map representative of the user input of the at least one haptic effect parameter.

22. The system of claim 18 , wherein the haptic texture map includes a monochromatic image representative of the haptic data modifiers.

23. The system of claim 22 , wherein at least some of the haptic pixels of the image have different values representative of different haptic data modifiers.

24. The system of claim 17 , wherein the haptic data modifiers include attenuation data for adjusting magnitudes of haptic effects defined by the haptic data in the haptic track.

25. The system of claim 17 , wherein the haptic data modifiers include attenuation data for adjusting at least one of frequency, waveform, sharpness, and actuator selection of haptic effects defined by the haptic data in the haptic track.

26. The system of claim 17 , wherein the video is selected from the group consisting of a 360-degree video, a virtual reality environment, a three-dimensional video, a three-dimensional video game, a three-dimensional modeling environment, and combinations thereof.

27. The system of claim 17 , wherein the video includes a plurality of frames, and wherein the steps of receiving the haptic track and the video texture map and generating the haptic data modifiers are repeated for at least some of a plurality of frames in the video.

28. The system of claim 17 , wherein each of the haptic pixels is associated with two or more of the video pixels.

29. The system of claim 17 , wherein two or more of the haptic pixels are associate

Assignees

• Immersion Corp

Inventors

• Ullrich Christopher

Classifications

• G06F3/016 Primary

  Input arrangements with force or tactile feedback as computer generated output to the user · CPC title

• G11B27/34 Primary

  Indicating arrangements  {(indicating means incorporated in magazine or cassette G11B23/046 and G11B23/0875; indicating measured values in general G01D)} · CPC title

• H04N21/816

  involving special video data, e.g 3D video · CPC title

• G11B27/031

  Electronic editing of digitised analogue information signals, e.g. audio or video signals · CPC title

• H04N21/8547

  involving timestamps for synchronizing content · CPC title