Enhanced experience from standard program content

Patent metadata

Field	Value
Publication number	US-9532115-B2
Application number	US-201514628098-A
Country	US
Kind code	B2
Filing date	Feb 20, 2015
Priority date	Mar 13, 2013
Publication date	Dec 27, 2016
Grant date	Dec 27, 2016

Abstract

Methods and apparatus are disclosed for enhancing the viewing experience of video content by analyzing the content to determine where enhanced sensory experience events may be appropriate, by identifying devices at the viewing location and devices personal to the viewer that can be controlled to provide an enhanced sensory experience, and by activating those devices in a way that is synchronized with the presentation of the content.

First claim

The invention claimed is:

1. A method comprising:
receiving a single audio/video content stream from a content provider;
analyzing the audio/video content stream prior to being presented to a viewer on a display device for events to be presented in the content stream, the display device being within a viewing environment;
determining, based on the analyzed audio/video content stream, an enhanced sensory experience for selected events in the content stream;
synchronizing the enhanced sensory experience with the content stream;
transmitting a query command within the viewing environment to identify enhanced sensory experience devices in the viewing environment that may be used to provide the enhanced sensory experience to the viewer while viewing the content stream, the enhanced sensory experience devices being separate from the display device;
querying a viewing location devices database to identify addresses, activation commands and enhanced sensory experience capabilities for each identified enhanced sensory experience device in the viewing environment;
displaying the video from the received content stream on the display device;
outputting audio from the received content stream on a speaker device; and
activating, based on the analyzed audio/video content stream and the identified activation commands, one or more of the enhanced sensory experience devices in synchronization with the content stream as the content stream is being displayed on the display device to provide an enhanced viewing experience.

2. A method of claim 1 wherein analyzing an audio/video content stream further comprises:
analyzing metadata within the content stream.

3. A method of claim 1 wherein analyzing an audio/video content stream further comprises:
analyzing closed caption and other text within the content stream.

4. A method of claim 1 wherein analyzing an audio/video content stream further comprises:
analyzing the video images within the content stream.

5. A method of claim 1 wherein analyzing an audio/video content stream further comprises:
analyzing audio within the content stream.

6. A method of claim 1 , wherein analyzing an audio/video content stream further comprises:
displaying the content stream on the display device; and
analyzing the content stream substantially simultaneously with the displaying of the content stream.

7. A method of claim 1 , wherein analyzing an audio/video content stream further comprises:
analyzing the content stream before displaying the content stream on the display device; and
displaying the content stream on the display device.

8. A method of claim 1 , wherein identifying enhanced sensory experience devices in the viewing environment further comprises:
identifying enhanced sensory experience devices that are at the viewing location; and
identifying enhanced sensory experience devices that are personal to the viewer.

9. A method of claim 8 , wherein activating one or more of the enhanced sensory experience devices in synchronization with the content stream as the viewer is viewing the content stream further comprises:
matching the synchronized enhanced sensory experience with the content stream to the enhanced sensory experience capability of each identified enhanced sensory experience device;
issuing commands to the enhanced sensory experience device; and
causing the enhanced sensory experience device to provide an enhanced sensory experience.

10. The method of claim 1 , wherein the identified enhanced sensory experience devices includes at least one of: a speaker, a heating/cooling/fan system, a home appliance, a home security device, a doorbell, a phone system, a lighting system, seat buzzers, special effects equipment, a music player, a blower, a wind machine, a fog machine, a strobe light, a dimmable light, a sound effect machine, a bubble machine, a haze machine, an aroma machine, a cryo machine, a vibrator, a tactile feedback device, a robot device, a haptic feedback device, a key fob, viewing glasses, a hat and headphones.

11. A viewing enhancement system comprising:
an input configured to receive a single video/audio content stream;
an analysis module configured to receive the content stream, to analyze the content stream, to determine based on the content stream enhanced sensory experiences for selected events in the content stream, and to output the enhanced sensory experiences;
a device database identifying devices, locations, addresses, activation commands and capabilities of enhanced sensory experience devices that are controllable at a content viewing location, the contents of the device database being determined at least in part by querying the viewing location to identify the enhanced sensory experience devices and their respective associated enhanced sensory experience capabilities that may be used to provide the enhanced sensory experience; and
a processing module configured to
receive the output containing enhanced sensory experiences from the analysis module,
analyze based on the content stream the received enhanced sensory experiences to create integrated enhanced sensory experiences for selected events in the content stream;
synchronize the integrated enhanced sensory experiences with the content stream to create content-synchronized integrated enhanced sensory experiences,
query the device database to identify the devices, locations, addresses, activation commands and capabilities of the enhanced sensory experience controllable devices in the viewing location,
determine, based on the content-synchronized integrated enhanced sensory experiences and the identified devices, locations, addresses, activation commands and capabilities of the enhanced sensory experience controllable devices in the viewing location, the commands to be sent to the enhanced sensory experience controllable devices that will provide an enhanced viewing experience while the viewer views the content on a display device in the viewing location, the display device being separate from the enhanced sensory experience devices, and
send commands to the enhanced sensory experience devices such that the operation of the enhanced sensory experience controllable devices is synchronized with the content stream to provide an enhanced viewing experience.

12. A system of claim 11 , wherein the processing module is further configured to:
determine the time delay between a command to an enhanced sensory experience controllable device and desired sensory experience from the command to that device;
delay the content presentation to a display device for at least as long as the determined time delay; and
look ahead in the content at least as far as the determined time delay so that the sent command will cause the desired sensory experience from the device to be synchronized with the enhanced sensory experience identified in the content.

13. A system of claim 11 , wherein the analysis module comprises:
a text analysis module configured to receive the content stream, to decode and analyze the text from closed-captioning, on-screen text, titles, and other text contained within the content stream to determine the enhanced sensory experiences for selected events in the content stream, and to output the enhanced sensory experiences.

14. A system of claim 11 , wherein the analysis module comprises:
a video analysis module configured to receive the content stream, to analyze the video content portion within the content stream to determine the enhanced sensory experiences for selected events in the content stream, and to output the enhanced sensory experiences.

15. A system of clai

Assignees

• Echostar Technologies Llc

Inventors

• Mickelsen Jeremy
• Schafer Adam
• Wolf Bradley

Classifications

• H04N21/4126

  The peripheral being portable, e.g. PDAs or mobile phones · CPC title

• H04N21/233

  Processing of audio elementary streams {(monitoring, identification or recognition of audio in broadcast systems H04H60/58)} · CPC title

• H04N21/8545 Primary

  for generating interactive applications · CPC title

• H04N21/4307

  Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen · CPC title

• H04N21/42202

  environmental sensors, e.g. for detecting temperature, luminosity, pressure, earthquakes · CPC title