Attention estimation to control the delivery of data and audio/video content

Patent metadata

Field	Value
Publication number	US-9729920-B2
Application number	US-201313840342-A
Country	US
Kind code	B2
Filing date	Mar 15, 2013
Priority date	Mar 15, 2013
Publication date	Aug 8, 2017
Grant date	Aug 8, 2017

Abstract

A method implemented in a computer system for controlling the delivery of data and audio/video content. The method delivers primary content to the subscriber device for viewing by a subscriber. The method also delivers secondary content to the companion device for viewing by the subscriber in parallel with the subscriber viewing the primary content, where the secondary content relates to the primary content. The method extracts attention estimation features from the primary content, and monitors the companion device to determine an interaction measurement for the subscriber viewing the secondary content on the companion device. The method calculates an attention measurement for the subscriber viewing the primary content based on the attention estimation features, and the interaction measurement, and controls the delivery of the secondary content to the companion device based on the attention measurement.

First claim

We claim:

1. A computing device for controlling the delivery of data and audio/video content, comprising:
a communication interface that connects the computing device to a network that connects to a subscriber device, and a companion device;
a memory device resident in the computing device; and
a processor disposed in communication with the communication interface and the memory device, the processor configured to:
deliver primary content to the subscriber device for viewing by a subscriber;
deliver secondary content to the companion device for viewing by the subscriber in parallel with the subscriber viewing the primary content, the secondary content relating to the primary content;
extract at least one attention estimation feature from the primary content;
monitor the companion device to determine an interaction measurement for the subscriber viewing the secondary content on the companion device;
calculate an attention measurement for the subscriber viewing the primary content based on said at least one attention estimation feature, and the interaction measurement; and
control the delivery of the secondary content to the companion device based on the attention measurement.

2. The computing device of claim 1 , wherein each of said at least one attention estimation feature is at least one of a visual feature in the primary content, an audio feature in the primary content, and a textual feature in the primary content.

3. The computing device of claim 1 , wherein to extract said at least one attention estimation feature from the primary content, the processor is further configured to:
retrieve a personal profile and viewing preferences for the subscriber;
compute a weight for each of said at least one attention estimation feature based on the personal profile and viewing preferences for the subscriber,
wherein the calculation of the attention measurement for the subscriber includes the weight for each said at least one attention estimation feature.

4. The computing device of claim 1 , wherein to extract said at least one attention estimation feature from the primary content, the processor is further configured to:
retrieve a past viewing history for the subscriber;
compute a weight for each of said at least one attention estimation feature based on the past viewing history for the subscriber,
wherein the calculation of the attention measurement for the subscriber includes the weight for each of said at least one attention estimation feature.

5. The computing device of claim 1 , wherein to monitor the companion device, the processor is further configured to:
receive a measurement from the companion device,
wherein the measurement is at least one of an ability of the subscriber to interact with the companion device, and an operation of the companion device by the subscriber.

6. The computing device of claim 1 , wherein to calculate the attention measurement, the processor is further configured to:
calculate a weight for each of said at least one attention estimation feature; and
adjust the weight for each of said at least one attention estimation feature based on the interaction measurement.

7. The computing device of claim 1 , wherein to calculate the attention measurement, the processor is further configured to:
calculate a weight for each of said at least one attention estimation feature; and
calculate a weight for the interaction measurement,
wherein the attention measurement is a fusion of the weighted said at least one attention estimation feature, and the weighted interaction measurement.

8. The computing device of claim 1 , wherein the attention measurement is based on said at least one attention estimation feature, and the interaction measurement as a function of time.

9. The computing device of claim 1 , wherein to control the delivery of the secondary content, the processor is further configured to:
adjust presentation or pacing of delivery of the secondary content to the companion device based on the attention measurement.

10. The computing device of claim 1 , wherein the processor is further configured to:
control the delivery of the primary content to the subscriber device based on the attention measurement.

11. A method implemented in a computer system for controlling the delivery of data and audio/video content, comprising:
delivering primary content to a subscriber device for viewing by a subscriber;
delivering secondary content to a companion device for viewing by the subscriber in parallel with the subscriber viewing the primary content, the secondary content relating to the primary content;
extracting at least one attention estimation feature from the primary content;
monitoring the companion device to determine an interaction measurement for the subscriber viewing the secondary content on the companion device;
calculating an attention measurement for the subscriber viewing the primary content based on said at least one attention estimation feature, and the interaction measurement; and
controlling the delivery of the secondary content to the companion device based on the attention measurement.

12. The method of claim 11 , wherein each of said at least one attention estimation feature is at least one of a visual feature in the primary content, an audio feature in the primary content, and a textual feature in the primary content.

13. The method of claim 11 , wherein the extracting of said at least one attention estimation feature from the primary content further comprises:
retrieving a personal profile and viewing preferences for the subscriber;
computing a weight for each of said at least one attention estimation feature based on the personal profile and viewing preferences for the subscriber,
wherein the calculation of the attention measurement for the subscriber includes the weight for each of said at least one attention estimation feature.

14. The method of claim 11 , wherein the extracting of said at least one attention estimation feature from the primary content further comprises:
retrieving a past viewing history for the subscriber;
computing a weight for each of said at least one attention estimation feature based on the past viewing history for the subscriber,
wherein the calculation of the attention measurement for the subscriber includes the weight for each of said at least one attention estimation feature.

15. The method of claim 11 , wherein the monitoring of the companion device further comprises:
receiving a measurement from the companion device,
wherein the measurement is at least one of an ability of the subscriber to interact with the companion device, and an operation of the companion device by the subscriber.

16. The method of claim 11 , wherein the calculating of the attention measurement further comprises:
calculating a weight for each of said at least one attention estimation feature; and
adjusting the weight for each of said at least one attention estimation feature based on the interaction measurement.

17. The method of claim 11 , wherein the calculating of the attention measurement further comprises:
calculating a weight for each of said at least one attention estimation feature; and
calculating a weight for the interaction measurement,
wherein the attention measurement is a fusion of the weighted said at least one attention estimation feature, and the weighted interaction measurement.

18. The method of claim 11 , wherein the attention measurement is based on said at least one attention estimation feature, and the interaction measurement as a func

Assignees

• Gen Instrument Corp
• Arris Entpr Inc

Inventors

• Needham Michael L
• Baum Kevin L
• Ishtiaq Faisal
• Li Renxiang
• Mohapatra Shivajit

Classifications

• H04N21/4307

  Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen · CPC title

• H04N21/262

  Content or additional data distribution scheduling, e.g. sending additional data at off-peak times, updating software modules, calculating the carousel transmission frequency, delaying a video stream transmission, generating play-lists {(scheduling strategies for dispatcher in multiprogramming arrangements G06F9/4881; arrangements for scheduling broadcast services or broadcast-related services H04H60/06; flow control in packet networks H04L47/10; establishing a time schedule or organising the servicing of application requests H04L67/62)} · CPC title

• H04N21/44218 Primary

  Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV programme (methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45) · CPC title

• H04N21/6543

  for forcing some client operations, e.g. recording {(remote booting in general G06F9/4416)} · CPC title

• H04N21/44213

  Monitoring of end-user related data (arrangements for monitoring the users' behaviour or opinions in broadcast systems H04H60/33) · CPC title