Automated closed captioning using temporal data

Patent metadata

Field	Value
Publication number	US-2016357746-A1
Application number	US-201514728201-A
Country	US
Kind code	A1
Filing date	Jun 2, 2015
Priority date	Jun 2, 2015
Publication date	Dec 8, 2016
Grant date	—

Abstract

One or more systems and/or techniques are provided for automatic closed captioning for media content. In an example, real-time content, occurring within a threshold timespan of a broadcast of media content (e.g., social network posts occurring during and an hour before a live broadcast of an interview), may be accessed. A list of named entities, occurring within the social network data, may be generated (e.g., Interviewer Jon, Interviewee Kathy, Husband Dave, Son Jack, etc.). A ranked list of named entities may be created based upon trending named entities within the list of named entities (e.g., a named entity may be ranked higher based upon a more frequent occurrence within the social network posts). A dynamic grammar (e.g., library, etc.) may be built based upon the ranked list of named entities. Speech recognition may be performed upon the broadcast of media content utilizing the dynamic grammar to create closed caption text.

First claim

1 . A system for increasing accuracy of computer speech recognition comprising:
a dynamic grammar builder computing device comprising one or more processing units and one or more computer-readable media comprising computer-executable instructions which, when executed by the one or more processing units, cause the dynamic grammar builder computing device to: obtain social network data occurring within a threshold timespan of a broadcast of media content; identify named entities from the obtained social network data that are trending within the obtained social network data; rank the identified named entities based upon the trending; and build a dynamic grammar comprising at least some of the named entities based upon the ranking; and a speech recognition computing device comprising one or more processing units and one or more computer-readable media comprising computer-executable instructions which, when executed by the one or more processing units, cause the speech recognition computing device to: perform speech recognition of spoken words, spoken by the broadcast of the media content, utilizing the dynamic grammar to create closed caption text for the broadcast of the media content.

2 . The system of claim 1 , wherein the computer-readable media of the dynamic grammar builder computing device comprise additional computer-executable instructions which, when executed by the one or more processing units of the dynamic grammar builder computing device cause the dynamic grammar builder computing device to:
access supplemental content associated with at least one of the named entities; identify a context associated with the at least one of the named entities or with an event occurring within the media content; and build the dynamic grammar based upon the identified context.

3 . A method for increasing accuracy of computer speech recognition comprising:
obtaining, by a computing device, social network data occurring within a threshold timespan of a broadcast of media content; identifying, on the computing device, named entities from the obtained social network data that are trending within the obtained social network data; ranking, on the computing device, the identified named entities based upon the trending; building, on the computing device, a dynamic grammar comprising at least some of the named entities based upon the ranking; and performing computer speech recognition of spoken words, spoken by the broadcast of the media content, utilizing the dynamic grammar to create closed caption text for the broadcast of the media content.

4 . The method of claim 3 , further comprising:
accessing, by the computing device, supplemental content associated with at least one of the named entities; identifying, with the computing device, a context associated with either the at least one of the named entities or with an event occurring within the media content; and building the dynamic grammar based upon the identified context.

5 . The method of claim 3 , wherein the trending comprises occurrence metrics identifying a number of times a particular named entity is referenced in the obtained social network data within trending timespan thresholds.

6 . The method of claim 3 , wherein the identifying the named entities comprises:
executing entity recognition functionality upon the social network data.

7 . The method of claim 3 , wherein the ranking comprises:
responsive to a refresh timer expiring, re-ranking the named entities based upon the trending within updated social network data.

8 . (canceled)

9 . The method of claim 3 , further comprising:
defining the threshold timespan based upon a type of event occurring within the media content.

10 . The method of claim 3 , wherein the media content is one of a live interview, a live telecast, or an online videogame.

11 . The method of claim 3 , wherein the dynamic grammar is
a speech recognition grammar specification file.

12 - 13 . (canceled)

14 . The method of claim 3 , further comprising:
filtering the social network data based upon a context of an event occurring within the media content.

15 . (canceled)

16 . The method of claim 3 , the social network data comprises at least one of a query log, a message, a microblog, a forum post, or user created data that is updated in real-time.

17 . The method of claim 3 , wherein the performing the speech recognition comprises:
loading the dynamic grammar into memory of a speech recognition server; and invoking speech recognition functionality, of the speech recognition server, to utilize the dynamic grammar as a library against which an audio fragment, of the broadcast of the media content, is compared for determining a probability that the audio fragment is a named entity.

18 . The method of claim 3 , further comprising:
correcting user generated closed captioning for the broadcast of the media content based upon the dynamic grammar.

19 . The method of claim 3 , wherein the performing speech recognition comprises:
utilizing the dynamic grammar to either increase or decrease a probability that an audio fragment, of the broadcast of the media content, is a named entity.

20 . A computing device comprising:
one or more processing units; and one or more computer-readable media comprising computer-executable instructions which, when executed by the one or more processing units, cause the computing device to: obtain real-time content occurring within a threshold timespan of a broadcast of media content; identify named entities from the obtained real-time content that are trending within the obtained real-time content; ranking the identified named entities based upon the trending; building a dynamic grammar comprising at least some of the named entities based upon the ranking; and utilizing the dynamic grammar for correcting user generated closed captioning for the broadcast of the media content.

21 . The system of claim 1 , wherein the trending comprises occurrence metrics identifying a number of times a particular named entity is referenced in the obtained social network data within trending timespan thresholds.

22 . The system of claim 1 , wherein the identifying the named entities comprises executing entity recognition functionality upon the social network data.

23 . The system of claim 1 , wherein the ranking comprises
responsive to a refresh timer expiring, re-ranking the named entities based upon the trending within updated social network data.

24 . The system of claim 1 , wherein the computer-readable media of the dynamic grammar builder computing device comprise additional computer-executable instructions which, when executed by the one or more processing units of the dynamic grammar builder computing device cause the dynamic grammar builder computing device to:
define the threshold timespan based upon a type of event occurring within the media content.

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Koul Anirudh
• Kulkarni Ranjitha Gurunath
• Tremblay Serge-Eric

Classifications

• G10L15/183

  using context dependencies, e.g. language models · CPC title

• H04N21/4888

  for displaying teletext characters · CPC title

• G06F16/24578 Primary

  using ranking · CPC title

• G06F40/295

  Named entity recognition · CPC title

• G06F16/9535

  Search customisation based on user profiles and personalisation · CPC title