Method and apparatus for extracting highlight of sporting event

Patent metadata

Field	Value
Publication number	US-2020110941-A1
Application number	US-201916536411-A
Country	US
Kind code	A1
Filing date	Aug 9, 2019
Priority date	Oct 4, 2018
Publication date	Apr 9, 2020
Grant date	—

Abstract

A method for highlight extraction capable of automatically extracting a highlight from a video including a sporting event is provided. The method for highlight extraction may include: identifying a video including a sporting event, log information that sequentially records events occurring in the sporting event, and a keyword related to the video; tagging the video with game information related to the video; extracting at least one piece of log information corresponding to the keyword and determining at least one frame that corresponds to the log information extracted from the tagged video; and creating a highlight video by combining the at least one determined frame.

First claim

What is claimed is:

1 . A method for highlight extraction, comprising:
identifying a video including a sporting event, log information that sequentially records events occurring in the sporting event, and a keyword related to the video; tagging the video with game information related to the video; extracting at least one piece of log information corresponding to the keyword and determining at least one frame that corresponds to the log information extracted from the tagged video; and creating a highlight video by combining the at least one determined frame.

2 . The method of claim 1 , wherein the creating of the highlight video comprises, when there is a single keyword, creating one highlight video corresponding to the single keyword, and when there are a plurality of keywords, creating the highlight video by combining at least one highlight video corresponding to the plurality of keywords.

3 . The method of claim 1 , wherein the tagging of the video comprises, when a scoreboard is displayed in the video, recognizing game information shown in the scoreboard and tagging the game information to a corresponding frame of the video, and, when a scoreboard is not displayed in the video, tagging a corresponding frame of the video with information indicating that there is no game information that corresponds to the scoreboard, and the game information shown in the scoreboard comprises score, ball count, and inning.

4 . The method of claim 3 , further comprising extracting a characteristic from the scoreboard displayed in a predetermined area of the video and classifying the extracted characteristic using data learned in advance.

5 . The method of claim 3 , wherein when the game information shown in the scoreboard further comprises runner status, the runner status is tagged to a corresponding frame of the video and when the game information shown in the scoreboard further comprises a pitch count, the pitch count is tagged to a corresponding frame of the video.

6 . The method of claim 1 , wherein the tagging of the video comprises tagging scene change information to a corresponding frame of the video by detecting scene change, and tagging scene recognition information to a corresponding frame of the video by performing scene recognition according to predetermined criteria.

7 . The method of claim 6 , wherein the scene change is detected by converting red-green-blue (RGB) into hue-saturation-value (HSV) and using a histogram difference of the HSV, wherein the histogram difference between a previous frame and a current frame for each channel of each of the HSV is determined and the scene change is determined by comparing the determined histogram difference with a predetermined reference.

8 . The method of claim 6 , wherein the scene recognition is set on the basis of a criterion matching a corresponding frame among predetermined criteria set by an apparatus for highlight extraction.

9 . The method of claim 1 , wherein the at least one frame comprises a frame from a starting frame of a target pitch count— 1 to a last frame of the target pitch count that correspond to a situation included in the keyword.

10 . The method of claim 1 , further comprising ruling out a frame that does not correspond to the keyword from frames from a starting frame of the target pitch count— 1 to a last frame of the target pitch count.

11 . An apparatus for highlight extraction, comprising:
a processor; and a memory including a computer-readable command, wherein the processor is configured to, when the command is executed in the processor, identify a video including a sporting event, log information that sequentially records events occurring in the sporting event, and a keyword related to the video, tag the video with game information related to the video, extract at least one piece of log information corresponding to the keyword and determine at least one frame that corresponds to the log information extracted from the tagged video, and create a highlight video by combining the at least one determined frame.

12 . The apparatus of claim 11 , wherein the processor is further configured to, when there is a single keyword, create one highlight video corresponding to the single keyword, and, when there are a plurality of keywords, create the highlight video by combining at least one highlight video corresponding to the plurality of keywords.

13 . The apparatus of claim 11 , wherein the processor is further configured to, when a scoreboard is displayed in the video, recognize game information shown in the scoreboard and tag the game information to a corresponding frame of the video, and, when a scoreboard is not displayed in the video, tag a corresponding frame of the video with information indicating that there is no game information that corresponds to the scoreboard, and the game information shown in the scoreboard comprises score, ball count, and inning.

14 . The apparatus of claim 13 , wherein the processor is further configured to extract a characteristic from the scoreboard displayed in a predetermined area of the video and classify the extracted characteristic using data learned in advance.

15 . The apparatus of claim 13 , wherein the processor is further configured to, when the game information shown in the scoreboard further comprises runner status, tag the runner status to a corresponding frame of the video, and, when the game information shown in the scoreboard further comprises a pitch count, tag the pitch count to a corresponding frame of the video.

16 . The apparatus of claim 11 , wherein the processor is further configured to tag scene change information to a corresponding frame of the video by detecting scene change and tag scene recognition information to a corresponding frame of the video by performing scene recognition according to predetermined criteria.

17 . The apparatus of claim 16 , wherein the processor is further configured to detect the scene change by converting red-green-blue (RGB) into hue-saturation-value (HSV) and using a histogram difference of the HSV, wherein the histogram difference between a previous frame and a current frame for each channel of each of the HSV is determined and the scene change is determined by comparing the determined histogram difference with a predetermined reference.

18 . The apparatus of claim 16 , wherein the scene recognition is set on the basis of a criterion matching a corresponding frame among predetermined criteria set by the apparatus for highlight extraction.

19 . The apparatus of claim 11 , wherein the at least one frame comprises a frame from a starting frame of a target pitch count— 1 to a last frame of the target pitch count that correspond to a situation included in the keyword.

20 . The apparatus of claim 19 , wherein the processor is further configured to rule out a frame that does not correspond to the keyword from frames from a starting frame of the target pitch count— 1 to a last frame of the target pitch count.

Assignees

• Ncsoft Corp

Inventors

• Lee Young Hyun
• Jung Hyun Jo

Classifications

• H04N21/23418

  involving operations for analysing video streams, e.g. detecting features or characteristics (television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59) · CPC title

• H04N21/8405

  represented by keywords · CPC title

• H04N21/8549 Primary

  Creating video summaries, e.g. movie trailer {(retrieval in video databases by using presentations in form of a video summary G06F16/739)} · CPC title

• H04N21/8456

  by decomposing the content in the time domain, e.g. in time segments · CPC title

• G11B27/22

  Means responsive to presence or absence of recorded information signals · CPC title