Methods and systems for scoreboard text region detection

Patent metadata

Field	Value
Publication number	US-12309439-B2
Application number	US-202418654287-A
Country	US
Kind code	B2
Filing date	May 3, 2024
Priority date	Jan 25, 2019
Publication date	May 20, 2025
Grant date	May 20, 2025

Abstract

A computing system automatically detects, within a digital video frame, a video frame region that depicts a textual expression of a scoreboard. The computing system (a) engages in an edge-detection process to detect edges of at least scoreboard image elements depicted by the digital video frame, with at least some of these edges being of the textual expression and defining alphanumeric shapes; (b) applies pattern-recognition to identify the alphanumeric shapes; (c) establishes a plurality of minimum bounding rectangles each bounding a respective one of the identified alphanumeric shapes; (d) establishes, based on at least two of the minimum bounding rectangles, a composite shape that encompasses the identified alphanumeric shapes that were bounded by the at least two minimum bounding rectangles; and (e) based on the composite shape occupying a particular region, deems the particular region to be the video frame region that depicts the textual expression.

First claim

What is claimed is:

1. A method for automatically detecting, within a digital video frame, a video frame region that depicts a particular expression of a scoreboard, the method comprising:
receiving, by a computing system, the digital video frame, wherein the digital video frame depicts scoreboard image elements, and wherein the computing system comprises a processing unit;
engaging, by the computing system, in an edge-detection process to detect edges of at least the depicted scoreboard image elements in the received digital video frame;
applying pattern-recognition, by the computing system, to identify the depicted scoreboard image elements, wherein applying the pattern-recognition comprises identifying one or more edges, of the detected edges, based on the one or more identified edges enclosing an area within the digital video frame;
establishing, by the computing system, a plurality of bounding rectangles;
establishing in the digital video frame, by the computing system, based on at least two of the bounding rectangles, a composite shape that encompasses the identified scoreboard image elements that were bounded by the at least two bounding rectangles; and
based on the established composite shape occupying a particular region of the digital video frame, deeming, by the computing system, the particular region to be the video frame region that depicts the particular expression of the scoreboard.

2. The method of claim 1 , carried out while receiving a real-time broadcast feed, wherein receiving the digital video frame comprises receiving the digital video frame as part of the real-time broadcast feed.

3. The method of claim 1 , wherein engaging in the edge-detection process comprises engaging in the edge-detection process in a predefined frame region within the digital video frame.

4. The method of claim 1 , wherein identifying one or more edges, of the detected edges, based on the one or more identified edges enclosing an area within the digital video frame comprises applying a contouring procedure that traces one or more of the detected edges to identify the one or more edges that enclose the area.

5. The method of claim 4 , wherein the edge-detection process generates a binary representation of the detected edges, and wherein applying the contouring procedure comprises applying the contouring procedure to the binary representation of the detected edges.

6. The method of claim 1 , wherein applying pattern-recognition to identify the alphanumeric shapes, wherein applying the pattern-recognition further comprises determining that the enclosed area has the alphanumeric characteristic, wherein determining that the enclosed area has the alphanumeric characteristic comprises:
determining an aspect ratio of the enclosed area, and
determining that the determined aspect ratio is indicative of the enclosed area having the alphanumeric characteristic.

7. The method of claim 1 , wherein the at least two bounding rectangles comprise first and second bounding rectangles, and wherein establishing the composite shape comprises:
extending (i) the first bounding rectangle by a first extent and (ii) the second bounding rectangle by a second extent, and
wherein the extending merges together the extended first bounding rectangle and the extended second bounding rectangle into the composite shape.

8. The method of claim 7 , wherein the first extent is the same as the second extent.

9. The method of claim 7 , wherein the digital video frame defines a horizontal axis and a vertical axis, wherein the digital video frame depicts the scoreboard image elements substantially along the horizontal axis of the digital video frame, and wherein the extending comprises extending (i) the first bounding rectangle substantially along the horizontal axis and (ii) the second bounding rectangle substantially along the horizontal axis.

10. The method of claim 7 ,
wherein establishing the plurality of bounding rectangles comprises generating a binary representation that represents (i) the first bounding rectangle as spanning a first set of pixels of the digital video frame and (ii) the second bounding rectangle as spanning a second set of pixels of the digital video frame, and
wherein the extending comprises updating the binary representation to represent (i) the extended first bounding rectangle as spanning the pixels of the first set in combination with a predetermined number of additional first pixels of the digital video frame and (ii) the extended second bounding rectangle as spanning the pixels of the second set in combination with a predetermined number of additional second pixels of the digital video frame.

11. The method of claim 10 ,
wherein the additional first pixels are substantially alongside the pixels of the first set in the digital video frame, and
wherein the additional second pixels are substantially alongside the pixels of the second set in the digital video frame.

12. The method of claim 10 , wherein the updated binary representation represents the composite shape as spanning a combined set of pixels that includes (i) the pixels of the first set, (ii) the pixels of the second set, (iii) the additional first pixels, and (iv) the additional second pixels, the method further comprising:
making a determination, by the computing system, that the established composite shape occupies the particular region, wherein making the determination comprises (i) applying a connected-component-labeling procedure to detect, in the updated binary representation, the pixels of the combined set that correspond to the composite shape and (ii) deeming the detected pixels to be representative of the particular region of the digital video frame.

13. The method of claim 1 , further comprising:
carrying out, by the computing system, an action based on the detecting of the video frame region that depicts the particular expression of the scoreboard, wherein carrying out the action comprises one or more of the following:
(i) recording the detected video frame region,
(ii) outputting a report related to the detected video frame region, and
(iii) recognizing the particular expression in the detected video frame region.

14. A computing system comprising:
a processing unit;
non-transitory data storage; and
program instructions stored in the non-transitory data storage and executable by the processing unit to carry out operations for automatically detecting, within a digital video frame, a video frame region that depicts a particular expression of a scoreboard, the operations comprising:
receiving the digital video frame, wherein the digital video frame depicts scoreboard image elements,
engaging in an edge-detection process to detect edges of at least the depicted scoreboard image elements in the received digital video frame,
applying pattern-recognition to identify the depicted scoreboard image elements, wherein applying the pattern-recognition comprises identifying one or more edges, of the detected edges, based on the one or more identified edges enclosing an area within the digital video frame,
establishing a plurality of bounding rectangles,
establishing in the digital video frame, based on at least two of the bounding rectangles, a composite shape that encompasses the identified scoreboard image elements that were bounded by the at least two bounding rectangles, and
based on the established composite shape occupying a particular region of the digital video frame, deeming the particular region to be the video frame region that depicts the particular expression of the scoreboard.

15. The computing system of claim 14 , wherein the

Assignees

• Gracenote Inc

Inventors

• Scott Jeffrey
• Cremer Markus Kurt Peter
• Parekh Nishit Umesh
• Lee Dewey Ho

Classifications

• G06V10/44

  Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components · CPC title

• G06V20/42

  of sport video content · CPC title

• G06V20/635

  Overlay text, e.g. embedded captions in a TV programme · CPC title

• G06T7/13

  Edge detection · CPC title

• G06T2207/30221

  Sports video; Sports image · CPC title