Focusing regions of interest using dynamic object detection for textual information retrieval

Patent metadata

Field	Value
Publication number	US-11657627-B2
Application number	US-202117365179-A
Country	US
Kind code	B2
Filing date	Jul 1, 2021
Priority date	Aug 1, 2019
Publication date	May 23, 2023
Grant date	May 23, 2023

Abstract

In various examples, frames of a video may include a first visual object that may appear relative to a second visual object within a region of the frames. Once a relationship between the first visual object and the region is known, one or more operations may be performed on the relative region. For example, optical character recognition may be performed on the relative region where the relative region is known to contain textual information. As a result, the identification of the first visual object may serve as an anchor for determining the location of the relative region including the second visual object—thereby increasing accuracy and efficiency of the system while reducing run-time.

First claim

What is claimed is:

1. A processor comprising:
processing circuitry to:
analyze a frame to determine a respective location of a dynamic object within the frame;
determine, based at least in part on a spatial relationship between the dynamic object and a related object region, a location of the related object region within the frame;
determine, based at least in part on the location, textual information within the related object region; and
perform, based at least in part on the textual information, one or more operations to augment an output of an application corresponding to the frame.

2. The processor of claim 1 , wherein the frame is one of a plurality of frames of a video, and the dynamic object is located at a different location from the respective location in at least one other frame of the plurality of frames.

3. The processor of claim 1 , wherein an appearance of the dynamic object is fixed, and an appearance of one or more related objects within the related object region is dynamic.

4. The processor of claim 1 , wherein the spatial relationship is determined based at least in part on analyzing a plurality of frames that include the dynamic object and one or more related objects positioned relative to the dynamic object.

5. The processor of claim 1 , wherein the determination of the location of the related object region includes determining, in image space, one or more pixel locations corresponding to the related object region based at least in part on one or more pixel locations corresponding to the respective location of the dynamic object.

6. The processor of claim 1 , wherein the analysis of the frame includes executing at least one of a computer vision algorithm, an object detection algorithm, or a neural network to identify the dynamic object in the frame.

7. The processor of claim 1 , wherein the determination of the textual information includes executing a character recognition operation within the related object region.

8. The processor of claim 1 , wherein the textual information within the related object region associated with the frame is different from other textual information associated with another frame of a video that includes the frame.

9. The processor of claim 1 , wherein the application corresponds to an instance of a cloud streaming application.

10. The processor of claim 1 , wherein the one or more operations includes at least one of generating a snapshot, generating a highlight, generating a recording, or updating an achievement or award.

11. A system comprising:
one or more processing units; and
one or more memory devices storing instructions that, when executed using the one or more processing units, cause the one or more processing units to execute:
determining a respective location of a dynamic object within a current frame;
determining, based at least in part on a known relationship between the dynamic object and a textual region, a location of the textual region within the current frame;
identifying textual information within the textual region based at least in part on the location; and
performing, based at least in part on the textual information, one or more operations to augment an output of an application corresponding to the current frame.

12. The system of claim 11 , wherein the known relationship is determined based at least in part on applying a plurality of frames to a machine learning model trained to detect a presence of objects and textual regions relative to the objects.

13. The system of claim 11 , wherein the known relationship is defined by a pixel distance between the dynamic object and the textual region.

14. The system of claim 13 , wherein the textual region is defined by at least one of an anchor pixel and a pixel dimension or pixel locations of one or more vertices of the textual region.

15. The system of claim 11 , wherein an appearance of the dynamic object is fixed and respective textual information corresponding to at least one other frame is different from the textual information corresponding to the current frame.

16. The system of claim 11 , wherein the current frame is comprised in a stream or a recording of one or more cloud gaming sessions, and the textual information includes information corresponding to one or more instances of a game from the one or more cloud gaming sessions.

17. The system of claim 11 , wherein the identifying the textual information includes executing an optical character recognition (OCR) algorithm within the textual region.

18. A method comprising:
determining a location of a dynamic object within a frame;
identifying an associated location of a textual region within the frame relative to the location of the dynamic object based at least in part on a spatial relationship between the dynamic object and the textual region;
determining, based at least in part on the associated location, textual information from within the textual region; and
performing, based at least in part on the textual information, one or more operations to augment an output of an application corresponding to the frame.

19. The method of claim 18 , wherein the frame is one of a plurality of frames of a stream, and the dynamic object is located at a different location from the location in at least one other frame of the plurality of frames.

20. The method of claim 18 , wherein an appearance of the dynamic object is fixed across two or more frames, and the textual information is different from respective textual information corresponding to at least one other frame.

Assignees

• Nvidia Corp

Inventors

• Van Welzen James
• White Jonathan
• Clark David
• Otterness Nathan
• Van Welzen Jack
• Yadav Prakash

Classifications

• G06V30/10

  Character recognition · CPC title

• H04N21/435

  Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream · CPC title

• G06V10/22

  by selection of a specific region containing or referencing a pattern; Locating or processing of specific regions to guide the detection or recognition · CPC title

• H04N21/235

  Processing of additional data, e.g. scrambling of additional data or processing content descriptors · CPC title

• H04N21/44008

  involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream (arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59) · CPC title