Method and system for analyzing live broadcast video content with a machine learning model implementing deep neural networks to quantify screen time of displayed brands to the viewer

Patent metadata

Field	Value
Publication number	US-11748785-B2
Application number	US-201916720708-A
Country	US
Kind code	B2
Filing date	Dec 19, 2019
Priority date	Dec 19, 2019
Publication date	Sep 5, 2023
Grant date	Sep 5, 2023

Abstract

A method for brand recognition in video by implementing a brand recognition application coupled to a streaming media player, for identifying an observed set of brands streamed in a broadcast video; receiving, by the brand recognition application, a broadcast video stream of a series of images contained in consecutive frames about an object of interest; extracting a set of brand features from each of image received by applying a trained brand recognition model with neural networks in order to detect one or more features related to each displayed object of interest in each frame, wherein the object of interest is associated with a brand image contained video content displayed to a viewer; and displaying, by a graphic user interface, information from the brand recognition application comprising at least time detected of the brand image in the video content of the broadcast video.

First claim

What is claimed is:

1. A method for brand recognition in broadcast video to be performed by a computing device, the method comprising:
receiving, by a brand recognition application executing on the computing device, a broadcast video stream comprising a series of digital images each comprising a plurality of pixels contained in consecutive frames;
automatically extracting, by the computing device, a plurality of objects of interest associated with a set of brand features from the digital images of the broadcast video stream by applying a trained brand recognition model that comprises a first neural network to each of the digital images in order to detect one or more features related to each object of interest contained in the pixels of the digital images, wherein each of the objects of interest is associated with a brand image
formulating a table comprising each of the objects of interest extracted from the trained brand recognition model, wherein the database associates each of the objects of interest with the brand image and at least one time during the broadcast video stream that each of the one or more objects was detected to appear in the video content of the broadcast video; and
providing the table as an output from the computing device.

2. The method of claim 1 , the brand recognition application further comprising:
performing one or more processing steps, by the brand recognition application, in implementing the trained brand recognition model comprising:
applying a feature extraction using the first neural network, wherein the first neural network is a trained neural network comprising a convolutional neural network (CNN) to classify one or more features in the low resolution images of the broadcast video stream;
applying a tensor mapping to each classify one or more features in a feature map; and
applying a region proposal for one or more regions covering the object of interest in each image to define one or more bounded regions of interest.

3. The method of claim 2 , further comprising:
identifying, by the brand recognition application, a foreground and a background for each region of the one or more regions based on the feature map for feature classification of features related to the object of interest of the selected brand image.

4. The method of claim 3 , further comprising:
applying, by the brand recognition application, a region of interest pooling layer (ROIP) to obtain a fixed vector representation in each region of interest to determine an identified feature in the low resolution images related to a selected brand.

5. The method of claim 4 , further comprising:
applying, by the brand recognition application, a second neural network comprising a region convolutional neural network (R-CNN) to the region of interest to determine a location of the selected brand within one or more of the low resolution images based on a stored set of brands classified by a trained CNN for brand recognition.

6. The method of claim 5 , further comprising:
training, by the brand recognition application, the trained brand recognition model by:
receiving a training set of one or more images in a training broadcast video stream comprising brand images with features in one or more frames associated with objects of brands; and
tagging features in one or more brand images of brand objects received in each frame of the training broadcast video stream to create a feature set for mapping each feature into the first neural network.

7. The method of claim 6 , further comprising:
extracting one or more features using a trained CNN and applying the tensor mapping to classify each feature in the trained brand recognition model; and
training one or more region proposal based on a set of classified features received in the training broadcast video stream.

8. The method of claim 1
wherein the database further tracks types of brands tracked, categories of brands, and number of times that each object of interest is displayed to the viewer.

9. The method of claim 1 , further comprising:
extracting images feature by feature in a live video broadcast using a trained CNN model for quantifying visual notice of object of interest in a video session.

10. A data processing system comprising a processor and a non-transitory data storage comprising computer-readable instructions that, when executed by the processor, perform an automated process comprising:
receiving, by a brand recognition application executing on the data processing system, a broadcast video stream comprising a series of digital images each comprising a plurality of pixels contained in consecutive frames;
automatically extracting a plurality of objects of interest associated with a set of brand features from the digital images of the broadcast video stream by applying a trained brand recognition model that comprises a first neural network to each of the digital images in order to detect one or more features related to each object of interest contained in the pixels of the digital images, wherein each of the objects of interest is associated with a brand image;
formulating a table comprising each of the objects of interest extracted from the trained brand recognition model, wherein the database associates each of the objects of interest with the brand image and at least one time during the broadcast video stream that each of the one or more objects was detected to appear in the video content of the broadcast video; and
providing the table as an output from the data processing system.

11. The data processing system of claim 10 , wherein the automated process further comprises:
performing one or more processing steps, by the brand recognition application, in implementing the trained brand recognition model comprising:
applying a feature extraction using the first neural network, wherein the first neural network is a trained neural network comprising a convolutional neural network (CNN) to classify one or more features in the pixels of each image;
applying a tensor mapping to each classify one or more features in a feature map; and
applying a region proposal for one or more regions covering the object of interest in each image to define one or more bounded regions of interest.

12. The data processing system of claim 11 , wherein the automated process further comprises:
identifying, by the brand recognition application, a foreground and a background for each region of the one or more regions based on the feature map for feature classification of features related to the object of interest of the selected brand image.

13. The data processing system of claim 12 , wherein the automated process further comprises:
applying, by the brand recognition application, a region of interest pooling layer (ROIP) to obtain a fixed vector representation in each region of interest to determine an identified feature in the image related to a selected brand.

14. The data processing system of claim 13 , wherein the automated process further comprises:
applying, by the brand recognition application, a second neural network to the region of interest to determine a location of the selected brand within the pixels of the image based on a stored set of brands classified by the first neural network for brand recognition, wherein the second neural network is a region convolutional neural network (R-CNN).

15. The data processing system of claim 14 , wherein the automated process further comprises:
training, by the brand recognition application, the trained brand recognition model by:
receiving a training set of one or more images in a training broadcast video stream compr

Assignees

• Sling Media Pvt Ltd
• Dish Network Technologies India Pvt Ltd

Inventors

• Lohumi Avinash Chandra
• Gangotri Arun Lokman

Classifications

• G06Q30/0272 Primary

  Period of advertisement exposure · CPC title

• G06F18/211

  Selection of the most significant subset of features · CPC title

• G06F18/214

  Generating training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title

• G06F18/2411

  based on the proximity to a decision surface, e.g. support vector machines · CPC title

• G06F18/2451

  linear, e.g. hyperplane · CPC title