System and method for recognition of items in media data and delivery of information related thereto

Patent metadata

Field	Value
Publication number	US-10779019-B2
Application number	US-201916426339-A
Country	US
Kind code	B2
Filing date	May 30, 2019
Priority date	Jan 18, 2011
Publication date	Sep 15, 2020
Grant date	Sep 15, 2020

Abstract

The system and method disclosed herein relate to the establishment of both the location and identity of individual items in pictures or videos and delivering of related information thereto. Initially, images in video source content are scanned and individual elements in the images are dynamically compared with digital images stored in a database to identify individual items in the image that match the digital images stored in the database. Upon detecting a match, information is accessed in the database that is related to the digital image that matches the identified individual item. The accessed information is then transmitted to a secondary system that is in communication with a display device, such that the accessed information is displayed on the secondary system in coordination with the video source content being displayed on the display device.

First claim

What is claimed is:

1. A system for dynamically recognizing individual items in images contained in video source content and delivering related metadata, the system comprising:
an electronic database configured to store a plurality of digital images and item metadata related to each of the plurality of digital images, wherein the plurality of digital images and the item metadata are organized into a plurality of subsets categorized by respective common characteristics for each of the plurality of subsets;
a source content interpreter configured to interpret video source content to determine a reference marker characteristic associated with the interpreted video source content before the video source content is received by a content consumption device;
a matching processor configured to:
identify at least one subset of the plurality of digital images and the related item metadata by matching the determined reference marker characteristic of the interpreted video source content with the respective common characteristic of the at least one subset, and
compare individual items in a plurality of images in the video source content with the plurality of digital images in the identified at least one subset to identify at least one individual item in the plurality of images that matches a corresponding digital image in the identified at least one subset, wherein the matching processor does not compare the individual items in the plurality of images with digital images in the electronic database that are not within the identified at least one subset;

a merging processor configured to merge the item metadata related to the identified at least one individual item with the interpreted source video content to create a modified content stream; and
a server configured to transmit the modified content stream towards the content consumption device, such that the respective item metadata related to the identified at least one individual item is configured to be accessed by the content consumption device when the modified content stream is being displayed thereon.

2. The system according to claim 1 , wherein the identified at least one individual item in the modified content stream transmitted to the content consumption device is configured to be selected by a user selection using at least one of a mouse pointer and a touch screen.

3. The system according to claim 1 , wherein the identified at least one individual item in the modified content stream transmitted to the content consumption device is configured to be interpreted and selected automatically by the display device.

4. The system according to claim 1 , wherein the video source content is one of live video content or prerecorded video content.

5. The system according to claim 1 , wherein the matching processor is further configured to generate coordinate position data indicating a position of the identified at least one individual item in the plurality of images contained in the video source content.

6. The system according to claim 5 , wherein the server is further configured to transmit the coordinate position data towards the content consumption device that is configured to display the modified content stream with at least one electronic indicator, which is based on the coordinate position data, for the identified at least one individual item such that the identified at least one individual item is rendered active and configured to be selected by a user of the content consumption device.

7. The system according to claim 6 , wherein the merging processor is configured to merge the item metadata related to the identified at least one individual item with the interpreted source video content, such that identification information of the identified at least one individual item is displayed on the content consumption device.

8. The system according to claim 1 , wherein the server is configured to transmit the item metadata, which links product information related to the identified at least one individual item with the video source content, towards the content consumption device that receives the video source content as a separate data stream from the product information.

9. The system of claim 6 , further comprising at least one processor configured to initiate an electronic shopping interaction in response to a selection of the active and identified at least one individual item.

10. The system of claim 1 , wherein the content consumption device is at least one of a computer, a smartphone, a tablet, a PDA, a television, a display with a set-top box type processor, a Blu-ray player, and a video game console.

11. The system of claim 7 , wherein the item metadata is configured to make active the identified at least one individual item with either a visible or an invisible indicator when the identification information is rendered by the content consumption device.

12. A method for dynamically recognizing individual items in images contained in video source content and delivering related metadata, the method comprising:
storing, by an electronic database, a plurality of digital images and item metadata related to each of the plurality of digital images, wherein the plurality of digital images and the item metadata are organized into a plurality of subsets categorized by respective common characteristics for each of the plurality of subsets;
interpreting video source content to determine a reference marker characteristic associated with the interpreted video source content before the video source content is received by a content consumption device;
identifying at least one subset of the plurality of digital images and the related item metadata by matching the determined reference marker characteristic of the interpreted video source content with the respective common characteristic of the at least one subset;
comparing individual items in a plurality of images in the video source content with the plurality of digital images in the identified at least one subset to identify at least one individual item in the plurality of images that matches a corresponding digital image in the identified at least one subset, wherein the individual items in the plurality of images are not compared with digital images in the electronic database that are not within the identified at least one subset;
merging the item metadata related to the identified at least one individual item with the interpreted source video content to create a modified content stream; and
transmitting the modified content stream towards the content consumption device, such that the respective item metadata related to the identified at least one individual item is configured to be accessed by the content consumption device when the modified content stream is being displayed thereon.

13. The method according to claim 12 , further comprising configuring the identified at least one individual item in the modified content stream transmitted to the content consumption device to be selected by a user selection using at least one of a mouse pointer and a touch screen.

14. The method according to claim 12 , further comprising configuring the identified at least one individual item in the modified content stream transmitted to the content consumption device to be interpreted and selected automatically by the display device.

15. The method according to claim 12 , wherein the video source content is one of live video content or prerecorded video content.

16. The method according to claim 12 , further comprising generating coordinate position data indicating a position of the identified at least one individual item in the plural

Assignees

• Hsni Llc

Inventors

• Mcdevitt John

Classifications

• G06V20/48

  Matching video sequences · CPC title

• G06V20/40

  in video content (extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44) · CPC title

• H04N21/482

  End-user interface for programme selection · CPC title

• H04N21/4884

  for displaying subtitles · CPC title

• H04N21/8133

  specifically related to the content, e.g. biography of the actors in a movie, detailed information about an article seen in a video programme · CPC title