Electronic apparatus for constructing a fingerprint database, control method thereof and electronic system

Patent metadata

Field	Value
Publication number	US-11302101-B2
Application number	US-201916517768-A
Country	US
Kind code	B2
Filing date	Jul 22, 2019
Priority date	Sep 18, 2018
Publication date	Apr 12, 2022
Grant date	Apr 12, 2022

Abstract

An electronic apparatus is provided. The electronic apparatus includes a communicator; and a processor configured to: receive multimedia data from an external apparatus through the communicator; identify a plurality of objects in a first frame among a plurality of frames included in the multimedia data; obtain content information corresponding to the plurality of objects in the first frame; obtain a first fingerprint based on the first frame; and control the communicator to transmit the content information and the first fingerprint to a server.

First claim

What is claimed is:

1. An electronic apparatus comprising:
a communicator; and
a processor configured to:
receive multimedia data from an external apparatus through the communicator;
identify a first plurality of objects in a first frame among a plurality of frames included in the multimedia data, the first plurality of objects comprising first text characters;
obtain first content information corresponding to the first plurality of objects in the first frame, the first content information comprising first text indicated by the first text characters;
obtain a first fingerprint based on the first frame;
control the communicator to transmit the first fingerprint to a server;
obtain second content information corresponding to a second plurality of objects in a second frame;
obtain third content information corresponding to a third plurality of objects in a third frame;
identify whether identical objects are included in each of the first plurality of objects, the second plurality of objects and the third plurality of objects; and
control the communicator to transmit the third content information to the server based on identifying the identical objects are included in each of the first plurality of objects, the second plurality of objects and the third plurality of objects,
wherein the first frame, the second frame and the third frame are continuous frames of the plurality of frames.

2. The electronic apparatus of claim 1 , wherein the plurality of frames comprises a fourth frame after the third frame,
wherein the fourth frame does not include content information, and
wherein the processor is further configured to:
obtain a fourth fingerprint based on the fourth frame; and
control the communicator to transmit the fourth fingerprint to the server.

3. The electronic apparatus of claim 1 , wherein the processor is further configured to control the communicator to transmit the third content information, a third fingerprint corresponding to the third frame and a third time point of reproduction of the third frame to the server based on identifying the identical objects are included in each of the first plurality of objects, the second plurality of objects and the third plurality of objects.

4. The electronic apparatus of claim 1 , wherein the processor is further configured to:
identify a first area of the first frame corresponding to the first plurality of objects;
identify a second area of the first frame that is adjacent the first area;
obtain a second fingerprint of the second frame based on a portion of the second frame that corresponds to the second area of the first frame;
obtain a third fingerprint of the third frame based on a portion of the third frame that corresponds to the second area of the first frame; and
control the communicator to transmit the second fingerprint and the third fingerprint to the server.

5. The electronic apparatus of claim 1 , further comprising a storage,
wherein the processor is further configured to identify the first plurality of objects in the first frame based on a content information acquisition model stored in the storage, and
wherein the content information acquisition model is acquired by training based on a plurality of sample images and a plurality of pieces of content information included in the plurality of sample images through an artificial intelligence algorithm.

6. The electronic apparatus of claim 1 , further comprising a display,
wherein the processor is further configured to:
control the display to sequentially display the plurality of frames; and
sequentially obtain a corresponding fingerprint for a displayed frame among the plurality of frames.

7. The electronic apparatus of claim 1 , wherein the third content information comprises a title of a content corresponding to the first frame, a reproduction time of the content and channel information of the content.

8. An electronic system comprising:
a server; and
an electronic apparatus configured to:
identify a first plurality of objects in a first frame among a plurality of frames included multimedia data received from an external apparatus, the first plurality of objects comprising first text characters;

obtain first content information corresponding to the first plurality of objects in the first frame, the first content information comprising first text indicated by the first text characters;
obtain a first fingerprint based on the first frame;
transmit the first fingerprint to the server;
obtain a second fingerprint based on a second frame;
obtain second content information corresponding to a second plurality of objects in the second frame;
obtain a third fingerprint based on a third frame;
obtain third content information corresponding to a third plurality of objects in the third frame;
identify whether identical objects are included in each of the first plurality of objects, the second plurality of objects and the third plurality of objects; and
transmit the third content information to the server based on identifying the identical objects are included in each of the first plurality of objects, the second plurality of objects and the third plurality of objects,

wherein the server is configured to:
receive the third content information and the third fingerprint from the electronic apparatus; and
map the third fingerprint with a content corresponding to the third content information among a plurality of pre-stored contents and store the third fingerprint.

9. The electronic system of claim 8 , wherein the plurality of frames comprises a fourth frame after the third frame,
wherein the fourth frame does not include content information,
wherein the electronic apparatus is further configured to:
obtain a fourth fingerprint corresponding to the fourth frame; and
transmit the fourth fingerprint to the server, and

wherein the server is further configured to:
receive the fourth fingerprint from the electronic apparatus; and
map the fourth fingerprint with the content corresponding to the third content information and store the fourth fingerprint.

10. The electronic system of claim 8 , further comprising another electronic apparatus configured to reproduce the content,
wherein the server is configured to, based on fingerprints from the electronic apparatus not being received, transmit a request for fingerprints of frames for which fingerprints were not received to the another electronic apparatus.

11. The electronic system of claim 9 , wherein the server is further configured to, based on the third fingerprint received from the electronic apparatus being pre-stored, transmit a request for stopping transmission of fingerprints for the content to the electronic apparatus, and
wherein the electronic apparatus is further configured to, based on receiving the request for stopping transmission, stop obtaining fingerprints.

12. The electronic system of claim 8 , wherein the server is further configured to:
identify an incomplete content for which fingerprints for all frames are not stored among the plurality of pre-stored contents; and
transmit a request for fingerprints for the incomplete content to the electronic apparatus.

13. A method of controlling an electronic apparatus comprising:
receiving multimedia data from an external apparatus;
identifying a first plurality of objects in a first frame among a plurality of frames included in the multimedia data, the first plurality of objects comprising first text characters;
obtaining first content information corresponding to the first plurality of objects in the first frame, the first content information comprising first text indicated

Assignees

• Samsung Electronics Co Ltd

Inventors

• Kim Younguk
• Kim Minsoo
• Kim Sungjin
• Kim Hyunhan
• Choi Jiwoong
• Choi Hyunsoo
• Kim Ilkoo

Classifications

• H04N21/6582 Primary

  Data stored in the client, e.g. viewing habits, hardware capabilities, credit card number (arrangements where receivers interact with the broadcast H04H20/38) · CPC title

• G06V20/46 Primary

  Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames · CPC title

• G06F18/214

  Generating training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title

• H04N21/23418

  involving operations for analysing video streams, e.g. detecting features or characteristics (television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59) · CPC title

• H04N21/2393

  involving handling client requests (scheduling or organising the servicing of application requests H04L67/60) · CPC title