System and method for using prior frame data for OCR processing of frames in video sources

Patent metadata

Field	Value
Publication number	US-9792895-B2
Application number	US-201514863512-A
Country	US
Kind code	B2
Filing date	Sep 24, 2015
Priority date	Dec 16, 2014
Publication date	Oct 17, 2017
Grant date	Oct 17, 2017

Abstract

Disclosed are systems, methods and computer program products for using prior frame data for OCR processing of frames in video sources to detect natural language text therein. An example includes receiving a frame from a video source and retrieving prior frame data associated with the video source. The OCR-processing includes using prior frame data to detect blobs similar to blobs described in the prior frame data; using detected similar blobs to detect in the frame character candidates similar to character candidates described in the prior frame data; using detected similar character candidates to detect in the frame text candidates similar to text candidates described in the prior frame data; and using detected similar text candidates to detect in the frame text strings similar to text strings described in the prior frame data.

First claim

The invention claimed is:

1. A method for OCR-processing of a current frame in a video source, the method comprising:
receiving, by a hardware processor, the current frame from the video source;
retrieving, from a memory, prior frame data of a prior frame associated with the video source;
detecting, by the hardware processor in at least a portion of the current frame, at least one current frame blob similar to at least one prior frame blob identified in the prior frame data;
designating, by the hardware processor, the at least one current frame blob as at least one current character candidate without performing character candidate detection operations; and
identifying, in the at least a portion of the current frame, at least one current text string using the at least one current frame blob.

2. The method of claim 1 , further comprising at least one of:
translating the at least one current text string into another language; or
converting the at least one current text string to audio.

3. The method of claim 1 , wherein the at least one current frame blob is similar to the at least one prior frame blob based on at least one of:
similarity in position within a respective frame,
similarity in orientation within a respective frame,
similarity in shape, or
similarity in content.

4. The method of claim 1 , wherein identifying, in the at least a portion of the current frame, at least one current text string using the at least one current frame blob, comprises:
grouping the at least one current frame blob designated as the at least one current character candidate and one or more other current character candidates into at least one current cluster of character candidates;
determining whether the at least one current cluster of character candidates is similar to at least one prior cluster of character candidates identified in the prior frame data;
in response to determining that the at least one current cluster of character candidates is similar to the at least one prior cluster of character candidates identified in the prior frame data, designating the at least one current cluster of character candidates as at least one current text candidate, without performing text candidate detection operations;
determining whether current text candidates comprising the at least one current cluster of character candidates designated as the at least one current text candidate are similar to prior text candidates identified in the prior frame data;
in response to determining that the current text candidates comprising the at least one current cluster of character candidates designated as the at least one current text candidate are similar to the prior text candidates identified in the prior frame data, designating the current text candidates as at least one current text string candidate, without performing text string candidate detection operations; and
determining that the current text candidates designated as the at least one current text string candidate is similar to at least one prior text string candidate identified in the prior frame data.

5. The method of claim 1 , wherein the detecting of the at least one current frame blob similar to the at least one prior frame blob identified in the prior frame data is based on at least one blob description in the prior frame data that comprises:
position of the at least one prior frame blob within the prior frame;
orientation of the at least one prior frame blob within the prior frame; and
shape of the at least one prior frame blob.

6. The method of claim 4 , wherein determining that the at least one current cluster of character candidates is similar to the at least one prior cluster of character candidates identified in the prior frame data is based on at least one character candidate description in the prior frame data that comprises:
position of the at least one prior cluster of character candidates within the prior frame;
orientation of the at least one prior cluster of character candidates within the prior frame; and
shape of the at least one prior cluster of character candidates.

7. The method of claim 4 , wherein determining that the current text candidates comprising the at least one current cluster of character candidates designated as the at least one current text candidate are similar to the prior text candidates identified in the prior frame data is based on at least one text candidate description in the prior frame data that comprises:
position of the prior text candidates within the prior frame;
orientation of the prior text candidates within the prior frame; and
shape of the prior text candidates.

8. The method of claim 4 , wherein determining that the current text candidates designated as the at least one current text string candidate is similar to the at least one prior text string candidate identified in the prior frame data is based on at least one text string description in the prior frame data that comprises:
position of the at least one prior text string within the prior frame;
orientation of the at least one prior text string within the prior frame; and
content of the at least one prior text string.

9. The method of claim 1 , wherein the hardware processor used for OCR-processing of the at least a portion of the current frame and at least one hardware processor used to generate the prior frame data are different processors.

10. The method of claim 1 , wherein the hardware processor used for OCR-processing of the at least a portion of the current frame and at least one hardware processor used to generate the prior frame data are different cores of a single multi-core processor.

11. The method of claim 4 further comprising storing in the memory new frame data comprising at least one of:
the at least one current frame blob;
the at least one current character candidate;
the at least one current text candidate; or
the at least one current text string.

12. The method of claim 4 , further comprising at least one of:
detecting in the at least a portion of the current frame at least one blob not similar to blobs described in the prior frame data;
detecting in the at least a portion of the current frame at least one character candidate not similar to character candidates described in the prior frame data;
detecting in the at least a portion of the current frame at least one text candidate not similar to text candidates described in the prior frame data; or
detecting in the at least a portion of the current frame at least one text string not similar to text strings described in the prior frame data.

13. A system for OCR-processing of a current frame in a video source, the system comprising:
a computer processor configured to receive the current frame from the video source; and
a memory coupled to the processor configured to:
store a prior frame data of a prior frame associated with the video source; detect, in at least a portion of the current frame, at least one current frame blob similar to at least one prior frame blob identified in the prior frame data;
designate at least one current frame blob as at least one current character candidate without performing character candidate detection operations; and
identify, in the at least a portion of the current frame, at least one current text string using the at least one current frame blob.

14. The system of claim 13 , wherein the processor is further configured to perform at least one of:
translating the at least one current text string into another language; or
converting the at least one current text string to audio.

15. The system of claim 13 , wherein the at least o

Assignees

• Abbyy Dev Llc

Inventors

• Khintsitskiy Ivan
• Isaev Andrey
• Fedorov Sergey

Classifications

• G06V30/147 Primary

  Determination of region of interest · CPC title

• G10L13/00

  Speech synthesis; Text to speech systems · CPC title

• G06V30/10

  Character recognition · CPC title

• G06K2209/01

  Physics · mapped topic

• G06K9/3233

  Physics · mapped topic