Method and a device for tracking characters that appear on a plurality of images of a video stream of a text

Patent metadata

Field	Value
Publication number	US-2016162733-A1
Application number	US-201514962557-A
Country	US
Kind code	A1
Filing date	Dec 8, 2015
Priority date	Dec 8, 2014
Publication date	Jun 9, 2016
Grant date	—

Abstract

The tracking method comprises, for at least a first image of the text having at least a first line of characters:
applying a prediction of a movement to which the text is subjected between the first image and a second image of the video stream, the movement prediction being applied to at least one second line of characters of the second image; determining at least one alignment hypothesis for aligning the first line with the second line after applying the movement prediction; estimating for each alignment hypothesis, a geometrical transformation between the first line and the second line resulting in that alignment; and evaluating a character match metric for each alignment hypothesis, the metric being evaluated from signatures calculated on the characters of at least one line of the first image and signatures calculated on the characters of at least one line of the second image put into correspondence with the characters of said at least one line of the first image after applying the geometrical transformation associated with the alignment hypothesis.

First claim

1 . A tracking method for tracking characters that appear in a plurality of images in a video stream of a text including at least one line of characters, said images being acquired by means of at least one sensor at different instants and each representing at least a portion of the text, the method comprising, for at least one first image of the video stream having at least one first line of characters:
an application step of applying a prediction of a movement to which the text is subjected between the first image and a second image of the video stream, the movement prediction being applied to at least one second line of characters of the second image; a determination step of determining at least one alignment hypothesis for aligning said at least one first line of characters of the first image with said at least one second line of characters of the second image after applying the movement prediction; an estimation step of estimating, for each alignment hypothesis, a geometrical transformation between the first line of characters and the second line of characters resulting in that alignment; and an evaluation step of evaluating a character match metric for each alignment hypothesis, the metric being evaluated from signatures calculated on the characters of at least one line of characters of the first image and signatures calculated on the characters of at least one line of characters of the second image put into correspondence with the characters of said at least one line of characters of the first image after applying the geometrical transformation associated with the alignment hypothesis.

2 . A tracking method according to claim 1 , comprising, when a plurality of alignment hypotheses are determined for said first image, a selection step of selecting for said first image the alignment hypothesis that corresponds to the best character match metric over said plurality of alignment hypotheses.

3 . A tracking method according to claim 1 , wherein the determination step further comprises a verification step of verifying compatibility between the first line of characters of the first image and the second line of characters of the second image after applying the movement prediction, an alignment hypothesis being determined only if the first line of characters of the first image is compatible with the second line of characters of the second image.

4 . A tracking method according to claim 3 , wherein during the verification step, the first line of characters of the first image and the second line of characters of the second image are considered as being compatible if a distance evaluated between the first line and the second line after application of the movement prediction is less than a predetermined threshold.

5 . A tracking method according to claim 1 , wherein during the determination step, an alignment hypothesis is determined for aligning a first line of characters of the first image with a second line of characters of the second image after applying the movement prediction, if the first and second lines present a number of characters put into correspondence by the alignment that is greater than a predetermined threshold.

6 . A tracking method according to claim 1 , wherein the geometrical transformation associated with the alignment hypothesis comprises a translation and/or a rotation.

7 . A tracking method according to claim 1 , wherein, during the evaluation step, the metric associated with an alignment hypothesis is evaluated while taking account of all of the lines of characters of the first image or all of the lines of characters of the first image with the exception of the first line of characters for which the alignment hypothesis was determined.

8 . A tracking method according to claim 7 , wherein the step of evaluating the metric associated with each alignment hypothesis comprises:
for each line of characters of the first image, evaluating a statistical combination of distances between the signatures calculated on the characters of said line and the signatures calculated on the corresponding characters of a line of characters of the second image that is compatible with said line; and calculating a statistical combination of the statistical combinations of the distances as evaluated in this way on the lines of characters of the first image.

9 . A tracking method according to claim 8 , wherein the evaluation step further comprises determining an offset of the line of characters of the first image relative to the line of characters of the second image, and weighting the statistical combination of the distances between the signatures by the offset.

10 . A tracking method according to claim 9 , wherein, if a line of characters of the first image is compatible with a plurality of lines of characters of the second image, the metric takes account of the weighted statistical combination of the distances that is the smallest among the statistical combinations of the distances evaluated over said plurality of lines of characters.

11 . A tracking method according to claim 1 , including a step of recognizing characters of the text by using the character match metrics evaluated for the alignment hypotheses.

12 . A computer program including instructions for executing steps of the tracking method according to claim 1 when said program is executed by a computer.

13 . A computer readable data medium storing a computer program including instructions for executing steps of the tracking method according to claim 1 .

14 . A tracking device for tracking characters that appear in a plurality of images in a video stream of a text including at least one line of characters, said images being acquired by means of at least one sensor at different instants and each representing at least a portion of the text, the device comprising modules that are activated for at least one first image of the video stream having at least one first line of characters:
to apply a prediction of a movement to which the text is subjected between the first image and a second image of the video stream, the movement prediction being applied to at least one second line of characters of the second image; to determine at least one alignment hypothesis for aligning said at least one first line of characters of the first image with said at least one second line of characters of the second image after applying the movement prediction; to estimate, for each alignment hypothesis, a geometrical transformation between the first line of characters and the second line of characters resulting in that alignment; and to evaluate a character match metric for each alignment hypothesis, the metric being evaluated from signatures calculated on the characters of at least one line of characters of the first image and signatures calculated on the characters of at least one line of characters of the second image put into correspondence with the characters of said at least one line of characters of the first image after applying the geometrical transformation associated with the alignment hypothesis.

15 . A system comprising:
at least one sensor suitable for acquiring a plurality of images of a video stream of a text comprising at least one line of characters, the images being acquired by means of said at least one sensor at different instants and representing at least a portion of the text; and a tracking device according to claim 14 for tracking characters appearing in the plurality of images.

Assignees

• Morpho

Inventors

• Rouh Alain
• Beaudet Jean

Classifications

• G06K9/325

  Physics · mapped topic

• G06T7/0038

  Physics · mapped topic

• G06T2207/30176

  Document · CPC title

• G06T7/204

  Physics · mapped topic

• G06K9/00711

  Physics · mapped topic