Image processing device, method and non-transitory computer readable recording medium for tracking a target in a motion picture using color information

Patent metadata

Field	Value
Publication number	US-10319097-B2
Application number	US-201615331923-A
Country	US
Kind code	B2
Filing date	Oct 24, 2016
Priority date	Oct 23, 2015
Publication date	Jun 11, 2019
Grant date	Jun 11, 2019

Abstract

An image processing device is provided. The image processing device includes: a tracking area setting section that sets a tracking area in a frame image of an original motion picture; a color information obtaining section that obtains color information of the tracking area set by the tracking area setting section; a tracking position estimating section that estimates a tracking position with reference to the tracking area with respect to each frame image of the original motion picture by using the color information obtained by the color information obtaining section; an output frame setting section that sets an output frame enclosing the tracking area set by the tracking area setting section and that updates the output frame based on the estimated tracking position; and a lock-on motion picture generating section that generates a motion picture that is based on the output frame.

First claim

What is claimed is:

1. An image processing device, comprising:
a processor configured to act as:
a tracking area setting unit configured to set a plurality of tracking areas on a target in a frame image of an original motion picture;
a color information obtaining unit configured to obtain color information with respect to each of the plurality of tracking areas set by the tracking area setting unit;
a tracking position estimating unit configured to estimate a plurality of tracking positions with reference to the respective plurality of tracking areas with respect to other frame image of the original motion picture based on the color information of the respective plurality of tracking areas obtained by the color information obtaining unit;
an output frame setting unit configured to set an output frame enclosing the tracking area set by the tracking area setting unit and to update the output frame based on the estimated tracking position;
a motion picture generating unit configured to generate a motion picture that is based on the output frame;
a tracking position adequacy determining unit configured to determine that the tracking positions estimated by the tracking position estimating unit are adequate when distance between the plurality of tracking positions that are estimated by the tracking position estimating unit with reference to the respective plurality of tracking areas set by the tracking area setting unit is equal to or less than a predetermined distance threshold; and
a tracking position re-estimating unit configured to re-estimate the tracking position if the tracking position adequacy determining unit determines that the tracking position is inadequate.

2. The image processing device according to claim 1 ,
wherein the output area setting unit is configured to suspend updating the output frame when the tracking position re-estimating unit re-estimates the tracking position.

3. The image processing device according to claim 2 ,
wherein the tracking position estimating unit is configured to estimate the tracking position with respect to the other frame of the original motion picture based on the tracking position re-estimated by the tracking position re-estimating unit; and
the output frame setting unit is configured to resume updating the output frame when the tracking position estimated by the tracking position estimating unit is included within a predetermined range based on the output frame.

4. The image processing device according to claim 3 ,
wherein the tracking position estimating unit estimates the plurality of tracking positions with reference to the respective plurality of tracking areas with respect to the other frame image of the original motion picture in a forward order in time,
wherein, at the time of the resumption of updating the output frame, the tracking position re-estimating unit is configured to re-estimate the tracking positions with reference to the respective plurality of tracking areas with respect to the other frame image in a reverse order in time from the frame image where the resumption of updating the output frame has occurred to the frame image where the suspension of updating the output frame has occurred.

5. The image processing device according to claim 1 ,
wherein the tracking area setting unit is configured to set a plurality of tracking areas on a plurality of targets in the frame image of the original motion picture; and
the image processing device further comprises a priority setting unit configured to set priority of tracking among the plurality of targets; and
the output frame setting unit is configured to set an output frame to be updated based on the priority set by the priority setting unit.

6. The image processing device according to claim 1 , further comprising:
a pattern information obtaining unit configured to obtain pattern information of the tracking area set by the tracking area setting unit,
wherein the tracking area estimating unit is configured to estimate the tracking position with respect to the other frame of the original motion picture further based on the pattern information obtained by the pattern information obtaining unit.

7. The image processing device according to claim 1 , further comprising:
an output frame position correcting unit configured to correct a position of the output frame based on a displacement of the position of the output frame updated by the output frame setting unit,
wherein the motion picture generating unit is configured to generate the motion picture that is based on the output frame corrected by the output frame position correcting unit.

8. An electronic device, comprising:
the image processing device according to claim 1 ;
a display unit integrally formed with a touch panel; and
a display control unit configured to control the display unit;
wherein the tracking area setting unit is configured to set the tracking area based on a user operation performed on the touch panel, and
the display control unit comprises a motion picture display control unit configured to display the motion picture generated by the motion picture generating unit on the display unit.

9. The electronic device according to claim 8 ,
wherein the display control unit comprises a tracking area display control unit configured to overlay the tracking area set by the tracking area setting unit on the motion picture in an identifiable displaying manner.

10. The electronic device according to claim 8 ,
wherein the display control unit comprises a first output frame display control unit configured to overlay the output frame updated by the output frame setting unit on the motion picture in an identifiable displaying manner.

11. The electronic device according to claim 10 ,
wherein the motion picture display control unit is configured to display the motion picture that shows an enlarged predetermined area including the output frame on the display unit, and
the first output frame display control unit is configured to overlay the output frame on the motion picture, in which a size of the output frame is adjusted corresponding to the enlarged predetermined area.

12. The electronic device according to claim 8 ,
wherein the display control unit comprises:
an original motion picture display control unit configured to display the original motion picture on the display unit; and
a second output frame display control unit configured to overlay the output frame updated by the output frame setting unit on the original motion picture in an identifiable displaying manner.

13. The electronic device according to claim 12 , further comprising:
a moving operation determining unit configured to make a determination as to whether a moving operation of moving the output frame currently overlaid on the original motion picture is performed through a user operation on the touch panel; and
an object detecting unit configured to detect an object present in the output frame if the moving operation determining unit determines that the moving operation of the output frame is performed,
wherein the color information obtaining unit is configured to obtain color information of the object detected by the object detecting unit, and
the tracking position estimating unit is configured to set an area of the object detected by the object detecting unit as the tracking area and to estimate the tracking position with respect to the other frame of the original motion picture.

14. An electronic device, comprising:
the image processing device according to claim 1 ;
a display unit integrally formed with a touch panel; and
a display control unit configured to contr

Assignees

• Morpho Inc

Inventors

• Kuroda Yasuhiro
• Hyakutake Akito
• Takizawa Naruo
• Ono Ryo

Classifications

• G06T2207/20021

  Dividing image into blocks, subimages or windows · CPC title

• G06T2207/10024

  Color image · CPC title

• G06T7/246 Primary

  using feature-based methods, e.g. the tracking of corners or segments · CPC title

• G06T7/20 Primary

  Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51) · CPC title

• G06F3/0485

  Scrolling or panning · CPC title