Method and system for processing video content

Patent metadata

Field	Value
Publication number	US-2016140392-A1
Application number	US-201414542294-A
Country	US
Kind code	A1
Filing date	Nov 14, 2014
Priority date	Nov 14, 2014
Publication date	May 19, 2016
Grant date	—

Abstract

Various aspects of a method and system to process video content for extraction of moving objects from image sequences of the video content are disclosed herein. In an embodiment, the method includes determination of one or more object contours of one or more moving objects in the video content. A first object bounding box (OBB) that encompasses a first object contour of a first moving object is created based on the determined one or more object contours. A first object mask for the first moving object is generated in a first destination image frame, based on infilling of the first object contour in the created first OBB.

First claim

What is claimed is:

1 . A method for processing a video content, the method comprising:
in a video processor:
determining one or more object contours of one or more moving objects in said video content;
creating a first Object Bounding Box (OBB) that encompasses a first object contour of a first moving object based on said determined said one or more object contours; and
dynamically generating a first object mask for said first moving object in a first destination image frame based on infilling of said first object contour in said created said first OBB.

2 . The method as claimed in claim 1 , further comprising detecting said one or more moving objects based on three consecutive image frames of said video content, wherein said three consecutive image frames comprise a previous image frame, a current image frame, and a next image frame.

3 . The method as claimed in claim 2 , further comprising:
aligning said current image frame to said previous image frame, and said next image frame to said current image frame; and determining a first absolute difference image between said aligned said current image frame and said previous image frame, and a second absolute difference image between said aligned said next image frame and said current image frame.

4 . The method as claimed in claim 3 , further comprising detecting said one or more object contours corresponding to said one or more moving objects in said determined said first absolute difference image, and said determined said second absolute difference image.

5 . The method as claimed in claim 4 , further comprising determining a first object contour image based on said detected said one or more object contours in said first absolute difference image, and a second object contour image based on said detected said one or more object contours in said second absolute difference image.

6 . The method as claimed in claim 5 , further comprising determining an intermediate object contour image based on an intersection between said determined said first object contour image, and said determined said second object contour image.

7 . The method as claimed in claim 6 , further comprising utilizing said determined said intermediate object contour image and a gradient information associated with said intermediate object contour image for said determination of said one or more object contours of said one or more moving objects.

8 . The method as claimed in claim 1 , further comprising:
setting a first pixel value in said first destination image frame; and extracting color attributes from said determined said one or more object contours such that a second pixel value is set in said determined said one or more object contours.

9 . The method as claimed in claim 8 , further comprising copying said created said first OBB to said first destination image frame at a predetermined location.

10 . The method as claimed in claim 9 , further comprising:
detecting whether a first pixel in said first OBB has a third pixel value, and a second adjacent pixel in said first destination image frame has said first pixel value; and assigning said first pixel value to said detected said first pixel when said adjacent second pixel is detected with said first pixel value.

11 . The method as claimed in claim 10 , wherein said first pixel value corresponds to a color gray, said second pixel value corresponds to a color white, and said third pixel value corresponds to a color black.

12 . The method as claimed in claim 10 , further comprising assigning said second pixel value to one or more pixels in said first OBB having third pixel value for said infilling of said first object contour, when two adjacent pixels in said first OBB are not detected in said third pixel value and said first pixel value.

13 . The method as claimed in claim 1 , further comprising creating a second OBB that encompasses a second moving object when said second moving object is detected in said video content.

14 . The method as claimed in claim 13 , further comprising copying said created said second OBB to a second destination image frame at a predetermined location, wherein said second destination frame corresponds to said first destination frame that comprises said generated said first object mask for said first moving object.

15 . The method as claimed in claim 14 , further comprising generating a second object mask of said second moving object in said second OBB, wherein said second moving object in said second OBB is associated with said generated said first object mask of said first moving object.

16 . The method as claimed in claim 15 , further comprising extracting said first moving object and said second moving object based on said generated said first object mask and said generated said second object mask.

17 . A system for processing a video content, the system comprising:
a video processor operable to:
determine one or more object contours of one or more moving objects in said video content;
create a first Object Bounding Box (OBB) that encompasses a first object contour of a first moving object based on said determined said one or more object contours; and
generate a first object mask for said first moving object in a first destination image frame based on infilling of said first object contour in said created said first OBB.

18 . The system as claimed in claim 17 , wherein said video processor is further operable to detect said one or more moving objects based on three consecutive image frames of said video content, wherein said three consecutive image frames comprise a previous image frame, a current image frame, and a next image frame.

19 . The system as claimed in claim 17 , wherein said video processor is further operable to:
set a first pixel value in said first destination image frame; and extract color attributes from said determined said one or more object contours such that a second pixel value is set in said determined said one or more object contours.

20 . The system as claimed in claim 19 , wherein said video processor is further operable to copy said created said first OBB to said first destination image frame at a predetermined location.

21 . The system as claimed in claim 20 , wherein said video processor is further operable to:
detect whether a first pixel in said first OBB has a third pixel value, and a second adjacent pixel in said first destination image frame has said first pixel value; and assign said first pixel value to said detected said first pixel when said adjacent second pixel is detected with said first pixel value.

22 . The system as claimed in claim 21 , wherein said first pixel value corresponds to a color gray, said second pixel value corresponds to a color white, and said third pixel value corresponds to a color black.

23 . The method as claimed in claim 21 , wherein said video processor is further operable to assign said second pixel value to one or more pixels in said first OBB having a third pixel value for said infilling of said first object contour, when two adjacent pixels in said first OBB are not detected in said third pixel value and said first pixel value.

Assignees

• Sony Corp

Inventors

• Gurbuz Sabri

Classifications

• G06T11/00 Primary

  Two-dimensional [2D] image generation · CPC title

• G06F18/22

  Matching criteria, e.g. proximity measures · CPC title

• G06V10/255

  Detecting or recognising potential candidate objects based on visual cues, e.g. shapes · CPC title

• G06K9/6215

  Physics · mapped topic

• G06K9/00624 Primary

  Physics · mapped topic