Computationally efficient frame rate conversion system

Patent metadata

Field	Value
Publication number	US-10110846-B2
Application number	US-201615014304-A
Country	US
Kind code	B2
Filing date	Feb 3, 2016
Priority date	Feb 3, 2016
Publication date	Oct 23, 2018
Grant date	Oct 23, 2018

Abstract

A system for frame rate conversion of a video that includes the use of key points.

First claim

We claim:

1. A method for frame rate conversion of a video in a digital video system comprising:
(a) receiving a series of frames having a first frame rate;
(b) determining a set of sparse key points in a first frame of said series of frames;
(c) characterizing a plurality of said key points using a key point description extraction process that extracts at least one binary feature descriptor;
(d) matching each of said characterized plurality of said key points to characteristics of another frame of said series of frames using a matching process based upon the at least one binary feature descriptor;
(e) estimating a blended parametric motion model based upon the results of said matching process, including:
identification of an estimated parametric motion model of a dominant motion combined with a parametric motion model of at least one object;

(f) based upon said blended parametric motion model, computing a dense motion field;
(g) computing a new frame for said series of frames based upon said dense motion field for display with a digital video system.

2. The method of claim 1 wherein said set of key points is based upon masking at least one of channel logos and moving text.

3. The method of claim 1 wherein said set of key points include at least one of edges and corner locations.

4. The method of claim 3 wherein said set of key points is modified to increase the overall uniformity of the spatial distribution of said set of key points.

5. The method of claim 1 wherein said estimating identifies outlier key points that do not fit the parametric motion model.

6. The method of claim 5 wherein said estimating is one of: (i) random sample consensus, (ii) a Hough Transform, and (iii) a histogram-based motion estimation technique.

7. The method of claim 1 wherein said blended parametric motion model is further based on identification of an outlier motion that is not included within either of said dominant motion and said motion of said at least one object.

8. The method of claim 7 wherein said parametric motion model for said dominant motion and said parametric motion model for at least one object are combined when being blended.

9. The method of claim 8 wherein a boundary of said at least one object is modified based upon at least one of a spatial refinement and a temporal refinement.

10. The method of claim 1 wherein a boundary of said at least one object is modified based upon at least one of a spatial refinement and a temporal refinement.

11. The method of claim 10 wherein said at least one object is identified based upon clustering of said key points.

12. The method of claim 1 wherein said at least one binary feature descriptor includes local binary pattern features.

13. The method of claim 1 wherein said blended parametric motion model is estimated by combining said parametric motion models based on a measure of local motion-compensated error.

14. The method of claim 13 wherein said measure of local motion-compensated error is applied to a small block of pixels.

15. The method of claim 1 wherein said computing said dense motion field includes assigning a local motion vector to a small block of pixels.

16. The method of claim 15 wherein said local motion vector is based on said blended parametric motion model and a measure of local characteristics of said first frame.

Assignees

• Sharp Laboratories America Inc

Inventors

• Chen Xu
• Van Beek Petrus J L

Classifications

• G06V10/758

  Involving statistics of pixels or of feature values, e.g. histogram matching · CPC title

• G06V10/44

  Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components · CPC title

• G06V10/462

  Salient features, e.g. scale invariant feature transforms [SIFT] · CPC title

• G06V10/50

  by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis · CPC title

• G06T2207/20021

  Dividing image into blocks, subimages or windows · CPC title