Method, device and system for producing a merged digital video sequence

Patent metadata

Field	Value
Publication number	US-9756348-B2
Application number	US-201414316227-A
Country	US
Kind code	B2
Filing date	Jun 26, 2014
Priority date	Jul 31, 2013
Publication date	Sep 5, 2017
Grant date	Sep 5, 2017

Abstract

A video processing device for producing a frame of a merged digital video sequence. A memory storing a first and a second digital video sequence depicting the same scene. The first digital video sequence has a higher pixel density than the second digital video sequence. A scaler generating an up-scaled version having the same pixel density as the first video sequence. A decoder decoding a frame of the first digital video sequence and a skip block identifying a position for a skip block and a non-skip block in the frame of the first digital video sequence. A block extractor extracting a block of pixels from the frame of the second digital video sequence based on the skip block and a block of pixels from the frame of the first digital video sequence based on the non-skip block. A merging unit merging both extracted blocks to produce the merged video sequence.

First claim

What is claimed is:

1. A method for producing a frame of a merged digital video sequence, comprising:
obtaining, using an image processor, a first digital video sequence;
analyzing, using an analysis unit, frames of the first digital video sequence to identify pixel blocks considered to be of relevance;
producing, using a mask generator, a mask indicating pixel blocks of the frames of the first digital video sequence being of relevance;
encoding, using an encoding unit using a video compression algorithm, a frame of the first digital video sequence using the mask into a frame in an encoded version of the first video sequence, wherein the pixel blocks being masked out as not being of relevance are encoded using skip blocks;
transmitting the encoded version of the first video sequence over a network to a client;
obtaining, using the image processor, a second digital video sequence, wherein the first and second digital video sequences both include at least one block of pixels that depicts approximately the same area of relevance and are not encoded as skip blocks, wherein the frames of the first digital video sequence have a higher number of pixels than the frames of the second digital video sequence;
transmitting the second digital video sequence over the network to the client;
scaling, using a scaler of the client, a frame of the second digital video sequence such that an up-scaled frame of the second digital video sequence having the same number of pixels as the frame of the first video sequence is obtained;
identifying, at the client, while decoding the encoded frame of the first digital video sequence and using a skip block identifier, a position for at least one skip block and a position for at least one non-skip block in the frame of the first digital video sequence;
extracting, using a block extractor of the client and based on the identified position for the at least one skip block in the frame of the first digital video sequence, at least one corresponding block of pixels from the up-scaled frame of the second digital video sequence;
extracting, using the block extractor and based on the identified position for the at least one non-skip block in the frame of the first digital video sequence, at least one corresponding block of pixels from the frame of the first digital video sequence; and
merging, using a merging unit of the client, the extracted block of pixels from the frame of the second digital video sequence and the extracted block of pixels from the frame of the first digital video sequence such that the frame of the merged video sequence is produced, wherein the pixels of the extracted block of pixels from the frame of the first digital video sequence are generated independently of the pixels of the extracted block of pixels from the frame of the second digital video sequence.

2. The method according to claim 1 , further comprising filtering, using a filter, the scaled frame of the second digital video sequence.

3. The method according to claim 1 , wherein the frames of the second digital video sequence is encoded using a video compression algorithm, wherein the method further comprises decoding, using the decoder, a frame of the second digital video sequence.

4. The method according to claim 1 , wherein the encoded frame of the first digital video sequence is an INTER-frame.

5. The method according to claim 1 , wherein the video compression algorithm is ITU-H.26X, ISO-MPEG, WMV, On2 or WebM compliant.

6. The method according to claim 1 , further comprising storing the frame of the merged video sequence in a memory.

7. The method according to claim 1 , further comprising play back of the frame of the merged video sequence.

8. A non-transitory computer-readable recording medium having recorded thereon a program for implementing the method according to claim 1 when executed on a device having processing capabilities.

9. A video processing device arranged to execute the program recorded on the non-transitory computer-readable recording medium according to claim 8 .

10. A system for producing a frame of a merged digital video sequence, the system comprising:
a digital video camera arranged to obtain a first digital video sequence;
an analysis unit, of the digital video camera, the analysis unit arranged to analyze frames of the first digital video sequence to identify pixel blocks considered to be of relevance;
a mask generator, of the analysis unit, the mask generator arranged to output a mask indicating pixel blocks of relevance in the frames of the first digital video sequence;
an encoding unit, of the digital video camera, the encoding unit arranged to encode a frame of the first digital video sequence using the mask indicating pixel blocks of relevance into a frame in an encoded version of the first video sequence, wherein the pixel blocks being masked out as not being of relevance are encoded using skip blocks;
the digital video camera is further arranged to transmit the encoded version of the first video sequence over a network to a client,
the digital video camera is further arranged to obtain a second digital video sequence, wherein the first and second digital video sequences both include at least one block of pixels that depicts approximately the same area of relevance and are not encoded as skip blocks wherein the frames of the first digital video sequence have a higher number of pixels than the second digital video sequence,
the digital video camera being further arranged to transmit the second digital video sequence over the network to the client,
the system further comprises a scaler at the client arranged to scale a frame of the second digital video sequence such that an up-scaled frame of the second digital video sequence is obtained, the up-scaled frame of the second digital video sequence having the same number of pixels as the frame of the first video sequence;
the system further comprises a decoder at the client arranged to decode the encoded frame of the first digital video sequence;
the system further comprises a skip block identifier arranged to, at the client, while decoding, using the decoder, the encoded frame of the first digital video sequence, identify a position for at least one skip block and a position for at least one non-skip block in the frame of the first digital video sequence;
the system further comprises a block extractor at the client arranged to extract based on the identified position for the at least one skip block in the frame of the first digital video sequence, at least one corresponding block of pixels from the up-scaled frame of the second digital video sequence and based on the identified position for the at least one non-skip block in the frame of the first digital video sequence, at least one corresponding block of pixels from the frame of the first digital video sequence; and
a merging unit at the client arranged to merge the extracted block of pixels from the frame of the second digital video sequence and the extracted block of pixels from the frame of the first digital video sequence such that the frame of the merged video sequence is produced, wherein the pixels of the extracted block of pixels from the frame of the first digital video sequence are generated independently of the pixels of the extracted block of pixels from the frame of the second digital video sequence.

11. The system according to claim 10 , further comprising an image sensor arranged to capture a video sequence of a scene; and wherein the digital video camera is arranged to process the video sequence of the scene captured by the image sensor into the first and the second digital video sequence depicting the same scene.

12. A me

Assignees

• Axis Ab

Inventors

• Lundberg Stefan

Classifications

• H04N19/29

  involving scalability at the object level, e.g. video object layer [VOL] · CPC title

• H04N19/187 Primary

  the unit being a scalable video layer · CPC title

• H04N19/167

  Position within a video image, e.g. region of interest [ROI] · CPC title

• H04N19/503

  involving temporal prediction (adaptive coding with adaptive selection between spatial and temporal predictive coding H04N19/107; adaptive coding with adaptive selection among a plurality of temporal predictive coding modes H04N19/109) · CPC title

• H04N19/176 Primary

  the region being a block, e.g. a macroblock · CPC title