Recording video from a bitstream

Patent metadata

Field	Value
Publication number	US-10819951-B2
Application number	US-201715425967-A
Country	US
Kind code	B2
Filing date	Feb 6, 2017
Priority date	Nov 30, 2016
Publication date	Oct 27, 2020
Grant date	Oct 27, 2020

Abstract

Techniques for recording video from a bitstream are described. In at least some implementations, video data generated as part of a communication session is recorded. According to various implementations, techniques described herein enable portions of an encoded bitstream of video data to be directly recorded as encoded frames and without requiring re-encoding of decoded frames.

First claim

What is claimed is:

1. A device for recording a video stream, comprising:
a processor; and
a memory in communication with the processor, the memory comprising executable instructions that, when executed by the processor, cause the processor to control the device to perform functions of:
receiving a first video stream comprising a first encoded frame and a plurality of second encoded frames following the first encoded frame in sequence;
decoding the received first video stream and outputting the decoded first video stream;
while the first video stream is being decoded and output, receiving an indication to record the received first video stream; and
in response to receiving the indication:
capturing a first decoded frame that is decoded from the first encoded frame of the first video stream;
storing, in a data storage, the second encoded frames of the first video stream without decoding and re-encoding the second encoded frames prior to storing the second encoded frames;
encoding the captured first decoded frame to generate an initial frame for a recorded second video stream; and
concatenating the stored second encoded frames to the initial frame to generate the recorded second video stream.

2. The device of claim 1 , wherein the instructions, when executed by the processor, further cause the processor to control the device to perform a function of determining, prior to capturing the first decoded frame, that the first encoded frame references a previous frame in the first video stream that is not available for decoding the first encoded frame.

3. The device of claim 1 , wherein the instructions, when executed by the processor, further cause the processor to control the device to perform a function of determining, prior to capturing the first decoded frame, that the first encoded frame is a predicted frame (p-frame) that references a previous frame in the first video stream that is not available for decoding the first encoded frame.

4. The device of claim 1 , wherein:
the instructions, when executed by the processor, further cause the processor to control the device to perform a function of determining, prior to capturing the first decoded frame, that the first encoded frame references a previous frame in the first video stream that is not available for decoding the first encoded frame, and
for determining that the first encoded frame references a previous frame, the instructions, when executed by the processor, further cause the processor to control the device to perform a function of monitoring a decoded picture buffer (DPB) of a codec that decodes the first encoded frame for information about the first encoded frame.

5. The device of claim 1 , wherein the instructions, when executed by the processor, further cause the processor to control the device to perform a function of decoding and outputting the first video stream for display.

6. The device of claim 1 , wherein, for encoding the captured first decoded frame, the instructions, when executed by the processor, further cause the processor to control the device to perform a function of encoding the first decoded frame as an Instantaneous Decoder Refresh (IDR) frame.

7. The device of claim 1 , wherein:
the first encoded frame comprises a predicted frame (P-frame), and
for encoding the captured first decoded frame, the instructions, when executed by the processor, further cause the processor to control the device to perform a function of encoding the first decoded frame as an Instantaneous Decoder Refresh (IDR) frame.

8. The device of claim 1 , wherein the instructions, when executed by the processor, further cause the processor to control the device to perform functions of:
determining that a third encoded frame following the first encoded frame of the first video stream references a previous frame that is not available for decoding the third encoded frame;
capturing a second decoded frame that is decoded from the third encoded frame;
encoding the second decoded frame as an intra-frame (I-frame); and
including the I-frame as part of the recorded second video stream.

9. The device of claim 1 , wherein;
for encoding the captured first decoded frame the instructions, when executed by the processor, further cause the processor to control the device to perform a function of encoding the first decoded frame as an Instantaneous Decoder Refresh (IDR) frame, and
the instructions, when executed by the processor, further cause the processor to control the device to perform functions of:
determining that the first encoded frame is a long-term reference (LTR) frame with an LTR number n that is non-zero; and
re-encoding the first decoded frame as an intra-frame (I-frame) with the LTR number n, and
to concatenate the stored second encoded frames to the initial frame, the instructions further include instructions that, when executed by the processor, cause the processor to control the device to perform a function of concatenating the I-frame to the initial frame.

10. The device of claim 1 , wherein the instructions, when executed by the processor, further cause the processor to control the device to perform functions of:
determining that there is a numbering gap between frame numbers for a set of frames of the stored second encoded frames and a frame number of the initial frame; and
modifying the frame numbers for the respective frames of the set of frames such that the frame numbers for the respective frames are consecutive from the frame number of the initial frame.

11. The device of claim 1 , wherein the instructions, when executed by the processor, further cause the processor to control the device to perform functions of:
determining that there is a numbering gap between frame numbers for a set of frames of the stored second encoded frames and a frame number of the initial frame; and
modifying the frame numbers for the respective frames of the set of frames such that the frame numbers for the respective frames are consecutive from the frame number of the initial frame,
wherein, to modify the frame numbers, the instructions further include instructions that, when executed by the processor, cause the processor to control the device to perform a function of modifying one or more frame number values in one or more slice headers for the set of frames.

12. The device of claim 1 , wherein, for capturing the first decoded frame, the instructions, when executed by the processor, further cause the processor to control the device to perform functions of:
determining that the first encoded frame references a previous encoded frame in the first video stream that has been decoded prior to receiving the indication; and
in response to determining that the first encoded frame references the previous encoded frame, capturing the first decoded frame from a buffer for a codec configured to decode the first video stream.

13. The device of claim 1 , wherein the first video stream comprises a long-term reference (LTR) frame.

14. A method of operating a device for recording a video stream, comprising:
receiving a first video stream comprising a first encoded frame and a plurality of second encoded frames following the first encoded frame in sequence;
decoding the received first video stream and outputting the decoded first video stream;
while the first video stream is being decoded and output, receiving an indication to record the received first video stream; and
in response to receiving the indication:
capturing a first decoded frame that is decoded from the first encoded frame of the first video stream;
storing, in a data storage, the second encoded frames of the first

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Zhou You
• Lin Chih-Lung
• Lu Mei-Hsuan
• Li Binlong
• Lee Ming-Chieh

Classifications

• H04N21/440218

  by transcoding between formats or standards, e.g. from MPEG-2 to MPEG-4 (conversion of standards in analogue television systems H04N7/01) · CPC title

• H04N21/4334

  Recording operations (recording of a television signal H04N5/76; arrangements for recording or accumulating broadcast information or broadcast-related information H04H60/27) · CPC title

• H04N5/765

  Interface circuits between an apparatus for recording and another apparatus · CPC title

• H04N7/155 Primary

  involving storage of or access to video conference sessions (tracking arrangements for later retrieval of a computer conference content or participants activities H04L12/1831) · CPC title

• H04L12/1831

  Tracking arrangements for later retrieval, e.g. recording contents, participants activities or behavior, network status · CPC title