Video encoding method and apparatus, device, and storage medium

Patent metadata

Field	Value
Publication number	US-10951818-B2
Application number	US-201615775069-A
Country	US
Kind code	B2
Filing date	Nov 23, 2016
Priority date	Mar 22, 2016
Publication date	Mar 16, 2021
Grant date	Mar 16, 2021

Abstract

A method and apparatus are provided. The method includes obtaining a panoramic video formed by plural panoramic picture frames, each being formed to record a scenery of a viewing angle. The panoramic video is divided into N viewing angle video sequences, each sequence being formed by areas that are in the panoramic picture frames, that do not overlap with each other, and that have a same horizontal viewing angle. The sum of the viewing angles of the N video sequences is a total horizontal viewing angle of the panoramic picture frame, N being a positive integer. The N video sequences are respectively encoded to obtain N respective groups of encoded data, and a viewing angle is determined. A viewing angle video sequence whose viewing angle comprises the determined viewing angle is selected, and encoded data of the selected viewing angle video sequence is transmitted.

First claim

What is claimed is:

1. A method comprising:
obtaining, by at least one central processing unit (CPU), a panoramic video, the panoramic video being formed by a plurality of panoramic picture frames, each of the plurality of panoramic picture frames being a picture frame formed to record a scenery of a viewing angle;
dividing, by the at least one CPU, the panoramic video into N viewing angle video sequences, each viewing angle video sequence being formed by areas that are in the plurality of panoramic picture frames, that do not overlap with each other, and that have a same horizontal viewing angle, the sum of the horizontal viewing angles of the N viewing angle video sequences being a total horizontal viewing angle of the panoramic picture frame, N being a positive integer;
encoding, respectively by the at least one CPU, the N viewing angle video sequences to obtain N respective groups of encoded data;
determining, by the at least one CPU, a viewing angle; and
selecting, by the at least one CPU, a viewing angle video sequence, from among the N viewing angle video sequences, whose horizontal viewing angle comprises the determined viewing angle, and transmitting only encoded data of the selected viewing angle video sequence,
wherein the method further comprises:
in response to determining, at a moment T i , that the viewing angle changes, skipping, by the at least one CPU, transmitting encoded data of an i th area in the viewing angle video sequence, where i is a positive integer;
selecting, by the at least one CPU, an updated viewing angle video sequence, and transmitting encoded data of an i th area in the updated viewing angle video sequence to the user, wherein the updated viewing angle video sequence is a viewing angle video sequence whose horizontal viewing angle matches the changed viewing angle; and
updating, by the at least one CPU, i to i+1, and cyclically performing the step of transmitting encoded data of an i th area in the updated viewing angle video sequence to the user at the moment T i ,
wherein macroblocks of adjacent areas in the plurality of picture frames are encoded such that the macroblocks of adjacent areas are not dependent on each other.

2. The method according to claim 1 , wherein the dividing the panoramic video to N viewing angle video sequences comprises:
dividing, by the at least one CPU, each panoramic picture frame of the panoramic video to N areas according to a viewing angle division rule and the horizontal viewing angle; and
obtaining, by the at least one CPU, the N viewing angle video sequences by, for each video angle video sequence, combining all areas of the plurality of panoramic picture frames that have the same horizontal viewing angle to form the video angle video sequence.

3. The method according to claim 1 , wherein the method further comprises, after the selecting the viewing angle video sequence:
obtaining, by the at least one CPU, viewing angle video sequences corresponding to n consecutive horizontal viewing angles that are adjacent to the horizontal viewing angle and less than the horizontal viewing angle; and obtaining viewing angle video sequences corresponding to m consecutive horizontal viewing angles that are adjacent to the horizontal viewing angle and greater than the horizontal viewing angle, wherein n and m are both positive integers, and n+m≤N−1; and
sequentially transmitting, by the at least one CPU, encoded data of each n+m areas that are in the n+m viewing angle video sequences and that belong to a same panoramic picture frame.

4. The method according to claim 1 , wherein
in each viewing angle video sequence, encoded data of a j th area is generated according to encoded data of a (j−1) th area; or encoded data of a j th area is generated according to encoded data of a (j−1) th area and encoded data of an l th area, where j and l are both positive integers, and l>j≥2; and
encoded data of any two areas of different viewing angle video sequences are irrelevant in a time domain.

5. The method according to claim 1 , wherein, for each area of the areas that comprises at least two macroblocks,
encoded data of a first macroblock of the at least two macroblocks is generated according to encoded data of a second macroblock of the at least two macroblocks that is adjacent to the first macroblock in a space domain; and
encoded data of two macroblocks that are of different areas and that are adjacent in a space domain is irrelevant.

6. A method comprising:
transmitting, by at least one central processing unit (CPU), a viewing angle to an encoder;
receiving, by the at least one CPU, encoded data from the encoder, the encoded data being of only a viewing angle video sequence selected from among N viewing angle video sequences according to the viewing angle that was transmitted, each of the N viewing angle video sequences formed by areas that are in a plurality of panoramic picture frames that comprise a panoramic video, that do not overlap with each other, and that have a same horizontal viewing angle, the sum of the horizontal viewing angles of the N viewing angle video sequences being a total horizontal viewing angle of the panoramic picture frame, N being a positive integer; and
decoding, by the at least one CPU, the encoded data that is received;
sequentially receiving, by the at least one CPU, encoded data of each of n+m areas that are in n+m viewing angle video sequences and that belong to a same panoramic picture frame, wherein the n+m viewing angle video sequences are viewing angle video sequences corresponding to n consecutive horizontal viewing angles that are adjacent to the horizontal viewing angle and less than the horizontal viewing angle, and viewing angle video sequences corresponding to m consecutive horizontal viewing angles that are adjacent to the horizontal viewing angle and greater than the horizontal viewing angle, n and m are both positive integers, and n+m≤N−1, and
in response to determining that the viewing angle changes at a moment T i :
selecting, by the at least one CPU, encoded data of an i th area in an updated viewing angle video sequence from the buffered encoded data of the n+m areas, decoding the encoded data that is selected, and playing the obtained i th area,
wherein the updated viewing angle video sequence is a viewing angle video sequence whose horizontal viewing angle matches the changed viewing angle, and
wherein macroblocks of adjacent areas in the plurality of picture frames are encoded such that the macroblocks of adjacent areas are not dependent on each other.

7. The method according to claim 6 , wherein the decoding the encoded data comprises:
decoding, by the at least one CPU, encoded data that is received at a current moment and that is of an area in a viewing angle video sequence whose horizontal viewing angle matches the viewing angle, and playing the obtained area; and
buffering, by the at least one CPU, encoded data of n+m areas in the n+m viewing angle video sequences that is received at the current moment.

8. The method according to claim 6 , wherein the decoding the encoded data comprises:
decoding, by the at least one CPU, encoded data that is received at a current moment and that is of an area in a viewing angle video sequence whose horizontal viewing angle matches the viewing angle, and playing the obtained area; and
decoding, by the at least one CPU, encoded data of n+m areas in the n+m viewing angle video sequences that is received at the current moment, and buffering the obtained n+m areas.

9. The method according to claim 8 , wherein the method further comprises, in response to determining that the viewing angle changes at a moment T 1 :
selecting, by the at least one CPU, an i th area

Assignees

• Tencent Tech Shenzhen Co Ltd

Inventors

• Wang Yao

Classifications

• H04N23/698 Primary

  for achieving an enlarged field of view, e.g. panoramic image capture · CPC title

• H04N21/2343 Primary

  involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements {(video transcoding H04N19/40; media packet handling at the source H04L65/762)} · CPC title

• H04N21/4402

  involving reformatting operations of video signals for household redistribution, storage or real-time display {(details of conversion of video standards at pixel level H04N7/01; video transcoding H04N19/40; adapting incoming signals to the display format of the display terminal G09G5/005; media handling at the source in data packet switching networks H04L65/764)} · CPC title

• H04N19/46

  Embedding additional information in the video signal during the compression process (H04N19/517, H04N19/68, H04N19/70 take precedence) · CPC title

• H04N21/6587

  Control parameters, e.g. trick play commands, viewpoint selection · CPC title