Data processing method, apparatus and electronic device

Patent metadata

Field	Value
Publication number	US-11044295-B2
Application number	US-201715845743-A
Country	US
Kind code	B2
Filing date	Dec 18, 2017
Priority date	Aug 19, 2016
Publication date	Jun 22, 2021
Grant date	Jun 22, 2021

Abstract

The embodiments of the present disclosure disclose a data processing method, apparatus and electronic device, the method including: acquiring material attribute information of a first material in a video image played on a first device terminal, wherein the video image includes a raw video image and the first material drawn on the raw video image using computer graphics; generating a transmission stream according to the raw video image and the material attribute information; and sending the transmission stream to at least one second device terminal.

First claim

What is claimed is:

1. A data processing method, comprising:
acquiring material attribute information of a first material in a video image played on a first device terminal, wherein the video image comprises a raw video image captured by the first device terminal and the first material drawn on the raw video image using computer graphics;
generating a transmission stream according to the raw video image and the material attribute information; and
sending the transmission stream to at least one second device terminal,
wherein the material attribute information comprises a drawing position of the first material to be drawn in the video image, and the drawing position is selected by the first device terminal based on a feature point of a target object that is determined from the raw video image.

2. The method according to claim 1 , wherein the generating a transmission stream according to the raw video image and the material attribute information comprises:
encoding the raw video image according to a set video encoding format, and generating a transmission stream according to the encoded raw video image and the material attribute information of the first material.

3. The method according to claim 1 , wherein the material attribute information further comprises at least one of the following: identification information of the first material and link information of the first material.

4. A non-transitory computer readable medium, storing a computer program thereon, the program, when executed by a processor, causes the processor to perform the data processing method according to claim 1 .

5. The method according to claim 1 , wherein the first material comprises one of a special effect with semantic information, a business object with a special effect containing advertisement information, and an object interacting with video audience.

6. The method according to claim 1 , wherein the determination of the drawing position according to the feature point of the target object comprises:
determining a type of the target object according to the feature point; and
determining the drawing position according to the type of the target object.

7. A data processing method, comprising:
receiving, by a second device terminal, a transmission stream from a first device terminal, the transmission stream comprising a raw video image captured by the first device terminal and material attribute information;
determining a second material that matches the material attribute information in the transmission stream; and
drawing the second material on the raw video image according to the material attribute information using computer graphics,
wherein the material attribute information comprises a drawing position of the second material to be drawn in the raw video image, and the drawing position is selected by the first device based on a feature point of a target object that is determined from the raw video image.

8. The method according to claim 7 , wherein the determining a second material that matches the material attribute information in the transmission stream comprises:
decoding the transmission stream in accordance with a video decoding format corresponding to a video encoding format of the raw video image;
determining the raw video image and the material attribute information from the decoded transmission stream; and
determining a matched second material according to the material attribute information.

9. The method according to claim 8 , wherein the transmission stream further carries video attribute information of the raw video image; and
the determining the raw video image and the material attribute information from the decoded transmission stream comprises:
determining the raw video image from the decoded transmission strea.
determining the material attribute information from a raw byte sequence payload (RBSP) of a network abstraction layer (NAL) unit; and
determining video attribute information of the raw video image from the decoded transmission stream.

10. The method according to claim 9 , wherein the determining the material attribute information from a raw byte sequence payload (RBSP) in a network abstraction layer (NAL) unit comprises:
determining the material attribute information from a supplemental enhancement information (SEI) unit of the RBSP.

11. The method according to claim 7 , wherein the drawing the second material on the raw video image according to the material attribute information using computer graphics comprises:
displaying the raw video image on a second device terminal according to the video attribute information of the raw video image;
determining a drawing position of the second material in the raw video image according to the material attribute information; and
drawing the second material on the raw video image according to the drawing position using computer graphics.

12. The method according to claim 7 , wherein the material attribute information further comprises at least one of the following: identification information of the first material and link information of the first material.

13. A non-transitory computer readable medium, storing a computer program thereon, the program, when executed by a processor, causes the processor to perform the data processing method according to claim 6 .

14. A data processing apparatus, comprising:
a processor;
an image acquisition device; and
instructions to cause the processor to perform operations, the operations comprising:
acquiring material attribute information of a first material in a video image played on a first device terminal, wherein the video image comprises a raw video image captured by the image acquisition device and the first material drawn on the raw video image using computer graphics;
generating a transmission stream according to the raw video image and the material attribute information; and
sending the transmission stream to at least one second device terminal,
wherein the material attribute information comprises a drawing position of the first material to be drawn in the video image, and the drawing position is selected by the processor based on a feature point of a target object that is determined from the video image.

15. The apparatus according to claim 14 , wherein the generating a transmission stream according to the raw video image and the material attribute information comprises:
encoding the raw video image according to a set video encoding format and generating a transmission stream according to the encoded raw video image and the material attribute information of the first material.

16. The apparatus according to claim 15 , wherein the encoding the raw video image according to a set video encoding format and generating a transmission stream according to the encoded raw video image and the material attribute information of the first material comprises:
encoding the raw video image according to an H.264 encoding format; and carrying the material attribute information in an encoded raw byte sequence payload (RBSP) of a network abstraction layer (NAL) unit to generate a transmission stream.

17. The apparatus according to claim 16 , wherein the operations further comprise:
carrying the material attribute information in a supplemental enhancement information (SEI) unit of the RBSP to generate a transmission stream.

18. A data processing apparatus, comprising:
a processor; and
instructions to cause the processor to perform operations, the operations comprising:
receiving a transmission stream, the transmission stream comprising a raw video

Assignees

• Beijing Sensetime Tech Development Co Ltd

Inventors

• Wang Lei
• Yang Fan
• Luan Qing
• Sun Lin

Classifications

• H04N21/42653 Primary

  for processing graphics · CPC title

• H04L65/61

  for supporting one-way streaming services, e.g. Internet radio · CPC title

• H04L65/70 Primary

  Media network packetisation · CPC title

• H04N21/4788

  communicating with other users, e.g. chatting {(arrangements for providing for computer conferences, e.g. chat rooms, to substation in data switching networks H04L12/1813; distributed application using peer-to-peer [P2P] networks H04L67/104)} · CPC title

• H04N21/234

  Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs · CPC title