Augmented reality data dissemination method, system and terminal and storage medium

Patent metadata

Field	Value
Publication number	US-11917265-B2
Application number	US-202016989488-A
Country	US
Kind code	B2
Filing date	Aug 10, 2020
Priority date	Jul 5, 2018
Publication date	Feb 27, 2024
Grant date	Feb 27, 2024

Abstract

This application relates to a method of processing augmented reality (AR) data at a terminal, and a storage medium, and the AR data propagation method includes: receiving AR data, the AR data carrying AR recognition data and being generated by a source terminal; displaying the AR data according to the AR recognition data; in response to receiving a user instruction, updating the AR data according to the user instruction; and sharing the updated AR data with a target terminal. With the solutions provided in this application, users can implement AR interactions with virtual world data by using a terminal with a relatively low configuration.

First claim

What is claimed is:

1. A method of processing augmented reality (AR) data at a terminal having a processor and memory storing a plurality of programs to be executed by the processor, the method comprising:
receiving AR data, the AR data carrying a reality scenario image, AR recognition data corresponding to the reality scenario image and virtual world data corresponding to the AR recognition data, the AR data being generated by an acquisition terminal;
displaying the AR data by overlay-displaying the virtual world data on the reality scenario image according to the AR recognition data;
in response to receiving a user instruction, updating the AR data according to the user instruction, further including:
updating the virtual world data according to the user instruction to obtain updated virtual world data; and
generating the updated AR data according to the reality scenario image, the updated virtual world data, and the AR recognition data; and

sharing the updated AR data with a target terminal, wherein the updated AR data is used for instructing the target terminal to extract the reality scenario image, the AR recognition data, and the updated virtual world data from the updated AR data and overlay-display the updated virtual world data on the extracted reality scenario image according to the extracted AR recognition data.

2. The method according to claim 1 , wherein the acquisition terminal generates the AR data by:
acquiring the reality scenario image;
recognizing the AR recognition data according to the reality scenario image;
obtaining the virtual world data corresponding to the AR recognition data; and
generating the AR data according to the reality scenario image, the AR recognition data, and the virtual world data.

3. The method according to claim 2 , wherein in a case that the reality scenario image comprises video frames in a video frame sequence, the AR data is AR stream data, and the AR stream data comprises the video frame sequence and further comprises AR recognition data corresponding to the video frames in the video frame sequence and virtual world data corresponding to the corresponding AR recognition data.

4. The method according to claim 3 , wherein the method further comprises:
overlaying, in a case that the video frames in the video frame sequence are played according to an order of the video frame sequence, the corresponding virtual world data on the corresponding video frames according to the corresponding AR recognition data for play.

5. The method according to claim 3 , wherein the acquiring a reality scenario image comprises:
acquiring video frames from a reality scenario frame by frame, and forming the video frames into a video frame sequence according to an acquisition time sequence;
the recognizing the AR recognition data according to the reality scenario image comprises:
recognizing the corresponding AR recognition data according to the video frames in the video frame sequence; and
the generating the AR data according to the reality scenario image, the AR recognition data, and the virtual world data comprises:
compressing the video frame sequence, the AR recognition data, and the virtual world data, to obtain the AR stream data.

6. The method according to claim 2 , wherein in a case that the reality scenario image is a reality scenario picture, the AR data is a picture; and
the generating the AR data according to the reality scenario image, the AR recognition data, and the virtual world data comprises:
adding the AR recognition data and the virtual world data to additional information of the reality scenario picture, to obtain the AR data.

7. The method according to claim 6 , wherein the acquiring a reality scenario image comprises:
obtaining an image acquisition instruction;
acquiring an image of a reality scenario in real time according to the image acquisition instruction;
recognizing AR recognition data according to the image; and
acquiring the reality scenario image in a case that the recognized AR recognition data meets a preset condition.

8. The method according to claim 1 , wherein the sharing the updated AR data with a target terminal further comprises:
transmitting the AR data to the target terminal;
receiving updated AR data returned by the target terminal;
extracting a reality scenario image, AR recognition data recognized according to the reality scenario image, and updated virtual world data from the updated AR data; and
overlay-displaying the updated virtual world data on the extracted reality scenario image according to the extracted AR recognition data.

9. The method according to claim 1 , wherein the displaying the AR data according to the AR recognition data comprises:
extracting a reality scenario image, AR recognition data recognized according to the reality scenario image, and virtual world data corresponding to the AR recognition data from the AR data;
overlaying the virtual world data on the reality scenario image according to the AR recognition data; and
displaying the reality scenario image on which the virtual world data is overlaid.

10. The method according to claim 9 , wherein in a case that the AR data is AR stream data, the extracting a reality scenario image, AR recognition data recognized according to the reality scenario image, and virtual world data corresponding to the AR recognition data from the AR data comprises:
extracting video frames comprising the reality scenario image according to a video frame time sequence from a video frame sequence in the AR stream data; and
extracting AR recognition data and virtual world data that correspond to the video frames from the AR stream data, the AR recognition data corresponding to the virtual world data.

11. The method according to claim 10 , wherein the overlaying the virtual world data on the reality scenario image according to the AR recognition data comprises:
overlaying the virtual world data corresponding to the extracted video frames on the extracted video frames according to the AR recognition data corresponding to the extracted video frames; and
the displaying the reality scenario image on which the virtual world data is overlaid comprises:
playing, according to the video frame time sequence, the video frames on which the virtual world data is overlaid.

12. The method according to claim 9 , wherein in a case that the AR data is a picture, the extracting a reality scenario image, AR recognition data recognized according to the reality scenario image, and virtual world data corresponding to the AR recognition data from the AR data comprises:
obtaining image data from the picture, to obtain the reality scenario image; and
extracting the AR recognition data recognized according to the reality scenario image and the virtual world data corresponding to the AR recognition data from additional information of the picture.

13. The method according to claim 1 , wherein the target terminal is configured to perform a plurality of operations including:
obtaining, from the updated AR data, comment information corresponding to the updated virtual world data; and
displaying the comment information in real time.

14. The method according to claim 1 , wherein the AR recognition data comprises at least one of positioning map information, plane information, light sensing information, or facial information.

15. A terminal, comprising a processor and a memory, the memory storing at least one instruction, and the instruction being loaded and executed by the processor to perform a plurality of instructions including:
receiving AR data, the AR data carrying a reality sc

Assignees

• Tencent Tech Shenzhen Co Ltd

Inventors

• Jiang Huifu
• Xue Feng
• Liu Nan
• Guo Yifan

Classifications

• H04N21/8146 Primary

  involving graphical data, e.g. 3D object, 2D graphics · CPC title

• G06T11/00

  Two-dimensional [2D] image generation · CPC title

• G06T2200/24

  involving graphical user interfaces [GUIs] · CPC title

• H04N21/44

  Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs · CPC title

• H04N21/816

  involving special video data, e.g 3D video · CPC title