Information display method and device, terminal, and storage medium

Patent metadata

Field	Value
Publication number	US-11790612-B2
Application number	US-202117242760-A
Country	US
Kind code	B2
Filing date	Apr 28, 2021
Priority date	Mar 18, 2019
Publication date	Oct 17, 2023
Grant date	Oct 17, 2023

Abstract

An information display method, including: obtaining target information and an actual image when an information display signal is received, the actual image being an image obtained by photographing a real environment by using a camera; generating an augmented reality (AR) image according to the target information and the actual image; and displaying the AR image in a user interface, an AR element in the AR image being used for displaying the target information. In this application, an AR image is generated according to the obtained target information and actual image, and the AR image is displayed on a user interface, where the AR image includes an AR element configured to display the target information.

First claim

What is claimed is:

1. An information display method, performed by a terminal, the method comprising:
obtaining target information and an actual image in response to receiving an information display signal, the actual image being an image obtained by the terminal by photographing a real environment with a camera;
generating an augmented reality (AR) image according to the target information and the actual image, comprising:
recognizing at least one plane in a three-dimensional world corresponding to the actual image, and constructing an AR environment according to the at least one plane;
changing the at least one plane of the AR environment according to changes of a position or an angle of the camera by using a dynamic tracking algorithm; and
adding an AR element to a specified position in the AR environment to overlay the AR element on the actual image, so as to generate the AR image;

displaying the AR image in a user interface, the AR element in the AR image being used for displaying the target information; and
receiving a trigger signal on the user interface.

2. The method according to claim 1 , wherein
the target information comprises video information, and the AR element comprises at least one of a television element, a projection screen element, a display window element, or a billboard element.

3. The method according to claim 1 , wherein
the target information comprises image-text information, and the AR element comprises at least one of a wall element, a bulletin board element, a picture frame element, a desk element, or an art gallery element.

4. The method according to claim 1 , wherein generating the AR image according to the target information and the actual image further comprises:
loading an AR model; and
generating the AR element according to the target information and the AR model.

5. The method according to claim 4 , wherein loading the AR model comprises:
loading a first AR model when a type of the target information is video information, the first AR model being any one of a television model, a projection screen model, a display window model, or a billboard model; and
loading a second AR model when the type of the target information is image-text information, the second AR model being any one of a wall model, a bulletin board model, a picture frame model, a desk model, and an art gallery model.

6. The method according to claim 4 , wherein loading the AR model comprises:
sending information of hardware capability of the terminal to a server, the hardware capability of the terminal being a capability of computing resources applicable to the AR model to be loaded by the terminal; and
loading a third AR model, the third AR model comprising an AR model configured by the server according to the hardware capability of the terminal.

7. The method according to claim 4 , wherein generating the AR element according to the target information and the AR model comprises:
obtaining a source file corresponding to the target information from a server;
converting a format of the source file into an AR format, the AR format being a format applicable to the AR environment; and
inputting the source file in the AR format to the AR model to generate the AR element.

8. The method according to claim 7 , wherein converting the format of the source file into the AR format comprises:
determining a uniform resource locator (URL) of a picture from the source file in a Hyper Text Markup Language (HTML) format through regular expression matching when a type of the target information is image-text information; and
downloading the picture according to the URL of the picture and determining the downloaded picture as the source file in the AR format.

9. The method according to claim 7 , wherein converting the format of the source file into the AR format comprises:
encapsulating the source file in a video stream format into a SKScene when a type of the target information is video information, and determining the SKScene as the source file in the AR format.

10. The method according to claim 4 , wherein recognizing at least one plane in the three-dimensional world corresponding to the actual image, and constructing the AR environment according to the at least one plane comprises:
recognizing the at least one plane in the three-dimensional world corresponding to the actual image by using a plane detection algorithm;
integrating the at least one plane by using a plane integration algorithm to obtain a plane recognition result in the actual image; and
constructing a plane of the AR environment according to the plane recognition result.

11. The method according to claim 4 , wherein the method further comprises:
controlling at least one of the position of the AR element or display of the target information in the AR element according to the trigger signal.

12. The method according to claim 11 , wherein the method further comprises:
transforming first coordinates of the trigger signal on the user interface into second coordinates in the AR environment; and
controlling the position of the AR element according to the trigger signal when the second coordinates correspond to the AR element.

13. The method according to claim 1 , wherein obtaining the target information and the actual image in response to receiving the information display signal comprises:
displaying an interface of information list, the interface of information list displaying a plurality of information control units, and each information control unit being configured for displaying one piece of information;
receiving a selection signal on a target information control unit in the plurality of information control units;
obtaining the target information corresponding to the target information control unit; and
obtaining the actual image by using the camera after receiving the selection signal.

14. The method according to claim 13 , wherein obtaining the actual image by using the camera comprises:
determining whether access to the camera is permitted; and
enabling the camera and obtaining the actual image when the access to the camera is permitted.

15. A terminal device, comprising a processor and a memory, the memory storing computer-readable instructions, the computer-readable instructions, when executed by the processor, causing the terminal device to perform the method of claim 1 .

16. A non-transitory computer-readable storage medium, storing computer-readable instructions, the computer-readable instructions, when executed by one or more processors of a terminal device, causing the terminal devices to perform the method of claim 1 .

17. An information display apparatus comprising:
a user interface, configured to receive an information display signal;
a camera, configured to capture an actual image by photographing a real environment;
an image processor, configured to generate an AR image according to a target information and the actual image by:
recognizing at least one plane in a three-dimensional world corresponding to the actual image, and constructing an AR environment according to the at least one plane;
changing the at least one plane of the AR environment according to changes of a position or an angle of the camera by using a dynamic tracking algorithm; and
adding an AR element to a specified position in the AR environment to overlay the AR element on the actual image, so as to generate the AR image; and

a display, configured to display the AR image, the AR element in the AR image being used for displaying the target information.

18. The informat

Assignees

• Tencent Tech Shenzhen Co Ltd

Inventors

• Xiu Haikun
• Ding Haifeng
• Zhang Yan

Classifications

• G06T19/006 Primary

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• G06F16/955

  using information identifiers, e.g. uniform resource locators [URL] · CPC title

• H04N13/117

  the virtual viewpoint locations being selected by the viewers or determined by viewer tracking · CPC title

• H04N13/139

  Format conversion, e.g. of frame-rate or size · CPC title