Method for displaying virtual object and electronic device

Patent metadata

Field	Value
Publication number	US-11776151-B2
Application number	US-202217718734-A
Country	US
Kind code	B2
Filing date	Apr 12, 2022
Priority date	Nov 8, 2019
Publication date	Oct 3, 2023
Grant date	Oct 3, 2023

Abstract

A method for displaying a virtual object is disclosed, applied to an electronic device ( 10 ) having a display component ( 323 ) and a camera ( 324 ). The method includes: detecting an operation of starting an application by a user; in response to the operation, downloading a global submap and storing the global submap in a simultaneous localization and mapping SLAM system of the electronic device ( 10 ), where the global submap ( 123 ) is a submap corresponding to a location of the electronic device ( 10 ) in a global map; and displaying a location and a posture of the virtual object on the display component ( 323 ), where the location and the posture of the virtual object are obtained by performing pose calculation based on at least a video image collected by the camera ( 324 ) and the global submap ( 123 ).

First claim

What is claimed is:

1. A method for displaying a virtual object, performed by an electronic device having a display and a camera, wherein the method comprises:
detecting an operation by a user to start an application;
in response to the operation, downloading a global submap and storing the global submap in a simultaneous localization and mapping (SLAM) system of the electronic device, wherein the global submap is a submap corresponding to a location of the electronic device in a global map; and
displaying a location and a posture of the virtual object on the display, wherein the location and the posture of the virtual object are calculated by the SLAM system by performing pose calculation based on at least a video image collected by the camera and the global submap.

2. The method according to claim 1 , wherein the location and the posture of the virtual object are calculated by the SLAM system by performing a pose calculation at a first frequency using at least the video image collected by the camera and the global submap.

3. The method according to claim 2 , wherein the pose calculation process uses the video image collected by the camera, the global submap, and motion data collected by the electronic device, to obtain the location and the posture of the virtual object, wherein the motion data comprises motion speed data and motion direction data.

4. The method according to claim 1 , wherein the downloading a global submap in response to the operation comprises:
in response to the operation, sending, to a server, indication information indicating an initial location of the electronic device; and
receiving the global submap from the server, wherein the global submap is determined based on the initial location of the electronic device.

5. The method according to claim 4 , wherein the indication information indicating the initial location of the electronic device comprises first location fingerprint information used to indicate the initial location of the electronic device, the global submap corresponds to second location fingerprint information, and the first location fingerprint information matches the second location fingerprint information.

6. The method according to claim 2 , wherein the method further comprises:
updating a SLAM map of the SLAM system based on the location and the posture of the virtual object.

7. The method according to claim 2 , wherein the displaying a location and a posture of the virtual object on the display comprises:
displaying a first interface on the display, and displaying the video image and the virtual object in the first interface based on the location and the posture of the virtual object, and the location and the posture of the virtual object is calculated by using at least the video image collected by the camera and the global submap.

8. A method for displaying a virtual object, performed by an electronic device having a display and a camera, wherein the method comprises:
obtaining a global submap and storing the global submap in a simultaneous localization and mapping (SLAM) system of the electronic device, wherein the global submap is a submap corresponding to a location of the electronic device in a global map;
performing pose calculation based on a video image collected by the camera and the global submap, to obtain pose data of the electronic device; and
displaying the virtual object on the display based on the pose data of the electronic device.

9. The method according to claim 8 , wherein the performing pose calculation based on the video image collected by the camera and the global submap, to obtain pose data of the electronic device comprises:
performing pose calculation at a first frequency using at least the video image collected by the camera and the global submap, to obtain the pose data of the electronic device.

10. The method according to claim 8 , wherein the performing pose calculation based on the video image collected by the camera and the global submap, to obtain pose data of the electronic device comprises:
performing pose calculation based on the video image collected by the camera, the global submap, and motion data collected by the electronic device, to obtain the pose data of the electronic device, wherein the motion data comprises motion speed data and motion direction data.

11. The method according to claim 8 , wherein the obtaining the global submap of the global map comprises:
sending, to a server, first location fingerprint information used to indicate an initial location of the electronic device; and
receiving the global submap from the server, wherein the global submap corresponds to second location fingerprint information, and the first location fingerprint information matches the second location fingerprint information.

12. The method according to claim 8 , wherein the virtual object is a virtual object in a virtual reality VR scenario, an augmented reality AR scenario, or a mixed reality MR scenario.

13. An electronic device for displaying a virtual object, comprising:
a display, a camera, one or more processors, a memory, a plurality of applications, and one or more computer programs, wherein the one or more computer programs are stored in the memory, the one or more computer programs comprise instructions, and when the instructions are executed by the electronic device, the electronic device is enabled to perform the following steps:
detecting an operation by a user to start an application;
in response to the operation, downloading a global submap and storing the global submap in a simultaneous localization and mapping (SLAM) system of the electronic device, wherein the global submap is a submap corresponding to a location of the electronic device in a global map; and
displaying a location and a posture of the virtual object on the display, wherein the location and the posture of the virtual object are calculated by the SLAM system by performing pose calculation based on at least a video image collected by the camera and the global submap.

14. The electronic device according to claim 13 , wherein location and the posture of the virtual object are calculated by the SLAM system by performing a pose calculation at a first frequency using at least the video image collected by the camera and the global submap.

15. The electronic device according to claim 13 , wherein when the instructions are executed by the electronic device, the electronic device is enabled to specifically perform the following step:
using the video image collected by the camera, the global submap, and motion data collected by the electronic device, to obtain the location and the posture of the virtual object, wherein the motion data comprises motion speed data and motion direction data.

16. The electronic device according to claim 13 , wherein when the instructions are executed by the electronic device, the electronic device is enabled to specifically perform the following steps:
in response to the operation, sending, to a server, indication information indicating an initial location of the electronic device; and
receiving the global submap from the server, wherein the global submap is determined based on the initial location of the electronic device.

17. The electronic device according to claim 13 , wherein when the instructions are executed by the electronic device, the electronic device is enabled to further perform the following step:
updating a SLAM map of the SLAM system based on the location and the posture of the virtual object.

18. The electronic device according to claim 17 , wherein

Assignees

• Huawei Tech Co Ltd

Inventors

• Jiang Jun
• Lin Tao
• Su Qi

Classifications

• G06T7/70 Primary

  Determining position or orientation of objects or cameras (camera calibration G06T7/80) · CPC title

• G06T19/006 Primary

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• G06F3/011 Primary

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title

• G06T17/05

  Geographic models · CPC title

• G06F2203/012

  Walk-in-place systems for allowing a user to walk in a virtual environment while constraining him to a given position in the physical environment · CPC title