Terminal viewpoint based virtual reality interaction method, and device and storage medium

Patent metadata

Field	Value
Publication number	US-12554340-B2
Application number	US-202318730672-A
Country	US
Kind code	B2
Filing date	Jan 17, 2023
Priority date	Jan 21, 2022
Publication date	Feb 17, 2026
Grant date	Feb 17, 2026

Abstract

A virtual reality interaction method and apparatus, a device and a storage medium are provided. The method includes: displaying, on a terminal, a real-time captured real-world scene image; wherein the real-world scene image includes a virtual model; extending a scene edge of the virtual model towards a screen edge of the terminal; switching a scene image displayed on the terminal from the real-world scene to a virtual scene in response to detecting that a viewpoint of the terminal crosses a plane where the scene edge of the virtual model located,; and obtaining an interactive instruction for the virtual scene and displaying an interactive effect corresponding to the interactive instruction.

First claim

What is claimed is:

1 . A virtual reality interaction method, comprising:
displaying, on a terminal, a real-time captured real-world scene image; wherein the real-world scene image includes a virtual model; extending a scene edge of the virtual model towards a screen edge of the terminal; switching a scene image displayed on the terminal from the real-world scene to a virtual scene in response to detecting that a viewpoint of the terminal crosses a plane where the scene edge of the virtual model is located; obtaining an interactive instruction for the virtual scene and displaying an interactive effect corresponding to the interactive instruction; and in response to a distance between the viewpoint of the terminal and the plane where the scene edge of the virtual model is located satisfying a preset threshold, complementing a current display image to fill an area that has been cropped.

2 . The method according to claim 1 , wherein the obtaining an interactive instruction for the virtual scene and displaying an interactive effect corresponding to the interactive instruction comprises:
detecting rotational motion data of the terminal; and displaying, on the terminal, a virtual scene image corresponding to the rotational motion data in the virtual scene.

3 . The method according to claim 2 , wherein the virtual scene image includes interactive guidance information.

4 . The method according to claim 2 , wherein the displaying, on the terminal, a virtual scene image corresponding to the rotational motion data in the virtual scene comprises:
determining a first target object corresponding to the rotational motion data in the virtual scene; obtaining depth information of the first target object; and rendering the first target object based on the depth information and displaying a rendering result on the terminal.

5 . The method according to claim 1 , wherein the obtaining an interactive instruction for the virtual scene and displaying an interactive effect corresponding to the interactive instruction comprises:
displaying, on the terminal, a first interactive effect in response to a triggering operation for a first object in second target objects in the virtual scene; and synchronously displaying the first interactive effect and a second interactive effect in response to a triggering operation for an N-th object in the second target objects in the virtual scene, wherein the first interactive effect is different from the second interactive effect, and N is a natural number greater than 1.

6 . The method according to claim 1 , wherein the obtaining an interactive instruction for the virtual scene and displaying an interactive effect corresponding to the interactive instruction comprises:
obtaining the interactive instruction for the virtual scene; obtaining a current virtual scene image and a current interactive image corresponding to the interactive instruction, wherein the current virtual scene image is a mixed image, and the current interactive image is an image to be mixed; performing a mixing operation on the current interactive image using the current virtual scene image to obtain a target virtual scene image; and displaying the target virtual scene image.

7 . The method according to claim 1 , further comprising:
switching the scene image displayed on the terminal from the virtual scene to the real-world scene in response to detecting that the viewpoint of the terminal crosses the plane where the scene edge of the virtual model is located again.

8 . The method according to claim 1 , wherein complementing a current display image comprises:
determining a target padding image based on a relative positional relationship between the viewpoint of the terminal and the plane where the scene edge of the virtual model is located; and complementing the current display image using the target padding image.

9 . The method according to claim 1 , wherein the detecting the viewpoint of the terminal crosses the plane where the scene edge of the virtual model is located comprises:
determining a first result of whether the viewpoint of the terminal crosses the plane where the scene edge of the virtual model is located, based on world coordinate positions of the scene edge of the virtual model in a previous frame and a current frame and a world coordinate position of the terminal in the current frame; determining a second result of whether the viewpoint of the terminal crosses the plane where the scene edge of the virtual model is located based on world coordinate positions of the terminal in the previous frame and the current frame and a world coordinate position of the scene edge of the virtual model in the current frame; and determining a target result of whether the viewpoint of the terminal crosses the plane where the scene edge of the virtual model is located based on at least one of the first result and the second result.

10 . The method according to claim 1 , wherein the detecting that the viewpoint of the terminal crosses the plane where the scene edge of the virtual model is located comprises:
extending the scene edge of the virtual model to completely cover the display image.

11 . An electronic device, comprising a memory and a processor, wherein the memory stores a computer program, wherein the computer program, when executed by the processor, cause the processor to:
display, on a terminal, a real-time captured real-world scene image; wherein the real-world scene image includes a virtual model; extend a scene edge of the virtual model towards a screen edge of the terminal; switch a scene image displayed on the terminal from the real-world scene to a virtual scene in response to detecting that a viewpoint of the terminal crosses a plane where the scene edge of the virtual model is located;
obtain an interactive instruction for the virtual scene and displaying an interactive effect corresponding to the interactive instruction; and
in response to a distance between the viewpoint of the terminal and the plane where the scene edge of the virtual model is located satisfying a preset threshold, complement a current display image to fill an area that has been cropped.

12 . The device according to claim 11 , wherein the processor is further caused to:
detect rotational motion data of the terminal; and display, on the terminal, a virtual scene image corresponding to the rotational motion data in the virtual scene.

13 . The device according to claim 12 , wherein the processor is further caused to:
determine a first target object corresponding to the rotational motion data in the virtual scene; obtain depth information of the first target object; and render the first target object based on the depth information and displaying a rendering result on the terminal.

14 . The device according to claim 11 , wherein the processor is further caused to:
display, on the terminal, a first interactive effect in response to a triggering operation for a first object in second target objects in the virtual scene; and synchronously display the first interactive effect and a second interactive effect in response to a triggering operation for an N-th object in the second target objects in the virtual scene, wherein the first interactive effect is different from the second interactive effect, and N is a natural number greater than 1.

15 . The device according to claim 11 , wherein the processor is further caused to:
obtain the interactive instruction for the virtual scene; obtain a current virtual scene image and a current interactive image corresponding to the interactive in

Assignees

• Beijing Zitiao Network Technology Co Ltd

Inventors

• Chen Xu
• Guan Zhenghao
• Wang Bei
• Xie Minhui
• Liang Xiaoting
• Peng Wei

Classifications

• G06F3/011 Primary

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title

• G06F2203/012

  Walk-in-place systems for allowing a user to walk in a virtual environment while constraining him to a given position in the physical environment · CPC title

• G06T19/006

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• G06F3/0346 Primary

  with detection of the device orientation or free movement in a three-dimensional [3D] space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors · CPC title