Method and apparatus for adjusting holographic content and computer readable storage medium

Patent metadata

Field	Value
Publication number	US-11409241-B2
Application number	US-201916654932-A
Country	US
Kind code	B2
Filing date	Oct 16, 2019
Priority date	Oct 18, 2018
Publication date	Aug 9, 2022
Grant date	Aug 9, 2022

Abstract

A method for adjusting a holographic content includes: a pose of an electronic device and a pose of a holographic content are obtained respectively under a current spatial coordinate system; difference information between the pose of the electronic device and the pose of the holographic content is obtained; and the pose of the holographic content is adjusted if it is determined that the pose of the holographic content is to be adjusted based on the difference information.

First claim

The invention claimed is:

1. A method of adjusting a holographic content, applied to an electronic device that is an Augmented Reality (AR) mobile phone or AR glasses, the method comprising:
obtaining a pose of the electronic device, and a pose of a holographic content respectively under a current spatial coordinate system, wherein an origin of the current spatial coordinate system is a position where the electronic device is enabled, the pose of the electronic device comprises values of the electronic device at six dimensions of the current spatial coordinate system and is measured by position sensor and an attitude sensor of the electronic device, the pose of the holographic content comprises values of the holographic content at the six dimensions of the current spatial coordinate system, and the six dimensions comprises three-dimensional coordinates, a yaw angle, a roll angle, and a pitch angle;
obtaining difference information between a display area of a display screen of the electronic device determined based on the pose of the electronic device and the pose of the holographic content;
adjusting the pose of the holographic content so that the holographic content is within the display area of the display screen of the electronic device when the difference information indicates that an average difference at a plurality of dimensions of the current spatial coordinate system is greater than a preset value;
obtaining a rendered holographic content by rendering the holographic content based on an adjusted pose;
upon obtaining a collection instruction, synthesizing the rendered holographic content and a real-world image currently collected by a first visual sensor in the electronic device to generate an image; and
upon obtaining a photographing instruction, storing the generated image.

2. The method according to claim 1 , wherein the adjusting the pose of the holographic content comprises:
rotating or moving the holographic content based on a rotation and a movement of the electronic device under the current spatial coordinate system.

3. The method according to claim 1 , wherein the adjusting the pose of the holographic content comprises:
determining kinematic information of the electronic device under the current spatial coordinate system; and
obtaining a holographic content matched with the kinematic information as an adjusted holographic content.

4. The method according to claim 1 , further comprising:
detecting whether a second visual sensor in the electronic device is enabled; and
upon the second visual sensor in the electronic device being enabled, obtaining the pose of the electronic device and the pose of the holographic content respectively under the current spatial coordinate system.

5. An electronic device, wherein the electronic device is an Augmented Reality (AR) mobile phone or AR glasses comprising:
a processor; and
a memory storing processor-executable instructions;
wherein when the processor-executable instructions are invoked, the processor is configured to:
obtain a pose of the electronic device and a pose of a holographic content respectively under a current spatial coordinate system, wherein an origin of the current spatial coordinate system is a position where the electronic device is enabled, the pose of the electronic device comprises values of the electronic device at six dimensions of the current spatial coordinate system and is measured by a position sensor and an attitude sensor of the electronic device, the pose of the holographic content comprises values of the holographic content at the six dimensions of the current spatial coordinate system, and the six dimensions comprises three-dimensional coordinates, a yaw angle, a roll angle, and a pitch angle;
obtain difference information between a display area of a diplay screen of the electronic device determined based on the pose of the electronic device and the pose of the holographic content;
adjust the pose of the holographic content so that the holographic content is within the display area of the display screen of the electronic device when the difference information indicates that an average difference at a plurality of dimensions of the current spatial coordinate system is greater than a preset value,
obtaining a rendered holographic content by rendering the holographic content based on an adjusted pose;
upon obtaining a collection instruction, synthesizing the rendered holographic content and a real-world image currently collected by a first visual sensor in the electronic device to generate an image; and
upon obtaining a photographing instruction, storing the generated image.

6. The electronic device according to claim 5 , wherein upon the pose of the holographic content being adjusted, the processor is further configured to:
rotate or move the holographic content based on a rotation and a movement of the electronic device under the current spatial coordinate system.

7. The electronic device according to claim 5 , wherein upon the pose of the holographic content being adjusted, the processor is further configured to:
determine kinematic information of the electronic device under the current spatial coordinate system; and
obtain a holographic content matched with the kinematic information as an adjusted holographic content.

8. The electronic device according to claim 5 , wherein the processor is further configured to:
detect whether a second visual sensor in the electronic device is enabled; and
upon the second visual sensor in the electronic device being enabled, obtain the pose of the electronic device and the pose of the holographic content respectively under the current spatial coordinate system.

9. A non-transitory computer-readable storage medium storing computer instructions for execution by a processor of an electronic device to perform following operations, wherein the electronic device is an Augmented Reality (AR) mobile phone or AR glasses, the operations comprising:
obtaining a pose of the electronic device and a pose of a holographic content respectively under a current spatial coordinate system, wherein an origin of the current spatial coordinate system is a position where the electronic device is enabled, the pose of the electronic device comprises values of the electronic at six dimensions of the current spatial coordinate system and is measured by a position sensor and an attitude sensor of the electronic device, the pose of the holographic content comprises values of the holographic context at six dimensions of the current spatial coordinate system, and the six dimensions comprises three-dimensional coordinates, a yaw angle, a roll angle, and a pitch angle;
obtaining difference information between a display area of a screen of the electronic device determined based on the pose of the electronic device and the pose of the holographic content;
adjusting the pose of the holographic content so that the holographic content is within the display area of the display screen of the electronic device when the difference information indicates that an average difference at a plurality of dimensions of the current spatial coordinate system is greater than a preset value;
obtaining a rendered holographic content by rendering the holographic content based on an adjusted pose;
upon obtaining a collection instruction, synthesizing the rendered holographic content and a real-world image currently collected by a first visual sensor in the electronic device to generate an image; and
upon obtaining a photographing instruction, storing the generated image.

Assignees

• Beijing Xiaomi Mobile Software Co Ltd

Inventors

• Zhou Liang
• Feng Zhiyong
• Ji Jiasong

Classifications

• G02B2027/0178

  Eyeglass type (eyeglass details G02C) · CPC title

• G03H1/2294

  Addressing the hologram to an active spatial light modulator · CPC title

• G06T3/00

  Geometric image transformations in the plane of the image · CPC title

• H04N13/344

  with head-mounted left-right displays · CPC title

• H04N13/106

  Processing image signals (for multi-view video sequence encoding H04N19/597) · CPC title