Three-dimensional point cloud-based initial viewing angle control and presentation method and system

Patent metadata

Field	Value
Publication number	US-11836882-B2
Application number	US-202017595808-A
Country	US
Kind code	B2
Filing date	Jun 28, 2020
Priority date	Jun 28, 2019
Publication date	Dec 5, 2023
Grant date	Dec 5, 2023

Abstract

Provided are an initial view angle control and presentation method and system based on a three-dimensional point cloud and a point cloud system. A three-dimensional media stream is read and parsed. An initial viewpoint, a normal vector and a forward direction vector in the three-dimensional media stream are parsed. A user can view an initial angle, that is, a region of interest, designated by a content producer when initially consuming three-dimensional media content. Moreover, scaling, that is, scale transformation, of the three-dimensional media content is allowed in an optimized manner. In a real-time interactive scene, the view range of the user can be adjusted based on the position of the user relative to the initial viewpoint. The freedom degree of visual media consumption is fully improved according to the interactive behavior of the user, providing an immersive user experience.

First claim

What is claimed is:

1. An initial view angle control and presentation method based on a three-dimensional point cloud, comprising:
reading and parsing a three-dimensional media stream to obtain encapsulated three-diminsional media data;
determining, in the encapsulated three-dimensional media data, an initial viewpoint, a normal vector direction of the initial viewpoint and a forward direction vector of the initial viewpoint; and
presenting media content in the three-dimensional media stream based on the initial viewpoint, the normal vector direction of the initial viewpoint and the forward direction vector of the initial viewpoint.

2. The method according to claim 1 , wherein in response to the media content being rotated, the method further comprises:
changing the initial viewpoint and the normal vector direction of the initial viewpoint to form a position of the changed viewpoint and a normal vector direction of the changed viewpoint.

3. The method according to claim 1 , wherein
the forward direction vector of the initial viewpoint is a direction vector parallel to a forward direction specified by a presentation device, and
the forward direction vector of the initial viewpoint comprises x coordinate information, y coordinate information and z coordinate information of a forward direction vector endpoint other than the initial viewpoint, wherein the x coordinate information, the y coordinate information and the z coordinate information are in a coordinate system whose origin is the initial viewpoint.

4. The method according to claim 1 , wherein indication information is added to encapsulation information of the encapsulated three-dimensional media data, wherein the indication information comprises:
information one: position information of the initial viewpoint;
information two: position information of a normal vector of the initial viewpoint relative to the initial viewpoint; and
information three: information about the forward direction vector of the initial viewpoint.

5. The method according to claim 1 , wherein a normal vector of the initial viewpoint comprises x coordinate information, y coordinate information and z coordinate information of a normal vector endpoint other than the initial viewpoint, wherein the x coordinate information, the y coordinate information and the z coordinate information are in a coordinate system whose origin is the initial viewpoint.

6. The system according to claim 5 , wherein the indication information further comprises:
information six: real-time relative displacement which is position information of a real-time user position relative to the initial viewpoint; and
information seven: a view range adjusted based on the real-time user position.

7. An initial view angle control and presentation system based on a three-dimensional point cloud, comprising a processor and a memory for storing execution instructions that when executed by the processor cause the processor to perform steps in following modules:
a parsing module configured to read and parse a three-dimensional media stream to obtain encapsulated three-dimensional media data;
an initial view angle determination module configured to determine, in the encapsulated three-dimensional media data, an initial viewpoint, a normal vector direction of the initial viewpoint and a forward direction vector of the initial viewpoint; and
a three-dimensional media presentation module configured to present media content in the three-dimensional media stream based on the initial viewpoint, the normal vector direction of the initial viewpoint and the forward direction vector of the initial viewpoint.

8. The system according to claim 7 , further comprising:
a changed viewpoint determination module configured to, in response to the media content being rotated, change the initial viewpoint and the normal vector direction to determine a position of the changed viewpoint and a normal vector direction of the changed viewpoint.

9. The system according to claim 7 , wherein indication information is added to encapsulation information of the encapsulated three-dimensional media data, wherein the indication information comprises:
information one: position information of the initial viewpoint;
information two: position information of a normal vector of the initial viewpoint relative to the initial viewpoint; and
information three: information about the forward direction vector of the initial viewpoint.

10. The system according to claim 7 , wherein the indication information further comprises rotation indication information for indicating whether the media content is rotatable.

11. The system according to claim 7 , wherein the indication information further comprises information for indicating whether real-time user position interaction is allowed during media playback.

12. The system according to claim 7 , wherein
the forward direction vector of the initial viewpoint is a direction vector parallel to a forward direction specified by a presentation device, and
the forward direction vector of the initial viewpoint comprises x coordinate information, y coordinate information and z coordinate information of a forward direction vector endpoint other than the initial viewpoint, wherein the x coordinate information, the y coordinate information and the z coordinate information are in a coordinate system whose origin is the initial viewpoint.

13. The system according to claim 7 , wherein a normal vector of the initial viewpoint comprises x coordinate information, y coordinate information and z coordinate information of a normal vector endpoint other than the initial viewpoint, wherein the x coordinate information, the y coordinate information and the z coordinate information are in a coordinate system whose origin is the initial viewpoint.

14. A non-transitory computer-readable storage medium storing computer-executable instructions which, when executed by a processor, cause the processor to perform steps in following modules:
a parsing module configured to read and parse a three-dimensional media stream to obtain encapsulated three-dimensional media data;
an initial view angle determination module configured to determine, in the encapsulated three-dimensional media data, an initial viewpoint, a normal vector direction of the initial viewpoint and a forward direction vector of the initial viewpoint; and
a three-dimensional media presentation module configured to present media content in the three-dimensional media stream based on the initial viewpoint, the normal vector direction of the initial viewpoint and the forward direction vector of the initial viewpoint.

Assignees

• Univ Shanghai Jiaotong

Inventors

• Xu Yiling
• Gao Linyao
• Zhu Wenjie
• Guan Yunfeng

Classifications

• G06T7/73

  using feature-based methods · CPC title

• G06T19/20 Primary

  Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts · CPC title

• G06F3/011

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title

• G06T2210/56

  Particle system, point based geometry or rendering · CPC title

• G06T2219/2016

  Rotation, translation, scaling · CPC title