360-degree video for large scale navigation with 3d interactable models

Patent metadata

Field	Value
Publication number	US-2022222938-A1
Application number	US-202217712533-A
Country	US
Kind code	A1
Filing date	Apr 4, 2022
Priority date	Jan 31, 2020
Publication date	Jul 14, 2022
Grant date	—

Abstract

A method and system for authoring a three-dimensional (3D) presentation can involve dividing a scene into a grid of equals-sized segments or non-equally sized segments, wherein one or more 360-degree cameras record the scene from a center of each segment, and a size of the each segment is proportional to a complexity of an environment captured in the scene. One or more objects can be created within the scene for a user interaction with a 3D presentation of the scene in the context of a video stream. A logical center of the object(s) can be used as a center point for recording a 360-degree video by the one or more 360-degree cameras for use as a low fidelity background video when the user interacts with the 3D presentation of the scene. The 3D presentation can be based on a 3D model.

First claim

1 - 20 . (canceled)

21 . A method of authoring a three-dimensional (3D) presentation, comprising:
dividing a scene into a grid of segments, wherein at least one 360-degree camera records the scene from a center of each segment among the segments, wherein a size of the each segment is proportional to a complexity of an environment captured in the scene; creating at least one object within the scene for a user interaction with the 3D presentation of the scene in a context of a video stream; and navigating between a plurality of hotspots in the 3D presentation for the user interaction with the at least one object using a teleporting method that translates a position of a user from a current hotspot among the plurality of hotspots and provides a motion blur as the video stream is transitioned to a new hotspot among the plurality of hotspots, which becomes a new position of the user.

22 . The method of claim 21 wherein the step of navigating comprising:
seeing construction or functionality of the object,
allowing decomposing the object,
seeing critical parameters that the object represents, configure the object, and troubleshoot the object.

23 . The method of claim 21 wherein the navigating around the scene includes interaction of avatars of one or more users in a virtual space of the 3D presentation.

24 . The method of claim 21 further comprising using a logical center of the at least one object as a center point ‘for recording a 360-degree video by the at least one 360-degree camera for use as a low fidelity background video when the user interacts with the 3D presentation of the scene, wherein the 3D presentation is based on a 3D.

25 . The method of claim 21 wherein each hotspot among the plurality of hotspots comprises a position of the at least one 360-degree camera.

26 . The method of claim 21 further comprising overlaying an asset at the new hotspot and in front of an image of the at least one object.

27 . The method of claim 26 further comprising:
if the user interacts with the asset, fading out the 360-degree video and replacing the 360-degree video with the low fidelity background video, which is associated with the at least one object;
simultaneously constructing the 3D model of the at least one object in the low fidelity background video;
fading into a position that the at least one object had occupied in the 360-degree video; and
allowing the user to interact with the 3D model of the at least one object, including allowing the user to freely move within the each segment and around the 3D model.

28 . The method of claim 26 further comprising:
after the user has completed an interaction with the 3D model, returning the user to an original hotspot among the hotspots from which the user started when interacting with the 3D model;
allowing the user to interact with a graphically displayed button at the original position that takes the user back to the 360-degree video;
fading out the low fidelity background video and the 3D model from the scene; and
deleting the 3D model and fading a high-fidelity 360-degree video into a view of the user.

29 . The method of claim 21 wherein the segments comprise equal-sized segments or non-equally sized segments.

30 . The method of claim 29 further comprising allowing the non-equally sized segments to be stitched together.

31 . A system of authoring a three-dimensional (3D) presentation, comprising:
at least one processor; and a non-transitory computer-usable medium embodying computer program code, the non-transitory computer-usable medium capable of communicating with the at least one processor, the computer program code comprising instructions executable by the at least one processor and configured for: dividing a scene into a grid of segments, wherein at least one 360-degree camera records the scene from a center of each segment among the segments, wherein a size of the each segment is proportional to a complexity of an environment captured in the scene; creating at least one object within the scene for a user interaction with a 3D presentation of the scene in a context of a video stream; and navigate between a plurality of hotspots in the 3D presentation for the user interaction with the at least one object using a teleporting method that translates a position of a user from a current hotspot among the plurality of hotspots and provides a motion blur as the video stream is transitioned to a new hotspot among the plurality of hotspots, which becomes a new position of the user.

32 . The system of claim 31 wherein the step of navigating comprising:
seeing construction or functionality of the object,
allowing decomposing the object,
seeing critical parameters that the object represents, configure the object, and troubleshoot the object.

33 . The method of claim 31 wherein the navigating around the scene includes interaction of avatars of one or more users in a virtual space of the 3D presentation.

34 . The system of claim 31 wherein the instructions are further configured to use a logical center of the at least one object as a center point for recording a 360-degree video by the at least one 360-degree camera for use as a low fidelity background video when the user interacts with the 3D presentation of the scene, wherein the 3D presentation is based on a 3D model.

35 . The system of claim 33 wherein each hotspot among the plurality of hotspots comprises a position of the at least one 360-degree camera.

36 . The system of claim 35 wherein the instructions are further configured for overlaying an asset at the new hotspot and in front of an image of the at least one object.

37 . The system of claim 36 wherein the instructions are further configured for:
if the user interacts with the asset, fading out the 360-degree video and replacing the 360-degree video with the low fidelity background video, which is associated with the at least one object;
simultaneously constructing the 3D model of the at least one object in the low fidelity background video;
fading into a position that the at least one object had occupied in the 360-degree video; and
allowing the user to interact with the 3D model of the at least one object, including allowing the user to freely move within the each segment and around the 3D model.

38 . The system of claim 36 wherein the instructions are further configured for:
after the user has completed an interaction with the 3D model, returning the user to an original hotspot among the hotspots from which the user started when interacting with the 3D model;
allowing the user to interact with a graphically displayed button at the original position that takes the user back to the 360-degree video;
fading out the low fidelity background video and the 3D model from the scene; and
deleting the 3D model and fading a high-fidelity 360-degree video into a view of the user.

39 . The system of claim 31 wherein the segments comprise equal-sized segments or non-equally sized segments.

40 . A system for authoring a three-dimensional (3D) presentation, comprising:
at least one 360-degree camera that records a scene that is divided into a grid of equal-sized segments, wherein the at least one 360-degree camera records the scene from a center of each segment among the equal-sized segments, and a size of the each segment is proportional to a complexity of an environment captured in the scene; a 3D presentation, wherein at least one object is created within the scene for a user interact

Assignees

• Honeywell Int Inc

Inventors

• Phillips Mark
• Kaakani Ziad
• Urso Jason

Classifications

• G06T19/003 Primary

  Navigation within 3D models or images · CPC title

• G06V20/647

  by matching two-dimensional images to three-dimensional objects · CPC title

• G06V20/20 Primary

  in augmented reality scenes · CPC title

• G06F3/011

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title

• G06T19/006

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title