See-through operator awareness tool using synthesized viewpoints

Patent metadata

Field	Value
Publication number	US-12506853-B2
Application number	US-202217860266-A
Country	US
Kind code	B2
Filing date	Jul 8, 2022
Priority date	Jun 12, 2018
Publication date	Dec 23, 2025
Grant date	Dec 23, 2025

Abstract

The disclosure is directed a method including capturing image data of a real-world environment at a first location and a second location different than the first location. The image data includes a plurality of intersecting image data points captured at both the first location and the second location, and a representation of an area occluded by an obstacle from a point of view of a user and; generating a synthesized field of view (“FOV”) that includes the plurality of intersecting image data points such that the synthesized FOV includes an unobstructed view of the area occluded by the obstacle from the point of view of the user; and rendering the synthesized FOV as a visual display.

First claim

What is claimed is:

1 . A method, comprising:
capturing first image data of a real-world environment at a first location via a first imaging device and second image data at a second location different than the first location via a second imaging device different than the first imaging device, wherein the first imaging device and the second imaging device are movable together with a user based on a user movement, and wherein the first image data and the second image data includes a plurality of intersecting image data points captured at both the first location and the second location, and a representation of an area in the real world occluded by an obstacle from a point of view of the user; generating a synthesized field of view (“FOV”) that includes the plurality of intersecting image data points such that the synthesized FOV includes an unobstructed view of the area in the real world occluded by the obstacle from the point of view of the user, wherein the unobstructed view is mapped into the synthesized FOV as a patch including the plurality of intersecting image data points included in both the first image data captured at the first location and the second image data captured at the second location; overlaying the synthesized FOV with video data corresponding to the point of view of the user; and rendering the synthesized FOV as a visual display.

2 . The method of claim 1 , wherein the first location is on a first side of the user and the second location is on a second side of the user.

3 . The method of claim 1 , wherein the synthesized FOV is rendered on a head mounted display configured to be worn by the user.

4 . The method of claim 1 , wherein the obstacle comprises a portion of a costume worn by the user and the representation of the area occluded by the obstacle comprises a view of the real-world environment outside the costume.

5 . The method of claim 1 , further comprising generating a three-dimensional model of the real-world environment based, at least in part, on the image data.

6 . The method of claim 5 , wherein generating the three-dimensional model comprises correlating a first position of one or more image data points of the plurality of intersecting image data points captured at the first location with a second position of one or more image data points of the plurality of intersecting image data points captured at the second location.

7 . The method of claim 5 , wherein the image data includes depth data and the three-dimensional model is based at least in part on the depth data.

8 . The method of claim 1 , further comprising calculating a spatial relationship between the first location, the second location, and the user, wherein the synthesized FOV is based, at least in part, on the spatial relationship.

9 . A non-transitory computer-readable medium having executable instructions stored thereon that, when executed by a processor, cause the processor to:
capture first image data of a real-world environment at a first location via a first imaging device and second image data at a second location different than the first location via a second imaging device different than the first imaging device, wherein the first imaging device and the second imaging device are movable together with a user based on a user movement, and wherein the first image data and the second image data includes a plurality of intersecting image data points captured at both the first location and the second location, and a representation of an area occluded by an obstacle from a point of view of a user; generate a synthesized field of view (“FOV”) that includes the plurality of intersecting image data points such that the synthesized FOV includes an unobstructed view of the area occluded by the obstacle from the point of view of the user, wherein the unobstructed view is mapped into the synthesized FOV as a patch including the plurality of intersecting image data points included in both the first image data captured at the first location and the second image data captured at the second location; overlay the synthesized FOV with video data corresponding to the point of view of the user; and render the synthesized FOV as a visual display.

10 . The non-transitory computer-readable medium of claim 9 , wherein the first location is on a first side of the user and the second location is on a second side of the user.

11 . The non-transitory computer-readable medium of claim 9 , wherein the synthesized FOV is rendered on a head mounted display configured to be worn by the user.

12 . The non-transitory computer-readable medium of claim 9 , wherein the obstacle comprises a portion of a costume worn by the user and the representation of the area occluded by the obstacle comprises a view of the real-world environment outside the costume.

13 . The non-transitory computer-readable medium of claim 9 , wherein executable instructions further cause the processor to generate a three-dimensional model of the real-world environment based, at least in part, on the image data.

14 . The non-transitory computer-readable medium of claim 9 , wherein executable instructions further cause the processor to calculate a spatial relationship between the first location, the second location, and the user, wherein the synthesized FOV is based, at least in part, on the spatial relationship.

15 . A visual display system, comprising:
a display; a first imaging device; and a second imaging device different than the first imaging device, a processor in communication with the display and configured to:
receive first image data of a real-world environment captured at a first location via the first imaging device and second image data at a second location different than the first location via the second imaging device, wherein the first imaging device and the second imaging device are movable together with a user based on a user movement, and wherein the first image data and the second image data includes a plurality of intersecting image data points captured at both the first location and the second location, and a representation of an area occluded by an obstacle from a point of view of a user;
generate a synthesized field of view (“FOV”) that includes the plurality of intersecting image data points such that the synthesized FOV includes an unobstructed view of the area occluded by the obstacle from the point of view of the user, wherein the unobstructed view is mapped into the synthesized FOV as a patch including the plurality of intersecting image data points included in both the first image data captured at the first location and the second image data captured at the second location;
overlay the synthesized FOV with video data corresponding to the point of view of the user; and
render the synthesized FOV on the display.

16 . The method of claim 1 , wherein the image data comprises video data.

17 . The method of claim 1 , wherein the first imaging device and the second imaging device comprise omnidirectional cameras.

18 . The method of claim 1 , wherein the first imaging device and the second imaging device are coupled to a costume configured to be worn by the user.

19 . The visual display system of claim 15 , further comprising:
a costume configured to be worn by the user, wherein the first imaging device and the second imaging device are coupled to an exterior portion of the costume.

Assignees

• Disney Entpr Inc

Inventors

• Chapman Steven
• Arana Mark
• Goslin Michael
• Popp Joseph
• Patel Mehul

Classifications

• H04N23/54

  Mounting of pick-up tubes, electronic image sensors, deviation or focusing coils · CPC title

• H04N13/243

  using three or more two-dimensional [2D] image sensors · CPC title

• H04N13/117

  the virtual viewpoint locations being selected by the viewers or determined by viewer tracking · CPC title

• H04N13/344

  with head-mounted left-right displays · CPC title

• H04N5/272

  Means for inserting a foreground image in a background image, i.e. inlay, outlay · CPC title