Method and system to combine video feeds into panoramic video

Patent metadata

Field	Value
Publication number	US-12249042-B2
Application number	US-202217701958-A
Country	US
Kind code	B2
Filing date	Mar 23, 2022
Priority date	Mar 23, 2022
Publication date	Mar 11, 2025
Grant date	Mar 11, 2025

Abstract

A computer implemented method, system and computer program product are provided that receive video feeds from corresponding cameras distributed at different positions and orientations throughout a physical environment, and obtain location data associated with the cameras. The location data is indicative of at least one of the position or orientation for the corresponding camera. The method, system and computer program product identify, from the multiple cameras, a camera combination that includes first and second cameras that have overlapping fields of view (FOV) based on the location data and form a panoramic video of at least a portion of the environment by combining the content from the first and second cameras identified based on the location data.

First claim

What is claimed is:

1. A computer implemented method, comprising:
under control of one or more processors configured with executable instructions,
receiving video feeds from corresponding cameras distributed at different positions and orientations throughout a physical environment;
obtaining location data associated with the cameras, the location data indicative of at least one of the position or the orientation for the corresponding cameras;
obtaining a supplemental image that includes each of the corresponding cameras and is captured by a supplemental camera;
identifying the position or the orientation of each of the corresponding cameras based on the supplemental image;
identifying, from the multiple cameras, a camera combination that includes first and second cameras that have overlapping fields of view (FOV) based on the position or the orientation of each of the corresponding cameras; and
forming a panoramic video of at least a portion of the environment by combining the content from the first and second cameras identified based on the position or the orientation of each of the corresponding cameras.

2. The method of claim 1 , wherein the location data is obtained independent of content of the video feeds from the cameras.

3. The method of claim 1 , wherein the location data is obtained without comparing the content from any of the video feeds with one another.

4. The method of claim 1 , wherein at least one of the multiple cameras moves within the environment while capturing the corresponding video feeds.

5. The method of claim 1 , further comprising:
receiving distance information from range sensors, the distance information indicative of a distance from the corresponding range sensor to objects within the FOV of the corresponding cameras; and
analyzing the distance information associated with the cameras to calculate the location data for the corresponding cameras.

6. The method of claim 5 , further comprising comparing the distance information associated with the first and second cameras to calculate the position and orientation of at least the second camera relative to at least one of i) a reference origin and orientation or ii) the position and orientation of the first camera.

7. The method of claim 1 , wherein the obtaining the location data further comprises calculating the location data based on information from at least one of geolocation device or gyroscopic devices associated with the cameras.

8. The method of claim 7 , further comprising receiving at least one of time-of-flight (ToF) or distance information from the geolocation devices associated with the cameras; and analyzing the at least one of ToF or distance information associated with the cameras to calculate the location data for the corresponding cameras.

9. The method of claim 1 , further comprising:
presenting a graphical representation of the physical environment on a display, the representation including camera graphical representations of the cameras;
receiving at least one of a position or orientation adjustment input at a user interface indicating an adjustment to at least one of a position or orientation of the camera graphical representation on the display; and
utilizing at least one of the position or orientation information to calculate the location data.

10. The method of claim 1 , wherein the panoramic video represents an interactive augmented reality environment.

11. A system, comprising:
a video input configured to receive video feeds from corresponding cameras distributed at different positions and orientations throughout a physical environment;
a processor; and
a memory storing instructions accessible by the processor, wherein, responsive to execution of the instructions, the processor configured to:
obtain location data associated with the cameras, the location data indicative of at least one of the position or orientation for the corresponding cameras;
obtaining a supplemental image captured by a supplemental camera;
identifying each of the corresponding cameras based on the supplemental image;
identifying the position or the orientation of each of the corresponding cameras based on the supplemental image;
identify, from the multiple cameras, a camera combination that includes first and second cameras that have overlapping fields of view (FOV) based on the position or the orientation of each of the corresponding cameras; and
form a panoramic video of at least a portion of the environment by combining the content from the first and second cameras identified based on the position or the orientation of each of the corresponding cameras.

12. The system of claim 11 , wherein the processor is further configured to obtain the location data without comparing the content from any of the video feeds with one another.

13. The system of claim 11 , further comprising the multiple cameras, the multiple cameras configured to remain stationary within the environment while capturing the corresponding video feeds.

14. The system of claim 11 , wherein the processor is further configured to:
receive distance information from range sensors, the distance information indicative of a distance from the corresponding range sensor to objects within the FOV of the corresponding cameras; and
analyze the distance information associated with the cameras to calculate the location data for the corresponding cameras.

15. The system of claim 14 , wherein the processor is further configured to: compare the distance information associated with the first and second cameras to calculate the position and orientation of at least the second camera relative to at least one of i) a reference origin and orientation or ii) the position and orientation of the first camera.

16. The system of claim 11 , wherein the processor is further configured to: receive at least one of time-of-flight (ToF) or distance information from geolocation devices associated with the cameras; and analyze the at least one of ToF or distance information associated with the cameras to calculate the location data for the corresponding cameras.

17. The system of claim 11 , further comprising:
a display configured to present a graphical representation of the physical environment, the representation including camera graphical representations of the cameras; and
a user interface configured to receive at least one of a position or orientation adjustment input indicating an adjustment to at least one of a position or orientation of the camera graphical representation on the display, the processor further configured to utilize at least one of the position or orientation information to calculate the location data.

18. A computer program product comprising a non-signal computer readable storage medium comprising computer executable code to:
receive video feeds from corresponding cameras distributed at different positions and orientations throughout a physical environment;
obtain location data associated with the cameras, the location data indicative of at least one of the position or orientation for the corresponding cameras;
obtaining a supplemental image captured by a supplemental camera;
identifying each of the corresponding cameras based on the supplemental image;
providing a user an input to designate which video feed of the video feeds corresponds to each of the corresponding cameras identified;
identifying the position or the orientation of each of the corresponding cameras based on the supplemental image;
identify, from the multiple cameras, a camera combination that includes first and second camer

Assignees

• Lenovo Singapore Pte Ltd

Inventors

• Wang Mengnan
• Nicholson John W
• Cromer Daryl C
• Locker Howard

Classifications

• G03B37/04

  with cameras or projectors providing touching or overlapping fields of view · CPC title

• H04N7/181

  for receiving images from a plurality of remote sources · CPC title

• H04N23/66

  Remote control of cameras or camera parts, e.g. by remote control devices · CPC title

• H04N23/62

  Control of parameters via user interfaces · CPC title

• H04N23/90

  Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums · CPC title