Plurality of wearable devices sharing partially obstructing view at different angles to overcome visual obstructions

Patent metadata

Field	Value
Publication number	US-9952429-B2
Application number	US-201614990152-A
Country	US
Kind code	B2
Filing date	Jan 7, 2016
Priority date	Jan 7, 2016
Publication date	Apr 24, 2018
Grant date	Apr 24, 2018

Abstract

Collaborative scene sharing for overcoming visual obstructions is provided. Video feeds of a scene are obtained from multiple devices viewing the scene from different angle. An obstruction obstructs portion(s) of the scene from view by a device. An aggregation grid that partitions views of the scene into cells is established and used in providing an aggregated view of the scene to the device. This includes identifying cell(s) of the aggregation grid for which the view by the device is obstructed by the obstruction, and providing to the device video portion(s) obtained from video feed(s) from the multiple devices. The video portion(s) correspond to the cell(s) for which the view by the device is obstructed, and the video feeds from which the video portions are obtained are from other device(s), of the multiple devices, for which the view for the cell(s) is unobstructed.

First claim

What is claimed is:

1. A computer-implemented method comprising:
obtaining a plurality of live video feeds of a scene from a plurality of wearable devices viewing the scene from different angles, wherein an obstruction at least partially obstructs a portion of the scene from view by a wearable device of the plurality of wearable devices;
establishing an aggregation grid partitioning views of the scene from the plurality of wearable devices into a plurality of cells; and
providing an aggregated view of the scene to the wearable device, the providing comprising:
identifying one or more cells of the aggregation grid for which the view by the wearable device is obstructed by the obstruction; and
providing to the wearable device one or more video portions obtained from at least one video feed of the plurality of video feeds, the one or more video portions corresponding to the one or more cells for which the view by the wearable device is obstructed, and the at least one video feed from which the one or more video portions are obtained being from at least one other wearable device, of the plurality of wearable devices, for which the view for the one or more cells is unobstructed.

2. The method of claim 1 , further comprising:
providing the established aggregation grid to the plurality of wearable devices to facilitate identification of the partitioning of the views of the scene into the plurality of cells; and
obtaining the plurality of live video feeds from the plurality of wearable devices based on the providing, wherein an obtained live video feed from the wearable device comprises video portions corresponding to cells, of the aggregation grid, for which the view by the wearable device is unobstructed.

3. The method of claim 2 , further comprising receiving from the wearable device an indication of the one or more cells of the aggregation grid for which the view by the wearable device is obstructed by the obstruction.

4. The method of claim 3 , further comprising using the received indication of the one or more cells to identify the one or more cells and provide the one or more video portions to the wearable device.

5. The method of claim 1 , further comprising continually providing the aggregated view of the scene to the wearable device over a period of time, wherein updated video portions from the at least one live video feed are continually provided to the wearable device for presentation to a user in place of the obstructed view for the one or more cells.

6. The method of claim 1 , further comprising building the aggregated view of the scene by stitching together video portions from the plurality of live video feeds, the video portions corresponding to the cells of the aggregation grid and showing unobstructed views for the cells to provide an unobstructed aggregated view of the scene.

7. The method of claim 1 , further comprising dynamically adjusting cell size of the aggregation grid based on quality of the aggregated view of the scene.

8. The method of claim 1 , wherein a live video feed of the plurality of live video feeds is obtained from a device viewing at least a portion of the scene from a vantage point of the obstruction, the at least a portion of the scene corresponding to a cell of the one or more cells for which the view by the wearable device is obstructed, and wherein at least one video portion provided to the wearable device for the cell comprises a video portion from the live video feed from the device.

9. The method of claim 1 , wherein the obtaining the plurality of live video feeds, the establishing the aggregation grid, and the providing the aggregated view of the scene are performed by the wearable device, and wherein the wearable device aggregates the view of the scene on a transparent display of the wearable device, in which the one or more video portions are provided on the transparent display for the one or more cells for which the view by the wearable device is being obstructed, and no video portions are provided on the transparent display for cells for which the view by the wearable device is unobstructed.

10. The method of claim 1 , wherein the obtaining the plurality of live video feeds, the establishing the aggregation grid, and the providing the aggregated view of the scene are performed by a server separate from the plurality of wearable devices, wherein the providing the one or more video portions comprises the server sending the one or more video portions to the wearable device.

11. The method of claim 1 , further comprising normalizing video portions of the obtained plurality of live video feeds prior to the providing the one or more video portions.

12. The method of claim 1 , further comprising:
based on detecting presence of another wearable device for participation, determining at least one cell of the aggregation grid for which a new view is desired; and
providing a suggestion of a position at which the another wearable device is to be situated such that provision of a live video feed by the another wearable device from the position provides a live video feed with video portions providing the new view.

13. The method of claim 1 , further comprising:
based on detecting an eye focus direction of a user of a first wearable device of the plurality of wearable devices, determining a target, in the scene, of the user's focus;
identifying a second wearable device, of the plurality of wearable devices, that has a view of the target from a position closer to the target than the first device; and
providing to the first wearable device at least a portion of a video feed obtained by the second wearable device from the position closer to the target, the at least a portion including a view of the target, to provide to the user with a view of the target from the position closer to the target.

14. A computer system comprising:
a memory; and
a processor in communications with the memory, wherein the computer system is configured to perform a method, the method comprising:
obtaining a plurality of live video feeds of a scene from a plurality of wearable devices viewing the scene from different angles, wherein an obstruction at least partially obstructs a portion of the scene from view by a wearable device of the plurality of wearable devices;
establishing an aggregation grid partitioning views of the scene from the plurality of wearable devices into a plurality of cells; and
providing an aggregated view of the scene to the wearable device, the providing comprising:
identifying one or more cells of the aggregation grid for which the view by the wearable device is obstructed by the obstruction; and
providing to the wearable device one or more video portions obtained from at least one video feed of the plurality of video feeds, the one or more video portions corresponding to the one or more cells for which the view by the wearable device is obstructed, and the at least one video feed from which the one or more video portions are obtained being from at least one other wearable device, of the plurality of wearable devices, for which the view for the one or more cells is unobstructed.

15. The computer system of claim 14 , wherein the method further comprises building the aggregated view of the scene by stitching together video portions from the plurality of live video feeds, the video portions corresponding to the cells of the aggregation grid and showing unobstructed views for the cells to provide an unobstructed aggregated view of the scene.

16. The computer system of claim 14 , wherein a live video feed of the plurality of live video feeds is obtained from a dev

Assignees

• IBM

Inventors

• Ekambaram Vijay
• Rakshit Sarbajit K

Classifications

• G02B2027/014

  comprising information/image processing systems · CPC title

• G02B2027/0178

  Eyeglass type (eyeglass details G02C) · CPC title

• G02B2027/0123

  comprising devices increasing the field of view · CPC title

• G02B27/0101 Primary

  characterised by optical features (G02B27/0172 takes precedence) · CPC title

• G02B2027/0138

  comprising image capture systems, e.g. camera · CPC title