Method, computer program product, and system for providing a sensor-based environment

Patent metadata

Field	Value
Publication number	US-2016110902-A1
Application number	US-201514675161-A
Country	US
Kind code	A1
Filing date	Mar 31, 2015
Priority date	Oct 15, 2014
Publication date	Apr 21, 2016
Grant date	—

Abstract

Method, computer program product, and system to provide an extended vision within an environment having a plurality of items, where the extended vision is based on a field of view of a person determined using a first visual sensor, and is further based on at least a second visual sensor disposed within the environment. Image information from the first and second visual sensors is associated to produce combined image information. Selected portions of the combined image information are displayed based on input provided through a user interface.

First claim

What is claimed is:

1 . A computer-implemented method to provide an extended vision within an environment having a plurality of items, the extended vision based on a field of view of a person having a first computing device coupled with a first visual sensor within the environment, the environment associated with at least a second computing device coupled with at least a second visual sensor disposed within the environment, at least one of the first and second computing devices coupled with a display device and including a user interface (UI) for traversing the extended vision, the method comprising:
analyzing first image information acquired using the first visual sensor to determine the field of view of the person, the first image information including one or more first items of the plurality of items; analyzing second image information acquired using the second visual sensor, the second image information including one or more second items of the plurality of items; based on the determined field of view, associating the first image information with the second image information to produce combined image information; and based on input provided through the UI, displaying via the display device at least portions of the combined image information to provide the extended vision.

2 . The method of claim 1 , wherein associating the first and second image information includes stitching the first and second image information.

3 . The method of claim 1 , wherein the combined image information is navigable via the display device.

4 . The method of claim 1 , wherein the first computing device is a body-worn computing device and the first visual sensor is a forward-looking camera integrated with the first computing device.

5 . The method of claim 1 , wherein the first computing device is a handheld computing device and the person manipulates an orientation of the handheld computing device to acquire the first image information.

6 . The method of claim 1 , wherein the at least a second visual sensor comprises a plurality of second visual sensors, the method further comprising:
determining that an arrangement of the plurality of second visual sensors results in at least a first deadspace area representing a portion of the environment not acquired in the image information of any of the plurality of second visual sensors; and upon determining that the field of view of the person overlaps at least part of the first deadspace area, storing the associated first image information using the second computing device.

7 . The method of claim 1 , wherein the at least a second visual sensor comprises a plurality of second visual sensors, the method further comprising:
determining that an arrangement of the plurality of second visual sensors results in at least a first overlap area representing a portion of the environment acquired in the image information of two or more of the plurality of second visual sensors; and analyzing, for each of the two or more second visual sensors corresponding to the first overlap area, corresponding image information to identify a respective number of the plurality of items; assigning, based on a largest of the numbers, the corresponding visual sensor of the two or more second visual sensors as a master view for the first overlap area; and assigning other visual sensors of the two or more second visual sensors as an auxiliary view for the first overlap area.

8 . The method of claim 6 , wherein none of the numbers of identified items meet a predetermined threshold value, the method further comprising:
upon determining that the field of view of the person overlaps at least part of the first overlap area, storing the associated first image information using the second computing device.

9 . The method of claim 1 , further comprising:
performing, in response to the input provided through the UI, at least one of reorienting and displacing the second visual sensor.

10 . A computer program product to provide an extended vision within an environment having a plurality of items, the extended vision based on a field of view of a person having a first computing device coupled with a first visual sensor within the environment, the environment associated with at least a second computing device coupled with at least a second visual sensor disposed within the environment, at least one of the first and second computing devices coupled with a display device and including a user interface (UI) for traversing the extended vision, the computer program product comprising:
a computer-readable storage medium having computer-readable program code embodied therewith, the computer-readable program code executable by one or more computer processors to perform an operation that includes:
analyzing first image information acquired using the first visual sensor to determine the field of view of the person, the first image information including one or more first items of the plurality of items;
analyzing second image information acquired using the second visual sensor, the second image information including one or more second items of the plurality of items;
based on the determined field of view, associating the first image information with the second image information to produce combined image information; and
based on input provided through the UI, displaying via the display device at least portions of the combined image information to provide the extended vision.

11 . The computer program product of claim 10 , wherein associating the first and second image information includes stitching the first and second image information.

12 . The computer program product of claim 10 , wherein the combined image information is navigable via the display device.

13 . The computer program product of claim 10 , wherein the at least a second visual sensor comprises a plurality of second visual sensors, the operation further comprising:
determining that an arrangement of the plurality of second visual sensors results in at least a first deadspace area representing a portion of the environment not acquired in the image information of any of the plurality of second visual sensors; and upon determining that the field of view of the person overlaps at least part of the first deadspace area, storing the associated first image information using the second computing device.

14 . The computer program product of claim 10 , wherein the at least a second visual sensor comprises a plurality of second visual sensors, the operation further comprising:
determining that an arrangement of the plurality of second visual sensors results in at least a first overlap area representing a portion of the environment acquired in the image information of two or more of the plurality of second visual sensors; and analyzing, for each of the two or more second visual sensors corresponding to the first overlap area, corresponding image information to identify a respective number of the plurality of items; assigning, based on a largest of the numbers, the corresponding visual sensor of the two or more second visual sensors as a master view for the first overlap area; and assigning other visual sensors of the two or more second visual sensors as an auxiliary view for the first overlap area.

15 . The computer program product of claim 14 , wherein none of the numbers of identified items meet a predetermined threshold value, the operation further comprising:
upon determining that the field of view of the person overlaps at least part of the first overlap area, storing the associated first image information using the second computing device

Assignees

• Toshiba Global Commerce Solutions Holdings Corp

Inventors

• Chirakansakcharoen Monsak Jason
• Herring Dean Frederick
• Singh Ankit
• Steiner David John

Classifications

• H04N7/181

  for receiving images from a plurality of remote sources · CPC title

• G01B11/00

  Measuring arrangements characterised by the use of optical techniques · CPC title

• G06F3/012

  Head tracking input arrangements · CPC title

• G06F3/013

  Eye tracking input arrangements (G06F3/015 takes precedence) · CPC title

• G01B11/02

  for measuring length, width or thickness (G01B11/08 takes precedence) · CPC title