Method, computer program product, and system for providing a sensor-based environment

Patent metadata

Field	Value
Publication number	US-2016110793-A1
Application number	US-201514674922-A
Country	US
Kind code	A1
Filing date	Mar 31, 2015
Priority date	Oct 15, 2014
Publication date	Apr 21, 2016
Grant date	—

Abstract

Method, computer program product, and system of visual identification of an item selected by a person during a transaction within an environment having a plurality of items. The method includes acquiring, using a visual sensor disposed within the environment, image information that includes at least a portion of the selected item, and analyzing the image information to determine a set of possible items for classifying the selected item. The method further includes selecting, based on personal profile information associated with the person, one of the set of possible items to thereby identify the selected item.

First claim

What is claimed is:

1 . A computer-implemented method of reducing the complexity of visual identification of an item selected by a person during a transaction within an environment having a plurality of items, the method comprising:
acquiring, using a visual sensor disposed within the environment, image information that includes at least a portion of the selected item; performing a limited image analysis on the acquired image information to determine a first set of possible items for classifying the selected item; determining, from the determined first set of possible items and based on personal profile information associated with the person, a reduced second set of possible items; and performing a full image analysis on the second set of possible items to identify the selected item.

2 . The method of claim 1 , wherein the personal profile information includes one or more of historical item interaction information, personal preference information, and medical information.

3 . The method of claim 1 , wherein the limited image analysis includes:
determining one or more physical attributes of the selected item based on the image information; and determining the first set of possible items based on matching the determined one or more physical attributes.

4 . The method of claim 3 , wherein determining the reduced second set of possible items includes:
determining, for each of the first set of possible items and based on the personal profile information, a respective confidence score indicating a likelihood that the corresponding possible item was selected by the person, wherein one or more possible items of the first set of possible items are not included in the second set of possible items when the corresponding confidence scores do not meet a confidence threshold value.

5 . The method of claim 1 , wherein performing a full image analysis on the second set of possible items to identify the selected item includes:
comparing the image information with reference image information for two or more of the second set of possible items; determining an identification score for each of the two or more possible items; and selecting, from the two or more possible items, the possible item corresponding to the largest identification score.

6 . The method of claim 5 , further comprising:
determining, for each of the two or more possible items and based on the comparison of the image information with the reference image information, a respective item similarity score; and determining, for each of the two or more possible items and based on the personal profile information, a respective confidence score indicating a likelihood that the corresponding possible item was selected by the person; wherein the identification score is determined based on the confidence score and the item similarity score.

7 . The method of claim 1 , wherein the image information does not include encoded identification information that is sufficient to identify the item.

8 . A computer program product for reducing the complexity of visual identification of an item selected by a person during a transaction within an environment having a plurality of items, the computer program product comprising:
a computer-readable storage medium having computer-readable program code embodied therewith, the computer-readable program code executable by one or more computer processors to perform an operation that includes:
acquiring, using a visual sensor disposed within the environment, image information that includes at least a portion of the selected item;
performing a limited image analysis on the acquired image information to determine a first set of possible items for classifying the selected item;
determining, from the determined first set of possible items and based on personal profile information associated with the person, a reduced second set of possible items; and
performing a full image analysis on the second set of possible items to identify the selected item.

9 . The computer program product of claim 8 , wherein the personal profile information includes one or more of historical item interaction information, personal preference information, and medical information.

10 . The computer program product of claim 8 , wherein the limited image analysis includes:
determining one or more physical attributes of the selected item based on the image information; and determining the first set of possible items based on matching the determined one or more physical attributes.

11 . The computer program product of claim 10 , wherein determining the reduced second set of possible items includes:
determining, for each of the first set of possible items and based on the personal profile information, a respective confidence score indicating a likelihood that the corresponding possible item was selected by the person, wherein one or more possible items of the first set of possible items are not included in the second set of possible items when the corresponding confidence scores do not meet a confidence threshold value.

12 . The computer program product of claim 8 , wherein performing a full image analysis on the second set of possible items to identify the selected item includes:
comparing the image information with reference image information for two or more of the second set of possible items; determining an identification score for each of the two or more possible items; and selecting, from the two or more possible items, the possible item corresponding to the largest identification score.

13 . The computer program product of claim 12 , the operation further including:
determining, for each of the two or more possible items and based on the comparison of the image information with the reference image information, a respective item similarity score; and determining, for each of the two or more possible items and based on the personal profile information, a respective confidence score indicating a likelihood that the corresponding possible item was selected by the person; wherein the identification score is determined based on the confidence score and the item similarity score.

14 . The computer program product of claim 8 , wherein the image information does not include encoded identification information that is sufficient to identify the item.

15 . A system for reducing the complexity of visual identification of an item selected by a person during a transaction within an environment having a plurality of items, the method comprising:
one or more computer processors; at least a first visual sensor disposed within the environment and communicatively coupled with the one or more computer processors; and a memory containing program code which, when executed by the one or more computer processors, performs an operation that includes:
acquiring, using a visual sensor disposed within the environment, image information that includes at least a portion of the selected item;
performing a limited image analysis on the acquired image information to determine a first set of possible items for classifying the selected item;
determining, from the determined first set of possible items and based on personal profile information associated with the person, a reduced second set of possible items; and
performing a full image analysis on the second set of possible items to identify the selected item.

16 . The system of claim 15 , wherein the personal profile information includes one or more of historical item interaction information, personal preference information, and medical information.

Assignees

• Toshiba Global Commerce Solutions Holdings Corp

Inventors

• Herring Dean Frederick
• Rodriguez Adrian Xavier
• Singh Ankit

Classifications

• H04N7/181

  for receiving images from a plurality of remote sources · CPC title

• G06F3/013

  Eye tracking input arrangements (G06F3/015 takes precedence) · CPC title

• G01B11/02

  for measuring length, width or thickness (G01B11/08 takes precedence) · CPC title

• G01G19/4144

  for controlling weight of goods in commercial establishments, e.g. supermarket, P.O.S. systems · CPC title

• G01B11/00

  Measuring arrangements characterised by the use of optical techniques · CPC title