Object recognition device, checkout terminal, and method for processing information

Patent metadata

Field	Value
Publication number	US-10062067-B2
Application number	US-201514590676-A
Country	US
Kind code	B2
Filing date	Jan 6, 2015
Priority date	Jan 7, 2014
Publication date	Aug 28, 2018
Grant date	Aug 28, 2018

Abstract

An object recognition device includes an operation unit configured to receive a user input about an item, a storage unit that stores image data of the item, an imaging unit configured to acquire an image of the item and generate image data therefrom, and a control unit configured to compare the generated image data with the stored image data, and cause information about updating the stored image data to be presented to a user, based on a comparison result.

First claim

What is claimed is:

1. A checkout terminal comprising:
an operation panel;
a data storage in which reference image data of each of a plurality of registered items are stored;
an imaging device configured to acquire a plurality of frame images of an item that is held within an imaging range of the imaging device and generate frame image data from each of the acquired frame images;
a display; and
a processor configured to carry out an image recognition operation and an accuracy check operation for image recognition,
the processor carrying out the accuracy check operation by:
comparing the frame image data of the plurality of frame images with the reference image data corresponding to one of the plurality of registered items that is selected by a user input made on the operation panel,
comparing the frame image data of a most-recently-acquired frame image among the plurality of frame images with the reference image data corresponding to the selected registered item,
displaying:
a sequence of the frame images that are being acquired by the imaging device in a first region of a screen on the display,
a first comparison result of the frame image data of the plurality of frame images with the reference image data corresponding to the selected registered item in a second region of the screen,
a second comparison result of the frame image data of the most-recently-acquired frame image among the plurality of frame images with the reference image data corresponding to the selected registered item in a third region of the screen,
based on the first comparison result, one of a plurality of recognition result messages on the screen indicating whether additional image data needs to be stored as the reference image data corresponding to the selected registered item, and
a guidance message on the screen for a user to move the item to improve the second comparison result, and

based on user input, updating the stored reference image data corresponding to the selected registered item based on the generated frame image data corresponding to at least one of the frame images, and
the processor carrying out the image recognition operation by:
controlling the imaging device to acquire an image of an item to be purchased and generating image data of the item to be purchased, and
identifying the item to be purchased, based on the updated reference image data.

2. The checkout terminal according to claim 1 , wherein
the processor is further configured to calculate a similarity value based on the comparison result of the frame image data of the plurality of frame images with the reference image data, and display a first of the plurality of recognition result messages recommending updating of the reference image data in a fourth region of the screen when the similarity value indicates less similarity than a first predetermined value.

3. The checkout terminal according to claim 2 , wherein
the processor is further configured to change the displayed one of the plurality of recognition result messages depending on the similarity value.

4. The checkout terminal according to claim 3 , wherein
the processor is further configured to display a second of the plurality of recognition result messages suggesting the updating in the fourth region when the similarity value indicates more similarity than a second predetermined value and less similarity than the first predetermined value, and display a third of the plurality of recognition result messages indicating that the updating is required in the fourth region when the similarity value indicates less similarity than the second predetermined value.

5. The checkout terminal according to claim 2 , wherein
the processor is further configured to display a second of the plurality of recognition result messages indicating that the updating is unnecessary in the fourth region when the similarity value indicates more similarity than the first predetermined value.

6. The checkout terminal according to claim 1 , wherein
the processor is further configured to:
calculate a similarity value between each of the generated frame image data and the reference image data,
determine the number of the frame image data of which similarity value indicates less similarity than a first predetermined value, and
display a first of the plurality of recognition result messages recommending updating of the reference image data in a fourth region of the screen when the determined number is greater than a second predetermined value.

7. The checkout terminal according to claim 6 , wherein
the processor is further configured to change the displayed one of the plurality of recognition result messages depending on the number.

8. A method for processing information for an object recognition comprising:
storing, in a data storage, reference image data of each of a plurality of registered items;
receiving, on an operation panel, a user input selecting one of the registered items;
acquiring a plurality of frame images of an item that is held within an imaging range and generating frame image data from each of the acquired frame images;
comparing the frame image data of the plurality of frame images with the reference image data;
comparing the frame image data of a most-recently-acquired frame image among the plurality of frame images with the reference image data corresponding to the selected registered item;
displaying:
a sequence of the frame images that are being acquired in a first region of a screen on a display,
a first comparison result of the frame image data of the plurality of frame images with the reference image data corresponding to the selected registered item in a second region of the screen,
a second comparison result of the frame image data of the most-recently-acquired frame image among the plurality of frame images with the reference image data corresponding to the selected registered item in a third region of the screen,
based on the first comparison result, one of a plurality of recognition result messages on the screen indicating whether additional image data needs to be stored as the reference image data corresponding to the selected registered item, and
a guidance message on the screen for a user to move the item to improve the second comparison result;

based on user input, updating the reference image data corresponding to the selected registered item based on the generated frame image data corresponding to at least one of the frame images;
controlling the imaging device to acquire an image of an item to be purchased and generating image data of the item to be purchased; and
identifying the item to be purchased, based on the updated reference image data.

9. The method according to claim 8 , further comprising:
calculating a similarity value based on the comparison result of the frame image data of the plurality of frame images with the reference image data; and
displaying a first of the plurality of recognition result messages recommending updating of the reference image data in a fourth region of the screen when the similarity value indicates less similarity than a first predetermined value.

10. The method according to claim 9 , further comprising:
changing the displayed one of the plurality of recognition result messages depending on the similarity value.

11. The method according to claim 10 , wherein
a second of the plurality of recognition result messages suggesting the updating is displayed in the fourth region when the similarity value indicates more similarity than a second predetermined value and less similarity than the first predetermined value, and a third of the plurality of recognition result messages indicating that the

Assignees

• Toshiba Tec Kk

Inventors

• Fukuda Masatsugu

Classifications

• G06Q20/208 Primary

  Input by product or record sensing, e.g. weighing or scanner processing · CPC title

• G07G1/0036 Primary

  Checkout procedures · CPC title

• G07G1/0009

  Details of the software in the checkout register, electronic cash register [ECR] or point of sale terminal [POS] · CPC title

• G07G1/12

  electronically operated · CPC title

• G07C1/00

  Registering, indicating or recording the time of events or elapsed time, e.g. time-recorders for work people (registering or indicating the working of machines or vehicles G07C3/00, G07C5/00) · CPC title