Image recognition scoring visualization

Patent metadata

Field	Value
Publication number	US-10339690-B2
Application number	US-201514975648-A
Country	US
Kind code	B2
Filing date	Dec 18, 2015
Priority date	Dec 18, 2015
Publication date	Jul 2, 2019
Grant date	Jul 2, 2019

Abstract

A system and method for image processing result visualization. The method includes receiving a job, wherein the job includes a panoramic image and image recognition information, and creating a visualization based on the job, wherein the visualization includes the panoramic image with the image recognition information layered over the panoramic image.

First claim

What is claimed is:

1. A method comprising:
receiving a request to view a job in a graphical user interface;
requesting, using one or more processors, the job by sending a token related to the job to a server storing data associated with the job via a network;
receiving, using the one or more processors, the job related to the token, wherein the job includes a panoramic image and metadata;
processing, using the one or more processors, the metadata to extract image recognition information, the image recognition information comprising features of a plurality of objects depicted within the panoramic image;
creating, using the one or more processors, a visualization of the job, wherein the visualization includes the panoramic image with the image recognition information overlaid on the panoramic image to highlight an object of the plurality of objects;
determining, using the one or more processors, an identity of the object of the plurality of objects depicted within the panoramic image by matching the features of the object of the plurality of objects with a potential candidate in a product database that has a highest recognition score;
determining, using the one or more processors, a correctness of the determined identity of the object of the plurality of objects based on the matching;
updating, using the one or more processors, the visualization of the job by removing portions of the visualization that do not include the plurality of objects; and
displaying in the graphical user interface, the updated visualization of the job and a correctness indicator signaling the correctness of the object of the plurality of objects.

2. The method of claim 1 wherein the object of the plurality of objects is a recognized product and creating the visualization of the job further comprises:
highlighting, using the one or more processors, the recognized product in the panoramic image using the image recognition information; and
displaying, using the one or more processors, a product recognition list in response to a user selecting the recognized product.

3. The method of claim 2 , comprising, creating the product recognition list by:
extracting, using the one or more processors, a listing of potential candidates that share similar features with the recognized product from the image recognition information; and
displaying, using the one or more processors, the listing of potential candidates.

4. The method of claim 2 , wherein an entry in the product recognition list includes a recognition score for a corresponding potential candidate, the recognition score indicating similarity of a feature shared between the recognized product and the corresponding potential candidate.

5. The method of claim 2 , wherein the correctness indicator further comprises:
a correct visual indicator in the visualization proximate to the object of the plurality of objects responsive to determining that the identity of the object of the plurality of objects is correctly identified; or
an incorrect visual indicator in the visualization proximate to the object of the plurality of objects responsive to determining that the identity of the object of the plurality of objects is incorrectly identified.

6. The method of claim 1 , wherein the plurality of objects are recognized products, the method further comprising:
highlighting, using the one or more processors, a portion of the panoramic image based on the recognized products sharing a characteristic.

7. The method of claim 1 , wherein the plurality of objects are recognized products, the method further comprising:
displaying, using the one or more processors, a boundary around the recognized products in the panoramic image.

8. The method of claim 1 , wherein the visualization is a first visualization of a first job, the method further comprising:
creating, using the one or more processors, a second visualization of a second job;
comparing, using the one or more processors, the first visualization with the second visualization in a single display screen of a user interface; and
highlighting, using the one or more processors, differences between the first visualization and the second visualization.

9. The method of claim 8 , wherein comparing the first visualization with the second visualization further comprises displaying the first visualization and the second visualization in an animation, wherein the animation toggles between the first visualization and the second visualization.

10. A non-transitory computer program product comprising a computer usable medium including a computer readable program, wherein the computer readable program when executed on a computer causes the computer to:
receive, using one or more processors, a request to view a job in a graphical user interface;
request, using the one or more processors, the job by sending a token related to the job to a server storing data associated with the job via a network;
receive, using the one or more processors, the job related to the token, wherein the job includes a panoramic image and metadata;
process, using the one or more processors, the metadata to extract image recognition information, the image recognition information comprising features of a plurality of objects depicted within the panoramic image;
create, using the one or more processors, a visualization of the job, wherein the visualization includes the panoramic image with the image recognition information overlaid on the panoramic image to highlight an object of the plurality of objects;
determine, using the one or more processors, an identity of the object of the plurality of objects depicted within the panoramic image by matching the features of the object of the plurality of objects with a potential candidate in a product database that has a highest recognition score;
determine a correctness of the determined identity of the object of the plurality of objects based on the matching;
update, using the one or more processors, the visualization of the job by removing portions of the visualization that do not include the plurality of objects; and
display in the graphical user interface, the updated visualization of the job and a correctness indicator signaling the correctness of the object of the plurality of objects.

11. The non-transitory computer program product of claim 10 wherein the object of the plurality of objects is a recognized product and the computer readable program causing the computer to create the visualization further causes the computer to:
highlight, using the one or more processors, the recognized product in the panoramic image using the image recognition information; and
display, using the one or more processors, a product recognition list in response to a user selecting the recognized product.

12. The non-transitory computer program product of claim 11 , wherein the computer readable program when executed on a computer further causes the computer to create the product recognition list to:
extract, using the one or more processors, a listing of potential candidates that share similar features with the recognized product from the image recognition information; and
display, using the one or more processors, the listing of potential candidates.

13. The non-transitory computer program product of claim 11 , wherein the correctness indicator further comprises:
a correct visual indicator displayed in the visualization proximate to the object of the plurality of objects responsive to determining that the identity of the object of the plurality of objects is correctly identified; or
an incorrect visual indicator displayed in the visualization proximate to the object of th

Assignees

• Findlay Roland
• Griffin Michael
• Romain Tiffany
• Graham Jamey
• Ricoh Co Ltd

Inventors

• Findlay Roland
• Griffin Michael
• Romain Tiffany
• Graham Jamey

Classifications

• G06V10/17

  using hand-held instruments · CPC title

• G06V10/16

  using multiple overlapping images; Image stitching · CPC title

• G06Q30/0623

  by investigating goods or services · CPC title

• G06T11/60

  Creating or editing images; Combining images with text · CPC title

• G06Q10/087

  Inventory or stock management, e.g. order filling, procurement or balancing against orders · CPC title