Method for product recognition from multiple images

Patent metadata

Field	Value
Publication number	US-9495606-B2
Application number	US-201414316627-A
Country	US
Kind code	B2
Filing date	Jun 26, 2014
Priority date	Feb 28, 2014
Publication date	Nov 15, 2016
Grant date	Nov 15, 2016

Abstract

A method for product recognition from multiple images includes producing a plurality of recognition results for a plurality of input images, stitching the plurality of input images into a single stitched image; merging the plurality of recognition results using information from stitching the plurality of input images to generate a merged recognition result; and outputting the merged recognition result. The disclosure also includes systems for implementing the method.

First claim

What is claimed is:

1. A method comprising:
producing a plurality of recognition results for a plurality of input images, the recognition results including a first recognition result and a second recognition result from the plurality of input images;
stitching the plurality of input images into a single stitched image;
identifying a first bounding box associated with the first recognition result and a second bounding box associated with the second recognition result;
generating a first warped bounding box associated with the first recognition result using the first bounding box and registration data associated with the plurality of input images and a second warped bounding box associated with the second recognition result using the second bounding box and the registration data associated with the plurality of input images;
comparing the first recognition result and the second recognition result, using information from stitching the plurality of input images, the first warped bounding box, and the second warped bounding box, to determine whether the first recognition result and the second recognition result represent a same real-world object in the plurality of input images;
in response to determining that the first recognition result and the second recognition result represent the same real-world object in the plurality of input images, merging the plurality of recognition results by combining the first recognition result with the second recognition result, to generate a merged recognition result; and
outputting the merged recognition result.

2. The method of claim 1 , wherein merging the plurality of recognition results comprises:
identifying a first area for the first warped bounding box corresponding to the first recognition result;
identifying a second area for the second warped bounding box corresponding to the second recognition result;
determining an overlap area between the first area and the second area; and
comparing the overlap area with an overlap threshold.

3. The method of claim 1 , wherein merging the plurality of recognition results comprises:
identifying a first image patch for the first bounding box corresponding to the first recognition result;
identifying a second image patch for the second bounding box corresponding to the second recognition result;
determining an image similarity score between the first image patch and the second image patch; and
comparing the image similarity score with an image similarity threshold.

4. The method of claim 1 , wherein merging the plurality of recognition results comprises comparing a first label of the first bounding box corresponding to the first recognition result with a second label of the second bounding box corresponding to the second recognition result, wherein the first label and the second label respectively identify objects recognized by the first bounding box and the second bounding box in the first recognition result and the second recognition result.

5. The method of claim 1 , wherein merging the plurality of recognition results comprises comparing the first bounding box corresponding to the first recognition result and the second bounding box corresponding to the second recognition result, to a seam mask image to determine whether the first bounding box or the second bounding box falls within a visible portion of the seam mask image.

6. The method of claim 1 , wherein producing the plurality of recognition results comprises comparing a bounding box size to an object size stored in a recognition database.

7. The method of claim 1 , wherein merging the plurality of recognition results comprises:
removing a recognition result having a lower priority; and
providing the recognition result having the lower priority as an alternative recognition result.

8. The method of claim 1 , wherein merging the plurality of recognition results comprises:
combining first metadata associated with the first recognition result with second metadata associated with the second recognition result.

9. A system comprising:
a processor; and
a memory storing instructions comprising:
a recognition module stored on the memory and executable by the processor, the recognition module configured to produce a plurality of recognition results for a plurality of input images, the recognition results including a first recognition result and a second recognition result from the plurality of input images, and identify a first bounding box associated with the first recognition result and a second bounding box associated with the second recognition result;
a joint stitching module stored on the memory and executable by the processor, the joint stitching module configured to stitch the plurality of input images into a single stitched image; and
a merge module stored on the memory and executable by the processor, the merge module configured to generate a first warped bounding box associated with the first recognition result using the first bounding box and registration data associated with the plurality of input images and a second warped bounding box associated with the second recognition result using the second bounding box and the registration data associated with the plurality of input images, compare the first recognition result and the second recognition result, using information from stitching the plurality of input images, the first warped bounding box, and the second warped bounding box, to determine whether the first recognition result and the second recognition result represent a same real-world object in the plurality of input images and in response to determining that the first recognition result and the second recognition result represent the same real-world object in the plurality of input images, merge the plurality of recognition results by combining the first recognition result with the second recognition result, to generate a merged recognition result.

10. The system of claim 9 , wherein the merge module is configured to:
identify a first area for the first warped bounding box corresponding to the first recognition result;
identify a second area for the second warped bounding box corresponding to the second recognition result;
determine an overlap area between the first area and the second area; and
compare the overlap area with an overlap threshold.

11. The system of claim 9 , wherein the merge module is configured to:
identify a first image patch for the first bounding box corresponding to the first recognition result;
identify a second image patch for the second bounding box corresponding to the second recognition result;
determine an image similarity score between the first image patch and the second image patch; and
compare the image similarity score with an image similarity threshold.

12. The system of claim 9 , wherein the merge module is configured to compare a first label of the first bounding box corresponding to the first recognition result with a second label of the second bounding box corresponding to the second recognition result, wherein the first label and the second label respectively identify objects recognized by the first bounding box and the second bounding box in the first recognition result and the second recognition result.

13. The system of claim 9 , wherein the merge module is configured to compare the first bounding box corresponding to the first recognition result and the second bounding box corresponding to the second recognition result, to a seam mask image to determine whether the first bounding box or the second bounding box falls within a visible portion of the seam mask image.

14. The system of claim 9 , wherein the recogni

Assignees

• Shi Shu
• Gormish Michael J
• Ricoh Co Ltd

Inventors

• Shi Shu
• Gormish Michael J

Classifications

• G06F2218/12

  Classification; Matching · CPC title

• G06V10/10 Primary

  Image acquisition (document image scanning and transmission H04N1/00; control of digital cameras H04N23/60) · CPC title

• G06F18/254

  of classification results, e.g. of results related to same input data · CPC title

• G06F18/25

  Fusion techniques · CPC title

• G06V10/16

  using multiple overlapping images; Image stitching · CPC title