Methods and arrangements for identifying objects

Patent metadata

Field	Value
Publication number	US-9600982-B2
Application number	US-201514839561-A
Country	US
Kind code	B2
Filing date	Aug 28, 2015
Priority date	Aug 30, 2011
Publication date	Mar 21, 2017
Grant date	Mar 21, 2017

Abstract

In some arrangements, product packaging is digitally watermarked over most of its extent to facilitate high-throughput item identification at retail checkouts. Imagery captured by conventional or plenoptic cameras can be processed (e.g., by GPUs) to derive several different perspective-transformed views—further minimizing the need to manually reposition items for identification. Crinkles and other deformations in product packaging can be optically sensed, allowing such surfaces to be virtually flattened to aid identification. Piles of items can be 3D-modeled and virtually segmented into geometric primitives to aid identification, and to discover locations of obscured items. Other data (e.g., including data from sensors in aisles, shelves and carts, and gaze tracking for clues about visual saliency) can be used in assessing identification hypotheses about an item. Logos may be identified and used—or ignored—in product identification. A great variety of other features and arrangements are also detailed.

First claim

The invention claimed is:

1. A method comprising:
receiving imagery of a packaged grocery item, captured by a portable user device camera from a vantage point;
determining that the vantage point is unsuitable for item recognition; and
following said determination that the vantage point is unsuitable for item recognition, instructing the user to move the camera to a different vantage point;
wherein the determining comprises a software-configured microprocessor either:
(a) determining that a logo is too dominant a fraction of the imagery; or
(b) determining that the imagery does not appear to span the full width or height of the item.

2. The method of claim 1 in which the determining comprises determining that the logo is too dominant a fraction of the imagery.

3. The method of claim 2 wherein the determining act includes:
identifying plural fingerprint keypoints in the received imagery;
checking a data structure to identify some of said fingerprint keypoints as being associated with a logo; and
based on said checking, determining that a number greater than 20% of the identified fingerprint keypoints are associated with a logo.

4. The mothod of claim 3 that includes determining that a number greater than 50% of the identified fingerprint keypoints are assoiated with a logo.

5. The method of claim 3 that includes deterining that a number greater than 70% of the identified fingerprint keypoint are associated with a logo.

6. The method of claim 1 in which the determining comprises determining that the imagery does not appear to span the full width or height of the item.

7. The method of claim 6 in which said determining includes analyzing the imagery and finding that the following criteria are not all met: the imagery includes a pair of complimentary edges spanning at least half of the image, with a first edge in one half of the image and a second edge in the other half of the image.

8. A smartphone comprising:
a camera;
a display screen;
a microprocessor; and
a memory, wherein the memory contains instructions that cause the microprocessor to control the smartphone to perform acts including:
receiving imagery of a packaged grocery item, captured by the camera from a vantage point;
determining that the vantage point is unsuitable for item recognition; and
following said determination that the vantage point is unsuitable for item recognition, instructing the user to move the smartphone to a different vantage point;
wherein the determining comprises either:
(a) determining that a logo is too dominant a fraction of the imagery; or
(b) determining that the imagery does not appear to span the full width or height of the item.

9. The smartphone of claim 8 in which the instructions cause the microprocessor to determine that a logo is too dominant a fraction of the imagery.

10. The smartphone of claim 9 wherein the determining comprises:
identifying plural fingerprint keypoints in the received imagery;
checking a data structure to identify some of said fingerprint keypoints as being associated with a logo; and
based on said checking, determining that a number greater than 20% of the identified fingerprint keypoints are associated with a logo.

11. The smartphone of claim 10 that includes determining that a number greater than 50% of the identified fingerprint keypoints are associated with a logo.

12. The smartphone of claim 10 that includes determining that a number greater than 70% of the identified fingerprint keypoints are associated with a logo.

13. The smartphone of claim 8 in which the instructions cause the microprocessor to determine that the imagery does not appear to span the full width or height of the item.

14. The smartphone of claim 13 in which the instructions cause the microprocessor to analyze the imagery and find that the following criteria are not all met: the imagery includes a pair of complimentary edges spanning at least half of the image, with a first edge in one half of the image and a second edge in the other half of the image.

15. A non-transitory computer readable medium containing software instructions that, when executed by a microprocessor of a camera-equipped smartphone, cause the smartphone to perform acts including:
receiving imagery of a packaged grocery item, captured by the camera from a vantage point;
determining that the vantage point is unsuitable for item recognition; and
following said determination that the vantage point is unsuitable for item recognition, instructing the user to move the smartphone to a different vantage point;
wherein the determining comprises either:
(a) determining that a logo is too dominant a fraction of the imagery; or
(b) determining that the imagery does not appear to span the full width or height of the item.

16. The medium of claim 15 in which the instructions cause the microprocessor to determine that a logo is too dominant a fraction of the imagery.

17. The medium of claim 16 in which the instructions cause the microprocessor to:
identify plural fingerprint keypoints in the received imagery;
check a data structure to identify some of said fingerprint keypoints as being associated with a logo; and
based on said check, determine that a number greater than 20% of the identified fingerprint keypoints are associated with a logo.

18. The medium of claim 17 in which the instructions cause the microprocessor to determine that a number greater than 50% of the identified fingerprint keypoints are associated with a logo.

19. The medium of claim 17 in which the instructions cause the microprocessor to determine that a number greater than 70% of the identified fingerprint keypoints are associated with a logo.

20. The medium of claim 15 in which the instructions cause the microprocessor to determine that the imagery does not appear to span the full width or height of the item.

Assignees

• Digimarc Corp

Inventors

• Macintosh Brian T

Classifications

• G06V20/10

  Terrestrial scenes (scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59) · CPC title

• G06V10/462

  Salient features, e.g. scale invariant feature transforms [SIFT] · CPC title

• G06Q30/00

  Commerce · CPC title

• G07G1/0063

  with means for detecting the geometric dimensions of the article of which the code is read, such as its size or height, for the verification of the registration · CPC title

• G06T2207/10024

  Color image · CPC title