Using available data to assist in object recognition

Patent metadata

Field	Value
Publication number	US-9600720-B1
Application number	US-201414218216-A
Country	US
Kind code	B1
Filing date	Mar 18, 2014
Priority date	Mar 18, 2014
Publication date	Mar 21, 2017
Grant date	Mar 21, 2017

Abstract

Processes such as image matching, computer vision, and object recognition can utilize additional data, such as spatial data, to attempt to improve the accuracy of the results of those processes. For example, a computing device acquiring scene data including a representation of an object can also determine spatial data (e.g., location and orientation data). By determining the spatial data, a set of potential matches can be found which can help to more quickly and accurately identify the object based on one or more objects known to be at a corresponding position. The data acquired by the computing device can also be used to update matching data stored for that location, which can assist with subsequent processing.

First claim

What is claimed is:

1. A computing system, comprising:
at least one processor;
memory including instructions that, when executed by the at least one processor, cause the computing system to:
in a first time period:
obtain first image data acquired using a camera of a first device;
compare at least a portion of the first image data to second image data previously acquired by at least one second device;
identify, using a first confidence threshold, a first previously-acquired image of the second image data, the first previously-acquired image matching at least a portion of the first image data;
determine the first device is approaching a global position;
obtain third image data associated with the global position, the third image data acquired by the at least one second device;

in a second time period after the first time period:
obtain fourth image data acquired using the camera, the fourth image data including a representation of an object;
compare at least a portion of the fourth image data to the third image data;
identify, using a second confidence threshold, a second previously-acquired image of the third image data, the second previously-acquired image matching at least a portion of the fourth image data, the second confidence threshold being lower than the first confidence threshold; and
provide, to the first device, at least one of the second previously-acquired image or information associated with the second previously-acquired image, wherein the information associated with the second previously-acquired image includes actionable content causing the computing system to provide a purchasing option for an item related to the object.

2. The computer system of claim 1 , wherein the instructions when executed further cause the computing system to:
identify at least one additional previously-acquired image of the third image data; and
provide the at least one additional previously-acquired image for display on the first device.

3. The computer system of claim 1 , wherein the instructions when executed further cause the computing system to:
compare at least one quality parameter for the fourth image data and the third image data; and
identify the second previously-acquired image based at least in part upon the at least one quality parameter.

4. The computer system of claim 1 , wherein the instructions when executed further cause the computing system to:
compare the fourth image data to the third image data using at least one image matching algorithm until a set of image features of the second previously-acquired image is determined to match a set of image features of the fourth image data with at least a minimum level of confidence.

5. A computer-implemented method, comprising:
in a first time period:
obtaining first image data acquired using a camera of a first device;
comparing at least a portion of the first image data to second image data previously-acquired by at least one second device;
identifying, using a first confidence threshold, a first previously-acquired image of the second image data, the first previously-acquired image matching at least a portion of the first image data;
determining that the first device is approaching a global position;
obtaining third image data associated with the global position, the third image data acquired by at least one second device;

in a second time period after the first time period:
obtaining fourth image data using the camera, the fourth image data including a representation of an object;
comparing the fourth image data to the third image data;
identifying, using a second confidence threshold, a second previously-acquired image of the third image data, the second previously-acquired image matching at least a portion of the fourth image data, the second confidence threshold being lower than the first confidence threshold; and
providing actionable content associated with the object, the actionable content enabling at least one action to be performed that is related to the object.

6. The computer-implemented method of claim 5 , wherein comparing the fourth image data to the third image data includes at least one of:
performing an object recognition process, or
sending at least a portion of the third image data to the first device for performing the object recognition process.

7. The computer-implemented method of claim 5 , wherein determining that the first device is approaching the global position further comprises:
determining one or more locations within an angular view of the camera at a location specified by location data and an orientation specified by orientation data.

8. The computer-implemented method of claim 5 , wherein the actionable content includes at least one purchasing option for an item related to the object.

9. The computer-implemented method of claim 5 , further comprising:
performing a matching process against media data using the fourth image data; and
adding at least a portion of the fourth image data to the third image data in response to the object being recognized using the media data.

10. The computer-implemented method of claim 5 , further comprising:
determining that a matching media file in the third image data has at least one improved quality parameter over a corresponding media file in the fourth image data; and
providing the matching media file for presentation via the first device.

11. The computer-implemented method of claim 10 , further comprising:
in a third time period after the second time period:
receiving a higher-resolution version of at least a portion of the fourth image data;
processing at least a portion of the higher-resolution version to extract additional information; and
storing the additional information with the matching media file to assist with future object recognition.

12. A non-transitory computer-readable storage medium storing instructions that, when executed by at least one processor of a computing system, cause the computing system to:
in a first time period:
obtain first image data acquired using a camera of a first device;
compare at least a portion of the first image data to second image data previously-acquired by at least one second device;
identify, using a first confidence threshold, a first previously-acquired image of the second image data, the first previously-acquired image matching at least a portion of the first image data;
determine that the first device is approaching a global position;
obtain third image data associated with the global position, the third image data acquired by at least one second device;

in a second time period after the first time period:
obtain fourth image data acquired using the camera, the fourth image data including a representation of an object;
compare the fourth image data to the third image data;
identify, using a second confidence threshold, a second previously-acquired image of the third image data, the second previously-acquired image matching at least a portion of the fourth image data, the second confidence threshold being lower than the first confidence threshold; and
provide actionable content associated with the object, the actionable content enabling at least one action to be performed that is related to the object.

13. The non-transitory computer-readable storage medium of claim 12 , wherein the fourth image data includes representations of a plurality of objects, and wherein the instructions when executed further cause the computing system to:
identify, using the second confidence threshold, a plurality of previously-acquired images of the third image data, the plurality of previously-acquired images including

Assignees

• Amazon Tech Inc

Inventors

• Gray Timothy Thomas

Classifications

• G06F18/25 Primary

  Fusion techniques · CPC title

• G06K9/00536 Primary

  Physics · mapped topic

• G06V2201/10

  Recognition assisted with metadata · CPC title