Systems and methods for image object recognition based on location information and object categories

Patent metadata

Field	Value
Publication number	US-9727803-B2
Application number	US-201615229008-A
Country	US
Kind code	B2
Filing date	Aug 4, 2016
Priority date	Dec 30, 2014
Publication date	Aug 8, 2017
Grant date	Aug 8, 2017

Abstract

Systems, methods, and non-transitory computer-readable media can identify a set of regions corresponding to a geographical area. A collection of training images can be acquired. Each training image in the collection can be associated with one or more respective recognized objects and with a respective region in the set of regions. Histogram metrics for a plurality of object categories within each region in the set of regions can be determined based at least in part on the collection of training images. A neural network can be developed based at least in part on the histogram metrics for the plurality of object categories within each region in the set of regions and on the collection of training images.

First claim

What is claimed is:

1. A computer-implemented method comprising:
identifying, by a computing system, a set of regions corresponding to a geographical area;
determining, by the computing system, based at least in part on a collection of training images, metrics for a plurality of object categories associated with each region in the set of regions; and
developing, by the computing system, a neural network based at least in part on the metrics for the plurality of object categories associated with at least one region in the set of regions and on the collection of training images.

2. The computer-implemented method of claim 1 , wherein the developing of the neural network further comprises:
generating a table representing the metrics for the plurality of object categories associated with the at least one region in the set of regions;
providing a location layer within the neural network, the location layer being associated with the table; and
training the neural network based at least in part on the collection of training images.

3. The computer-implemented method of claim 2 , further comprising:
acquiring location information associated with a particular image to be inputted into the neural network;
inputting the particular image and the location information associated with the particular image into the neural network; and
causing the neural network to produce, based at least in part on the location information associated with the particular image, a predicted object recognition output for the particular image.

4. The computer-implemented method of claim 2 , further comprising:
determining, based at least in part on the training of the neural network, a preferred radius associated with each object category in the plurality of object categories.

5. The computer-implemented method of claim 1 , wherein the at least one region in the set of regions is associated with a portion of the neural network via a connection in a group of connections, and wherein the connection in the group of connections has a weight.

6. The computer-implemented method of claim 5 , wherein the developing of the neural network includes training the neural network, wherein the training of the neural network includes modifying the weight of the connection in the group of connections, and wherein the modifying of the weight of the connection in the group of connections causes the neural network to produce a predicted object recognition output, for a particular training image, that is closer to one or more recognized objects associated with the particular training image.

7. The computer-implemented method of claim 1 , wherein one or more respective recognized objects associated with each training image are recognized based at least in part on at least one of: 1) an image classification process applied to each training image or 2) one or more respective hashtags provided for each training image.

8. The computer-implemented method of claim 1 , wherein a respective region in the set of regions associated with each training image is determined based at least in part on acquiring respective location information for each training image and identifying the respective region corresponding to the respective location information.

9. The computer-implemented method of claim 8 , wherein the respective location information for each training image includes Global Positioning System (GPS) data indicating where each training image was captured.

10. The computer-implemented method of claim 1 , further comprising:
normalizing the metrics for the plurality of object categories associated with the at least one region in the set of regions, wherein the metrics are normalized based at least in part on at least one of: 1) a first ratio of a first object quantity for a particular object category associated with the at least one region relative to a second object quantity for all object categories in the plurality of object categories associated with the at least one region or 2): a second ratio of the first object quantity for the particular object category associated with the at least one region relative to a third object quantity for the particular object category associated with all regions in the set of regions.

11. A system comprising:
at least one processor; and
a memory storing instructions that, when executed by the at least one processor, cause the system to perform:
identifying a set of regions corresponding to a geographical area;
determining, based at least in part on a collection of training images, metrics for a plurality of object categories associated with each region in the set of regions; and
developing a neural network based at least in part on the metrics for the plurality of object categories associated with at least one region in the set of regions and on the collection of training images.

12. The system of claim 11 , wherein the developing of the neural network further comprises:
generating a table representing the metrics for the plurality of object categories associated with the at least one region in the set of regions;
providing a location layer within the neural network, the location layer being associated with the table; and
training the neural network based at least in part on the collection of training images.

13. The system of claim 12 , wherein the instructions cause the system to further perform:
acquiring location information associated with a particular image to be inputted into the neural network;
inputting the particular image and the location information associated with the particular image into the neural network; and
causing the neural network to produce, based at least in part on the location information associated with the particular image, a predicted object recognition output for the particular image.

14. The system of claim 11 , wherein the at least one region in the set of regions is associated with a portion of the neural network via a connection in a group of connections, and wherein the connection in the group of connections has a weight.

15. The system of claim 14 , wherein the developing of the neural network includes training the neural network, wherein the training of the neural network includes modifying the weight of the connection in the group of connections, and wherein the modifying of the weight of the connection in the group of connections causes the neural network to produce a predicted object recognition output, for a particular training image, that is closer to one or more recognized objects associated with the particular training image.

16. A non-transitory computer-readable storage medium including instructions that, when executed by at least one processor of a computing system, cause the computing system to perform:
identifying a set of regions corresponding to a geographical area;
determining, based at least in part on a collection of training images, metrics for a plurality of object categories associated with each region in the set of regions; and
developing a neural network based at least in part on the metrics for the plurality of object categories associated with at least one region in the set of regions and on the collection of training images.

17. The non-transitory computer-readable storage medium of claim 16 , wherein the developing of the neural network further comprises:
generating a table representing the metrics for the plurality of object categories associated with the at least one region in the set of regions;
providing a location layer within the neural network, the location layer being associated with the table; and
traini

Assignees

• Facebook Inc

Inventors

• Tang Kevin Dechau
• Bourdev Lubomir
• Paluri Balamanohar
• Fergus Robert D

Classifications

• G06V10/82 Primary

  using neural networks · CPC title

• G06F18/214

  Generating training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title

• G06N3/084

  Backpropagation, e.g. using gradient descent · CPC title

• G06F18/24

  Classification techniques · CPC title

• G06F18/2413

  based on distances to training or reference patterns · CPC title