Smartphone-based asset management system

Patent metadata

Field	Value
Publication number	US-9805331-B2
Application number	US-201113023679-A
Country	US
Kind code	B2
Filing date	Feb 9, 2011
Priority date	Feb 9, 2011
Publication date	Oct 31, 2017
Grant date	Oct 31, 2017

Abstract

A method of tracking an inventory of objects via a mobile communications device includes acquiring an image of one or more of the objects via the mobile communications device, which also collects a location of the mobile communications device while acquiring the image of the one or more of the objects. The location and image are transferred from the mobile communications device to a remote server via a wireless network, such that the one or more of the objects are identified at the server based on the image, and the location and identity of the one or more objects are stored on a database associated with the server.

First claim

The invention claimed is:

1. A computer-implemented method for image processing and object recognition, the method comprising:
receiving, by a remote image processing server, an image of a plurality of objects and location information corresponding to the image from a mobile communications device via a wireless network;
performing, by an image analyzer of the remote image processing server, segmentation to divide the received image into a plurality of regions, wherein performing segmentation comprises:
generating an edge map from the image;
removing components of the edge map that are smaller than a predetermined threshold size from the edge map;
determining a dominant angle for substantially parallel lines of the edge map; and
identifying the plurality of regions of the image, wherein the plurality of regions of the image correspond to regions of the edge map between lines of the edge map within a predetermined threshold angle of the dominant angle;
for at least one region of the plurality of regions, extracting, by the remote image processing server, features of the region and matching the features of the region with a database entry to determine an object identity of a single object corresponding to the region;
determining, by the remote image processing server, a localized position of the single object within a room based on the received location information and further based on previously acquired training information;
storing, by the remote image processing server, the localized position of the single object with the corresponding object identity in a database;
receiving, by the remote image processing server, a query for an object: determining: by the remote image processor server, in response to the query, a localized position for the object based on information stored in the database; and
outputting, by the remote image processor server, the localized position for the object.

2. The method according to claim 1 , wherein the image received from mobile communications device was acquired via a built in camera of the mobile communications device.

3. The method according to claim 1 , wherein the location information received from the mobile communications device was determined by the device based on local wireless signals.

4. The method according to claim 3 , wherein the location information received from the mobile communications device was further determined based on sensor signals from the one or more accelerometers hosted within the mobile communications device.

5. A non-transitory computer-readable medium, part of a remote image processing server, having processor-executable instructions stored thereon for image processing and object recognition, the processor-executable instructions, when executed by a processor, causing the following steps to be performed:
receiving an image of a plurality of objects and location information corresponding to the image from a mobile communications device via a wireless network;
performing segmentation to divide the received image into a plurality of regions, wherein performing segmentation comprises:
generating an edge map from the image;
removing components of the edge map that are smaller than a predetermined threshold size from the edge map;
determining a dominant angle for substantially parallel lines of the edge map; and
identifying the plurality of regions of the image, wherein the plurality of regions of the image correspond to regions of the edge map between lines of the edge map within a predetermined threshold angle of the dominant angle;
for at least one region of the plurality of regions, extracting features of the region and matching the features of the region with a database entry to determine an object identity of a single object corresponding to the region;
determining a localized position of the single object within a room based on the received location information and farther based on previously acquired training information: and
storing the localized position of the single object with the corresponding object identity in a database;
receiving a query for an object;
determining, in response to the query, a localized position for the object based o information stored in the database; and
outputting the localized position for the object.

6. The non-transitory computer-readable medium according to claim 5 , wherein the received image was acquired via a built in camera of the mobile communications device.

7. The non-transitory computer-readable medium according to claim 5 , wherein the location information was determined by the mobile communications device based on local wireless signals.

8. The non-transitory computer-readable medium according to claim 7 , wherein the location information was further determined based on sensor signals from one or more accelerometers hosted within the mobile communications device.

9. The non-transitory computer-readable medium according to claim 5 , wherein the objects are books.

10. The non-transitory computer-readable medium according to claim 9 , wherein the received image is an image of the contents of a book shelf.

11. The non-transitory computer-readable medium according to claim 9 , wherein the received image is an image of the contents of a single rack of a book shelf having multiple racks.

12. A remote image processing server, the server comprising:
a receiver, configured to receive an image of a plurality of objects and insertion information corresponding to the image from a mobile communications devise via a wireless network;
a processor, configured to:
perform segmentation to divide the received image into a plurality of regions, wherein performing segmentation comprises:
generating an edge map from the image;
removing components of the edge map that are smaller than a predetermined threshold size from the edge map; determining a dominant angle for substantially parallel lines of the edge map; and
identifying the plurality of regions of the image, wherein the plurality of regions of the image correspond to regions of the edge map between lines of the edge map within a predetermined threshold angle of the dominant angle; for at least one region of the plurality of regions extract features of the region and matching the features of the region with a database entry to determine an object identity of a single object corresponding to the region, wherein a Speeded Up Robust Features (SURF) algorithm is used to match the features of the region with the database entry; and determine a localized position of the single object within a room leased on the received location information and farther based on previously acquired training information; and
a database, configured to store the localized position of the single object with the corresponding object identity;
wherein the receiver is further configured to receive a query for an object; and
wherein the processor is further configured to determine and output the localized position for the object based on information stored in the database.

13. The server of claim 12 , wherein the localized position of the single object within the room is received from the mobile communications device.

14. The method according to claim 1 , wherein the objects are books.

15. The method according to claim 14 , wherein the image received from the mobile communications device is an image of the contents of a book shelf.

16. The method according to claim 14 , wherein the image received from the mobile communications device is an image of the contents a single rack of a book shelf having multiple racks.

Assignees

• Hsu Cheng-Hsin
• Kim Kyu-Han
• Singh Jatinder Pal
• Chen David M
• Tsai Sam
• Girod Bernd
• Deutsche Telekom Ag
• Univ Leland Stanford Junior

Inventors

• Hsu Cheng-Hsin
• Kim Kyu-Han
• Singh Jatinder Pal
• Chen David M
• Tsai Sam
• Girod Bernd

Classifications

• G06Q10/087 Primary

  Inventory or stock management, e.g. order filling, procurement or balancing against orders · CPC title