Systems and methods for remembering held items and finding lost items using wearable camera systems

Patent metadata

Field	Value
Publication number	US-2016028917-A1
Application number	US-201514807373-A
Country	US
Kind code	A1
Filing date	Jul 23, 2015
Priority date	Jul 23, 2014
Publication date	Jan 28, 2016
Grant date	—

Abstract

Apparatuses and methods are provided for storing information related to objects associated with a hand of a user via a wearable camera system. In one implementation, a wearable apparatus for storing the information is provided comprising a wearable image sensor configured to capture a plurality of images from the environment of the user, and at least one processing device programmed to process the images. The processing device may detect the hand of the user, and an object associated with the user's hand. The processing device may proceed to store information related to the object. Consistent with disclosed embodiments, the stored information may be used for various purposes, such as warning the user of dangers, catering advertising to the user, and helping the user find objects when they are lost.

First claim

What is claimed is:

1 . A wearable apparatus for storing information related to objects associated with a hand of a user, the wearable apparatus comprising:
a wearable image sensor configured to capture a plurality of images from an environment of the user; and at least one processing device programmed to:
process the plurality of images to detect the hand of the user in at least one of the plurality of images;
process the at least one image to detect an object that is associated with the hand of the user; and
store information related to the object.

2 . The wearable apparatus of claim 1 , wherein the information related to the object includes one or more of: a time that the object was associated with the hand of the user, a location of the user when the object was associated with the hand of the user, a duration in which the object was associated with the hand of the user, and an identifier of the object.

3 . The wearable apparatus of claim 1 , wherein the object associated with the hand of the user includes one or more of: an object that the user holds, an object that the user points to, an object that the user touches, an object that the user drops, an object that the user operates, an object that the user manipulates, and an object that the user grabs.

4 . The wearable apparatus of claim 1 , further comprising a communications interface, and wherein the at least one processing device is further programmed to cause transmission of the information related to the object, via the communications interface, to a remotely located computing device for inclusion in a catalog of objects associated with the user.

5 . The wearable apparatus of claim 4 , wherein the at least one processing device is further programmed to identify a type of the object, and the catalog of objects is organized according to types of detected objects.

6 . The wearable apparatus of claim 4 , wherein the at least one processing device is further programmed to identify the object by comparing an image of the object with stored images.

7 . The wearable apparatus of claim 1 , wherein the at least one processing device is further programmed to identify that the object is a product.

8 . The wearable apparatus of claim 7 , wherein the at least one processing device is further programmed to identify a brand associated with the product.

9 . The wearable apparatus of claim 1 , further comprising a communications interface, and wherein the at least one processing device is further programmed to cause transmission of the information related to the object, via the communications interface, to a remotely located computing device for use in the selection of one or more advertisements.

10 . The wearable apparatus of claim 9 , wherein the one or more advertisements is for a product related to the at least one object.

11 . The wearable apparatus of claim 1 , further comprising a communications interface, and wherein the at least one processing device is further programmed to cause transmission of the information related to the object, via the communications interface, to a remotely located computing device for use in the selection of one or more warnings to the user.

12 . The wearable apparatus of claim 11 , wherein the object is food and the one or more warnings is associated with allergy risks or diet concerns.

13 . The wearable apparatus of claim 11 , wherein the object is a drug and the one or more warnings is associated with drug side effects.

14 . The wearable apparatus of claim 1 , wherein the at least one processing device is further programmed to cause storage of one or more recent images of the object and delete one or more older images of the object.

15 . The wearable apparatus of claim 1 , wherein the at least one processing device is further programmed to process the at least one image to identify at least one action associated with the object.

16 . The wearable apparatus of claim 15 , wherein the at least one action includes one or more of: a grabbing action associated with the object, and a dropping action associated with the object.

17 . The wearable apparatus of claim 16 , wherein the at least one processing device is further programmed to identify a last known location of the detected object in response to identification of the grabbing action or the dropping action.

18 . The wearable apparatus of claim 17 , wherein the wearable apparatus includes a communications interface, and the at least one processing device is programmed to cause transmission of information related to the last known location of the object to a remotely located computing device.

19 . The wearable apparatus of claim 18 , wherein the information includes an image showing where the lost item was last grabbed by the user.

20 . The wearable apparatus of claim 18 , wherein the information includes an image showing where the lost item was last dropped by the user.

21 . A wearable apparatus for determining a last known location of an object, the wearable apparatus comprising:
a wearable image sensor configured to capture a plurality of images from an environment of a user; and at least one processing device programmed to:
process the plurality of images to detect an image showing an object of interest; and
identify a location associated with the detected image and produce location information related to the location; and
store, in a memory, the location information with information associated with the object of interest.

22 . The wearable apparatus of claim 21 , wherein the location information includes an image showing a last known location of the object of interest.

23 . The wearable apparatus of claim 21 , wherein the at least one processing device is further programmed to store at least one recent image of the object of interest, and delete at least one older image of the object of interest.

24 . A method for storing information related to objects associated with a hand of a user of a wearable device, the method comprising:
processing a plurality of images captured by a wearable image sensor included in the wearable device to detect the hand of the user in at least one of the plurality of images; processing the at least one of the plurality of images to detect an object associated with the hand of the user; and storing information related to the object.

25 . The method of claim 24 , further comprising:
transmitting the information related to the object to a remotely located computing device for inclusion in a catalog of objects associated with the user.

26 . The method of claim 24 , further comprising:
transmitting the information related to the object to a remotely located computing device for use in selecting one or more advertisements for the user.

27 . The method of claim 26 , wherein the information includes a last known location of the object.

28 . A software product stored on a non-transitory computer readable medium and comprising data and computer implementable instructions for carrying out the method of claim 24 .

Assignees

• Orcam Technologies Ltd

Inventors

• Wexler Yonaton
• Shashua Amnon

Classifications

• G06F1/1686 Primary

  the I/O peripheral being an integrated camera · CPC title

• G06V10/776

  Validation; Performance evaluation · CPC title

• G06Q30/0267

  Wireless devices · CPC title

• H04N23/50

  Constructional details · CPC title

• H04N23/61

  based on recognised objects · CPC title