Object outlining to initiate a visual search

Patent metadata

Field	Value
Publication number	US-9230171-B2
Application number	US-201213400258-A
Country	US
Kind code	B2
Filing date	Feb 20, 2012
Priority date	Jan 6, 2012
Publication date	Jan 5, 2016
Grant date	Jan 5, 2016

Abstract

Methods and devices for initiating a search of an object are disclosed. In one embodiment, a method is disclosed that includes receiving video data from a camera on a wearable computing device and, based on the video data, detecting a movement that defines an outline of an area in the video data. The method further includes identifying an object that is located in the area and initiating a search on the object. In another embodiment, a server is disclosed that includes an interface configured to receive video data from a camera on a wearable computing device, at least one processor, and data storage comprising instructions executable by the at least one processor to detect, based on the video data, a movement that defines an outline of an area in the video data, identify an object that is located in the area, and initiate a search on the object.

First claim

The invention claimed is:

1. A method comprising:
receiving video data from a camera on a wearable computing device, wherein a real-world field of view corresponds to the wearable computing device;
based on the video data, detecting a movement of a pointing device within the real-world field of view;
determining that the movement of the pointing device defines at least a portion of an outline of an area in the video data;
identifying a portion of the video data that represents a real-world object and is substantially located in the determined area; and
initiating a visual search on the portion of the video data that represents the real-world object.

2. The method of claim 1 , further comprising causing the wearable computing device to display information corresponding to results of the search.

3. The method of claim 1 , wherein initiating the visual search comprises:
beginning the visual search before the movement is complete; and
periodically updating the visual search during the movement.

4. The method of claim 3 :
wherein beginning the visual search comprises causing the wearable computing device to display information corresponding to results of the search; and
wherein periodically updating the visual search comprises causing the wearable computing device to display updated information corresponding to results of the updated search.

5. The method of claim 1 , further comprising:
during the visual search, caching information corresponding to results of the visual search; and
causing the wearable computing device to display at least some of the cached information.

6. The method of claim 1 , wherein:
the movement comprises movement of a pointing device; and
detecting the movement comprises tracking the pointing device.

7. The method of claim 6 , wherein the pointing device comprises a finger.

8. The, method of claim 1 wherein:
the camera is configured to be offset from an eye of a user by an angle and a distance; and
detecting the movement is further based on the angle and the distance.

9. The method of claim 1 , wherein identifying the object comprises:
using edge detection to detect one or more edges of the object in the outlined area in the video data, wherein the detected edges define the portion of the video data that represents the real-world object.

10. The method of claim 1 , wherein identifying the object comprises:
segmenting the video data into layers; and
aligning the outline with at least one layer.

11. A device comprising:
an interface configured to receive video data from a camera on a wearable computing device, wherein a real-world field of view corresponds to the wearable computing device;
at least one processor; and
data storage comprising instructions executable by the at least one processor to:
based on the video data, detect a movement of a pointing device within the real-world field of view;
determining that the movement of the pointing device defines at least a portion of an outline of an area in the video data;
identify a portion of the video data that represents a real-world object and-is substantially located in the determined area; and
initiate a visual search on the portion of the video data that represents the real-world object.

12. The device of claim 11 , further comprising a display configured to display information corresponding to results of the visual search.

13. The device of claim 12 , wherein the display comprises a head-mounted display.

14. The device of claim 12 , wherein the display is further configured to display the video data.

15. The device of claim 14 , wherein the visual search comprises:
beginning the visual search before the movement is complete; and
periodically updating the visual search during the movement.

16. The device of claim 11 , wherein:
the movement comprises movement of a pointing device; and
detecting the movement comprises tracking the pointing device.

17. A non-transitory computer-readable medium having stored therein instructions executable by a computing device to cause the computing device to perform functions comprising:
receiving video data from a camera on a wearable computing device, wherein a real-world field of view corresponds to the wearable computing device;
based on the video data, detecting a movement of a pointing device within the real-world field of view;
determining that the movement of the pointing device defines at least a portion of an outline of an area in the video data;
identifying a portion of the video data that represents a real-world object and is substantially located in the determined area; and
initiating a visual search on the portion of the video data that represents the real-world object.

18. The non-transitory computer-readable medium of claim 17 , the functions further comprising causing the wearable computing device to display information corresponding to results of the visual search.

19. The non-transitory computer-readable medium of claim 17 , wherein performing the visual search comprises:
beginning the visual search before the movement is complete; and
periodically updating the visual search during the movement.

Assignees

• Starner Thad Eugene
• Essa Irfan
• Google Inc

Inventors

• Starner Thad Eugene
• Essa Irfan

Classifications

• G06V20/20 Primary

  in augmented reality scenes · CPC title

• G06T19/006

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• G06K9/00671 Primary

  Physics · mapped topic

• G06T2207/20112

  Image segmentation details · CPC title

• G06T7/0083

  Physics · mapped topic