Storing metadata related to captured images

Patent metadata

Field	Value
Publication number	US-10469755-B2
Application number	US-201715602661-A
Country	US
Kind code	B2
Filing date	May 23, 2017
Priority date	May 16, 2017
Publication date	Nov 5, 2019
Grant date	Nov 5, 2019

Abstract

The present disclosure relates to user-selected metadata related to images captured by a camera of a client device. User-selected metadata may include contextual information and/or information provided by a user when the images are captured. In various implementations, a free form input may be received at a first client device of one or more client devices operated by a user. A task request may be recognized from the free form input, and it may be determined that the task request includes a request to store metadata related to one or more images captured by a camera of the first client device. The metadata may be selected based on content of the task request. The metadata may then be stored, e.g., in association with one or more images captured by the camera, in computer-readable media. The computer-readable media may be searchable by the metadata.

First claim

What is claimed is:

1. A method comprising:
streaming data captured by one or more cameras to an electronic viewfinder of a first client device of one or more client devices operated by a user;
invoking an automated assistant at least partially in response to the streaming;
receiving, at the first client device while the data captured by the camera is streamed to the electronic viewfinder, a free form input from the user directed at the automated assistant;
recognizing a task request from the free form input;
determining that the task request comprises a request for the automated assistant to store metadata related to one or more images captured by one or more of the cameras, wherein the metadata is selected based on content of the task request; and
storing the metadata in one or more computer-readable mediums, wherein the one or more computer-readable mediums are searchable by the automated assistant using the metadata.

2. The method of claim 1 , wherein the free-form input is a first input, and the method further comprises:
receiving, at the first client device or a second client device of the one or more client devices, a second free form input;
recognizing another task request from the second free form input;
determining that the metadata related to the one or more images captured by the camera is responsive to the another task request; and
in response to determining that the metadata is responsive to the another task request, providing, as output via one or more output devices of the first or second client device, content indicative of the metadata.

3. The method of claim 1 , further comprising providing, as output via one or more output devices of the first client device, the task request as a suggestion to the user, wherein the task request is selected based on one or more signals generated by one or more sensors of the first client device.

4. The method of claim 3 , wherein the one or more signals include data captured by the camera.

5. The method of claim 3 , wherein the one or more signals include position coordinate data from a position coordinate sensor.

6. The method of claim 1 , further comprising:
performing image processing on the one or more images;
based on the image processing, identifying an object depicted in the one or more images; and
storing the metadata in association with another stored image that depicts the same object or another object sharing one or more attributes with the object.

7. The method of claim 1 , further comprising performing optical character recognition on a portion of the one or more images to determine textual content depicted in the one or more images.

8. The method of claim 7 , wherein the metadata further includes at least some of the textual content.

9. The method of claim 1 , wherein the metadata includes at least some of the content of the task request.

10. The method of claim 1 , wherein the metadata includes a position coordinate obtained simultaneously with capture of the one or more images.

11. At least one non-transitory computer-readable medium comprising instructions that, in response to execution of the instructions by one or more processors, cause the one or more processors to perform the following operations:
streaming data captured by one or more cameras to an electronic viewfinder of a first client device of one or more client devices operated by a user;
invoking an automated assistant at least partially in response to the streaming;
receiving, at the first client device while the data captured by the camera is streamed to the electronic viewfinder, a free form input from the user directed at the automated assistant;
recognizing a task request from the free form input;
determining that the task request comprises a request for the automated assistant to store metadata related to one or more images captured by one or more of the cameras, wherein the metadata is selected based on content of the task request; and
storing the metadata in one or more computer-readable mediums, wherein the one or more computer-readable mediums are searchable by the automated assistant using the metadata.

12. The method of claim 1 , wherein invoking the automated assistant is further based on an invocation phrase that is activated to invoke the automated assistant during the streaming.

13. A system comprising one or more processors and memory storing instructions that, in response to execution of the instructions by the one or more processors, cause the one or more processors to:
stream data captured by one or more cameras to an electronic viewfinder of a first client device of one or more client devices operated by a user;
invoke an automated assistant at least partially in response to the streaming;
receive, at the first client device while the data captured by the camera is streamed to the electronic viewfinder, a free form input from the user directed at the automated assistant;
recognize a task request from the free form input;
determine that the task request comprises a request for the automated assistant to store metadata related to one or more images captured by one or more of the cameras, wherein the metadata is selected based on content of the task request; and
store the metadata in one or more computer-readable mediums, wherein the one or more computer-readable mediums are searchable by the automated assistant using the metadata.

14. The system of claim 13 , wherein the free-form input is a first input, and the system further comprises instructions to:
receive, at the first client device or a second client device of the one or more client devices, a second free form input;
recognize another task request from the second free form input;
determine that the metadata related to the one or more images captured by the camera is responsive to the another task request; and
in response to the determination that the metadata is responsive to the another task request, provide, as output via one or more output devices of the first or second client device, content indicative of the metadata.

15. The system of claim 13 , further comprising providing, as output via one or more output devices of the first client device, the task request as a suggestion to the user, wherein the task request is selected based on one or more signals generated by one or more sensors of the first client device.

16. The system of claim 15 , wherein the one or more signals include data captured by the camera.

17. The system of claim 15 , wherein the one or more signals include position coordinate data from a position coordinate sensor.

18. The system of claim 13 , further comprising instructions to:
perform image processing on the one or more images;
based on the image processing, identify an object depicted in the one or more images; and
store the metadata in association with another stored image that depicts the same object or another object sharing one or more attributes with the object.

19. The system of claim 13 , further comprising instructions to perform optical character recognition on a portion of the one or more images to determine textual content depicted in the one or more images.

Assignees

• Google Inc
• Google Llc

Inventors

• Badr Ibrahim
• Bakir Gökhan
• Kunkle Daniel
• Ilangovan Kavin Karthik
• Burakov Denis

Classifications

• G06V30/18057

  Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN] · CPC title

• G06F16/5846 Primary

  using extracted text · CPC title

• G06V20/62

  Text, e.g. of license plates, overlay texts or captions on TV images · CPC title

• G06V10/82

  using neural networks · CPC title

• H04N23/632

  for displaying or modifying preview images prior to image capturing, e.g. variety of image resolutions or capturing parameters · CPC title