Storing metadata related to captured images

Patent metadata

Field	Value
Publication number	US-10893202-B2
Application number	US-201916585085-A
Country	US
Kind code	B2
Filing date	Sep 27, 2019
Priority date	May 16, 2017
Publication date	Jan 12, 2021
Grant date	Jan 12, 2021

Abstract

The present disclosure relates to user-selected metadata related to images captured by a camera of a client device. User-selected metadata may include contextual information and/or information provided by a user when the images are captured. In various implementations, a free form input may be received at a first client device of one or more client devices operated by a user. A task request may be recognized from the free form input, and it may be determined that the task request includes a request to store metadata related to one or more images captured by a camera of the first client device. The metadata may be selected based on content of the task request. The metadata may then be stored, e.g., in association with one or more images captured by the camera, in computer-readable media. The computer-readable media may be searchable by the metadata.

First claim

What is claimed is:

1. A method implemented using one or more processors, comprising:
streaming data captured by one or more cameras to a camera application active on a first client device of one or more client devices operated by a user;
invoking an automated assistant at least partially based on the camera application being active on the first client device;
performing image recognition analysis on the data captured by one or more of the cameras to detect a vehicle;
in response to detection of the vehicle, provide to the user, as output from the automated assistant, a suggested task request to remember a parking location associated with the depicted vehicle;
receiving, at the first client device while the data captured by the one or more cameras is streamed to the camera application, confirmation from the user to perform the suggested task request; and
storing metadata indicative of the parking location in one or more computer-readable mediums, wherein the one or more computer-readable mediums are searchable by the automated assistant using the metadata.

2. The method of claim 1 , wherein the method further comprises:
receiving, at the first client device or a second client device of the one or more client devices, a free form input;
recognizing another task request from the free form input;
determining that the metadata indicative of the parking location is responsive to the another task request; and
in response to determining that the metadata is responsive to the another task request, providing, as output via one or more output devices of the first or second client device, content indicative of the metadata.

3. The method of claim 1 , further comprising performing optical character recognition on a portion of the data captured by one or more of the cameras to determine textual content depicted in the data captured by one or more of the cameras.

4. The method of claim 3 , wherein the metadata further includes at least some of the textual content.

5. The method of claim 1 , wherein the metadata includes at least some of a content of the suggested task request.

6. The method of claim 1 , wherein the metadata includes a position coordinate obtained simultaneously with capture of the data captured by one or more of the cameras.

7. A system comprising:
one or more processors;
one or more cameras operably coupled with the one or more processors;
a microphone operably coupled with one or more of the processors; and
memory storing instructions that, in response to execution of the instructions by one or more of the processors, cause one or more of the processors to operate a camera application and at least a portion of an automated assistant, wherein the automated assistant is invoked at least in part based on the camera application, and the one or more processors are to:
perform image recognition analysis on data captured by one or more of the cameras to detect a vehicle;
in response to detection of the vehicle, cause the automated assistant to provide a suggested task request to remember a parking location associated with the depicted vehicle;
receive confirmation input from a user to perform the suggested task request; and
store metadata indicative of the parking location in one or more computer-readable mediums, wherein the one or more computer-readable mediums are searchable by the automated assistant using the metadata.

8. The system of claim 7 , wherein the automated assistant is further to:
receive a free form input;
recognize another task request from the free form input;
determine that the metadata related to the data captured by one or more of the cameras is responsive to the another task request; and
in response to determining that the metadata is responsive to the another task request, providing, as output via one or more output devices, content indicative of the metadata.

9. The system of claim 7 , wherein one or more of the processors are to perform optical character recognition on a portion of the data captured by one or more of the cameras to determine textual content depicted in the data captured by one or more of the cameras.

10. The system of claim 9 , wherein the metadata further includes at least some of the textual content.

11. The system of claim 7 , wherein the metadata includes at least some of the content of the suggested task request.

12. At least one non-transitory computer-readable medium comprising instructions that, in response to execution of the instructions by one or more processors, cause the one or more processors to perform the following operations:
streaming data captured by one or more cameras to a camera application active on a first client device of one or more client devices operated by a user;
invoking an automated assistant at least partially based on the camera application being active on the first client device;
performing image recognition analysis on data captured by one or more of the cameras to detect a vehicle;
in response detection of the vehicle, provide to the user, as output from the automated assistant, a suggested task request to remember a parking location associated with the depicted vehicle;
receiving, at the first client device, confirmation a free form input from the user to perform the suggested task request; and
storing metadata indicative of the parking location in one or more computer-readable mediums, wherein the one or more computer-readable mediums are searchable by the automated assistant using the metadata.

Assignees

• Google Llc

Inventors

• Badr Ibrahim
• Bakir Gökhan
• Kunkle Daniel
• Ilangovan Kavin Karthik
• Burakov Denis

Classifications

• G06V30/18057

  Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN] · CPC title

• G06F16/5846 Primary

  using extracted text · CPC title

• G06V20/62

  Text, e.g. of license plates, overlay texts or captions on TV images · CPC title

• G06V10/82

  using neural networks · CPC title

• H04N23/632

  for displaying or modifying preview images prior to image capturing, e.g. variety of image resolutions or capturing parameters · CPC title