Low Power Framework for Controlling Image Sensor Mode in a Mobile Image Capture Device

Patent metadata

Field	Value
Publication number	US-2018367752-A1
Application number	US-201816109708-A
Country	US
Kind code	A1
Filing date	Aug 22, 2018
Priority date	Dec 30, 2015
Publication date	Dec 20, 2018
Grant date	—

Abstract

The present disclosure provides an image capture, curation, and editing system that includes a resource-efficient mobile image capture device that continuously captures images. In particular, the present disclosure provides low power frameworks for controlling image sensor mode in a mobile image capture device. On example low power frame work includes a scene analyzer that analyzes a scene depicted by a first image and, based at least in part on such analysis, causes an image sensor control signal to be provided to an image sensor to adjust at least one of the frame rate and the resolution of the image sensor.

First claim

1 .- 20 . (canceled)

21 . A computer-implemented method, comprising:
obtaining, by one or more server computing devices, data descriptive of a pre-trained artificial neural network, the pre-trained artificial neural network having been previously trained based at least in part on a set of first training images; receiving, by the one or more server computing devices, a set of user training images provided by a user, the set of user training images including at least one image not included in the set of first training images; re-training, by the one or more server computing devices, the pre-trained artificial neural network based at least in part on the set of user training images to form a re-trained artificial neural network; and transmitting, by the one or more server computing devices, the re-trained artificial neural network to a user computing device associated with the user for implementation at the user computing device.

22 . The computer-implemented method of claim 21 , wherein at least one of the set of user training images has been edited by the user.

23 . The computer-implemented method of claim 21 , wherein at least one of the set of user training images has been hand-labeled by the user.

24 . The computer-implemented method of claim 21 , wherein the user computing device comprises an image capture device associated with the user.

25 . The computer-implemented method of claim 24 , wherein the set of user training images comprise images previously captured by the image capture device.

26 . The computer-implemented method of claim 21 , wherein the pre-trained artificial neural network and the re-trained artificial neural network comprise convolutional artificial neural networks.

27 . The computer-implemented method of claim 21 , wherein the pre-trained artificial neural network and the re-trained artificial neural network comprise image classification artificial neural networks.

28 . The computer-implemented method of claim 21 , wherein the pre-trained artificial neural network and the re-trained artificial neural network comprise face detection artificial neural networks.

29 . The computer-implemented method of claim 21 , wherein the pre-trained artificial neural network and the re-trained artificial neural network comprise image content artificial neural networks.

30 . A training computing system for training personalized artificial neural networks based on user-submitted training data, the training computing system comprising one or more server computing devices configured to perform operations, the operations comprising:
obtaining data descriptive of a pre-trained artificial neural network, the pre-trained artificial neural network having been previously trained based at least in part on a set of first training images; receiving a set of user training images selected by a user, the set of user training images including at least one image not included in the set of first training images; re-training the pre-trained artificial neural network based at least in part on the set of user training images to form a re-trained artificial neural network; and transmitting the re-trained artificial neural network to a user computing device associated with the user for implementation at the user computing device.

31 . The training computing system of claim 30 , wherein at least one of the set of user training images has been edited by the user.

32 . The training computing system of claim 30 , wherein at least one of the set of user training images has been hand-labeled by the user.

33 . The training computing system of claim 30 , wherein the user computing device comprises an image capture device associated with the user.

34 . The training computing system of claim 32 , wherein the set of user training images comprise images previously captured by the image capture device.

35 . The training computing system of claim 30 , wherein the pre-trained artificial neural network and the re-trained artificial neural network comprise convolutional artificial neural networks.

36 . The training computing system of claim 30 , wherein the pre-trained artificial neural network and the re-trained artificial neural network comprise image classification artificial neural networks.

37 . The training computing system of claim 30 , wherein the pre-trained artificial neural network and the re-trained artificial neural network comprise face detection artificial neural networks.

38 . The training computing system of claim 30 , wherein the pre-trained artificial neural network and the re-trained artificial neural network comprise image content artificial neural networks.

39 . A user computing system comprising:
an image capture system comprising an artificial neural network, the image capture system configured to capture images based at least in part on an output of the artificial neural network; and one or more computing devices configured to:
obtain a set of images captured by the image capture device;
receive user input that edits one or more of the set of images captured by the image capture device to form a set of edited images; and
after receiving the user input, provide the set of edited images to a training computing system; and
wherein the image capture system is configured to:
receive and store a re-trained version of the artificial neural network that has been re-trained by the training computing system based on the set of edited images provided to the training computing system by the one or more computing devices; and
capture images based at least in part on an output of the re-trained version of the artificial neural network.

40 . The user computing system of claim 39 , wherein the image capture system comprises a mobile image capture device.

Assignees

• Google Llc

Inventors

• Donsbach Aaron Michael
• Vanik Benjamin
• Clapper Jon Gabriel
• Lentz Alison
• Lovejoy Joshua Denali
• Fritz Iii Robert Douglas
• Duleba Krzysztof
• Zhang Li
• Payne Juston
• Fortuna Emily Anne
• Bialynicka-Birula Iwona
• Aguera-Arcas Blaise
• Ramage Daniel
• Mcmahan Benjamin James
• Lange Oliver Fritz
• Holbrook Jess

Classifications

• G06V30/19173

  Classification techniques · CPC title

• G06V10/82

  using neural networks · CPC title

• H04N23/65

  Control of camera operation in relation to power supply · CPC title

• G06N3/045

  Combinations of networks · CPC title

• H04N23/611

  where the recognised objects include parts of the human body · CPC title