Contextual HDR determination

Patent metadata

Field	Value
Publication number	US-9607366-B1
Application number	US-201414576770-A
Country	US
Kind code	B1
Filing date	Dec 19, 2014
Priority date	Dec 19, 2014
Publication date	Mar 28, 2017
Grant date	Mar 28, 2017

Abstract

A system uses machine learning techniques to train models for determining whether to activate high dynamic range (HDR) imaging in devices. The system may train machine learning components/models based on customized image metrics (such as customized histograms) that can be used across devices. The system may also train HDR systems on non-image data like GPS data, etc. to make improved HDR recommendations based on the context of a particular image.

First claim

What is claimed is:

1. A computer implemented method comprising:
by a server device:
configuring a customized shape for an image section, the image section including pixels for calculating a histogram value, wherein:
the customized shape is configured by multiplying first dimensions by second dimensions;
the first dimensions are of a first shape used by a first hardware configuration type to calculate histograms; and

the second dimensions are of a second shape used by a second hardware configuration type to calculate histograms,
determining a first training image,
determining a second training image,
determining a first training histogram value for the first training image using the customized shape,
determining a second training histogram value for the second training image using the customized shape,
determining a first training recommendation for the first training image, wherein the first training recommendation comprises a first indication that the first training image is to be processed by high dynamic range (HDR) imaging,
determining a second training recommendation for the second training image, wherein the second training recommendation comprises a second indication that the second training image is to be processed by HDR imaging,
training a model for use by a neural network component when determining whether to perform HDR imaging for future image data associated with an image received by the server device in the future, wherein the training is based at least in part on the first training image, the second training image, the first training histogram value, the second training histogram value, the first training recommendation, and the second training recommendation, and
sending the model to a client device.

2. The computer implemented method of claim 1 , further comprising:
by the client device, wherein the client device is a device of the first hardware configuration type:
receiving the model from the server device,
obtaining a first image from a camera of the client device,
identifying a first region of the first image, the first region having the customized shape,
determining a plurality of subregions of the first region, wherein each subregion is of the first shape,
determining a first histogram value for pixels within each of the plurality of subregions,
adding each of the first histogram values to obtain a second histogram value, the second histogram value corresponding to pixels in the first region,
determining, using a neural network component, to perform HDR imaging on the first image based on the second histogram value and the model, and
performing HDR imaging on the first image.

3. The computer implemented method of claim 2 , further comprising:
by the server device:
determining first time training data for the first training image, the first time training data corresponding to a time the first training image was taken,
determining second time training data for the second training image, the second time training data corresponding to a time the second training image was taken,
wherein the training is further based at least in part on the first time training data and the second time training data; and

by the client device:
determining first time data for the client device, the first time data corresponding to a time the first image was taken,
wherein the determining to perform HDR imaging is further based on the first time data.

4. A computer-implemented method comprising:
determining a third image metric, wherein the third image metric is determined based on a first image metric measured by a first hardware configuration type and a second image metric measured by a second hardware configuration type;
determining a first training image;
determining a second training image;
determining a first training metric value for the first training image using the third image metric;
determining a second training metric value for the second training image using the third image metric;
determining a first training recommendation for the first training image, wherein the first training recommendation comprises a first indication that the first training image is to be processed by high dynamic range (HDR) imaging;
determining a second training recommendation for the second training image, wherein the second training recommendation comprises a second indication that the second training image is to be processed by HDR imaging; and
training a model for determining whether to perform HDR imaging for an image, wherein the training is based at least in part on the first training metric value, the second training metric value, the first training recommendation, and the second training recommendation.

5. The computer-implemented method of claim 4 , wherein the first image metric, second image metric, and third image metric are histograms.

6. The computer-implemented method of claim 4 , wherein the model is a neural network model.

7. The computer-implemented method of claim 4 , further comprising:
receiving a plurality of first image metric values from a first device of the first hardware configuration type, the plurality of first image metric values corresponding to a first image;
determining a third image metric value using the plurality of first image metric values;
determining to perform HDR imaging by the first device based at least in part on the third image metric value and the model; and
sending an indication to the first device to perform HDR imaging.

8. The computer-implemented method of claim 4 , further comprising:
determining a non-image metric;
determining a first training non-image metric value for the first training image using the non-image metric;
determining a second training non-image metric value for the second training image using the non-image metric; and
training an inference engine using the first training non-image metric value and the second training non-image metric value, and
wherein the training of the model is further based at least in part on output from the inference engine.

9. The computer-implemented method of claim 4 , further comprising:
determining a non-image metric;
determining a first training non-image metric value for the first training image using the non-image metric; and
determining a second training non-image metric value for the second training image using the non-image metric, and
wherein the training is further based on the first training non-image metric value and the second training non-image metric value.

10. The computer-implemented method of claim 9 , further comprising training an inference engine using the first training non-image metric value and the second training non-image metric value, wherein the training of the model is further based on output from the inference engine.

11. The computer-implemented method of claim 9 , wherein the non-image metric is one or more of a location, a direction, a device orientation, motion sensor data, time, or date.

12. The computer-implemented method of claim 9 , further comprising:
receiving a first non-image metric value from a first device, the first non-image metric value corresponding to a first image;
determining to perform HDR imaging by the first device based at least in part on the first non-image metric value and the model; and
sending an indication to the first device to perform HDR imaging.

13. A system comprising:
at least one processor; and
a first memory including instructions operable to be executed by the at least one processor to perform a set of actions to configure the system for:
determining a non-image metric;
determining a first training image;
determin

Assignees

• Amazon Tech Inc

Inventors

• Brailovskiy Ilya Vladimirovich
• Tryndin Igor Aleksandrovich

Classifications

• G06V10/82 Primary

  using neural networks · CPC title

• G06V10/776

  Validation; Performance evaluation · CPC title

• G06F18/217

  Validation; Performance evaluation; Active pattern learning techniques · CPC title

• G06V10/507

  Summing image-intensity values; Histogram projection analysis · CPC title

• G06T2207/20081

  Training; Learning · CPC title