Artificial intelligence (AI)-based control of imaging parameters of image-capture apparatus

Patent metadata

Field	Value
Publication number	US-11076088-B2
Application number	US-201916581670-A
Country	US
Kind code	B2
Filing date	Sep 24, 2019
Priority date	Sep 24, 2019
Publication date	Jul 27, 2021
Grant date	Jul 27, 2021

Abstract

An image-capture apparatus and method for an artificial intelligence (AI) based control of imaging parameters of the image-capture apparatus is provided. The image-capture apparatus controls the imaging sensor based on a set of imaging parameters associated with the imaging sensor, to acquire imaging information. The acquired imaging information includes a first object of a plurality of objects. The image-capture apparatus generates by, a neural network model, a first classification result based on the acquired imaging information and modifies one or more first imaging parameters of the set of imaging parameters based on the generated first classification result for the first object. The image-capture apparatus further controls the imaging sensor based on the modified set of imaging parameters, to reacquire the imaging information to maximize a confidence of the neural network model for the detection of the first object in the reacquired imaging information.

First claim

What is claimed is:

1. An image-capture apparatus, comprising:
an imaging sensor;
a memory configured to store a neural network model which is trained to detect a plurality of objects in a field-of-view (FOV) of the imaging sensor; and
control circuitry coupled with the imaging sensor and the memory, wherein the control circuitry is configured to:
control, based on a set of imaging parameters associated with the imaging sensor, the imaging sensor to acquire imaging information comprising a first object of the plurality of objects;
generate, by the neural network model, a first classification result for the first object based on the acquired imaging information,
wherein the generated first classification result indicates a confidence of the neural network model for a detection of the first object in the acquired imaging information;

modify one or more first imaging parameters of the set of imaging parameters based on the generated first classification result; and
control, based on the modified set of imaging parameters, the imaging sensor to reacquire the imaging information to maximize the confidence of the neural network model for the detection of the first object in the reacquired imaging information.

2. The image-capture apparatus according to claim 1 , wherein
the control circuitry is further configured to update the neural network model based on a training of the neural network model on the acquired imaging information, and
in the training, a set of neural weights of the neural network model is updated based on an output of the neural network model for the detection of the first object in the acquired imaging information.

3. The image-capture apparatus according to claim 1 , wherein the control circuitry is further configured to:
receive a user input for a selection of a file that comprises a set of neural network parameters of the neural network model; and
deploy the set of neural network parameters as the neural network model on the image-capture apparatus based on the selection,
wherein the set of neural network parameters comprises at least one a network topology parameter, a set of neural weights, or a loss function.

4. The image-capture apparatus according to claim 1 , wherein the control circuitry is further configured to:
receive a first user input for a selection of the first object;
transmit, to a server, a request to train the neural network model based on the received first user input; and
receive, from the server, the trained neural network model based on the transmitted request.

5. The image-capture apparatus according to claim 1 , wherein the set of imaging parameters associated with the imaging sensor comprises at least one of a focus parameter, an f-stop parameter, an exposure parameter, a shutter speed parameter, an aperture parameter, a gain parameter, a backlight parameter, a brightness parameter, a contrast parameter, a sharpness parameter, a white balance parameter, a sharpness parameter, a ISO sensitivity parameter, a noise reduction parameter, a demosaic parameter, a denoise parameter, a color parameter, a high dynamic range (HDR) parameter, or a deblur parameter.

6. The image-capture apparatus according to claim 1 , wherein the control circuitry is further configured to:
extract, by the neural network model, a region-of-interest from the acquired imaging information, wherein the region-of-interest includes the first object; and
generate, by the neural network model, the first classification result for the first object based on the extracted region-of-interest.

7. The image-capture apparatus according to claim 1 , wherein the first classification result comprises a probability score that indicates the confidence of the detection of the first object by the neural network model.

8. The image-capture apparatus according to claim 1 , wherein the control circuitry is further configured to:
compare the generated first classification result for the first object with a previous classification result for the first object generated by the neural network model; and
modify one or more second imaging parameters of the set of imaging parameters based on the comparison, wherein the one or more second imaging parameters are different from the one or more first imaging parameters.

9. The image-capture apparatus according to claim 8 , wherein the control circuitry is configured to modify the one or more second imaging parameters of the set of imaging parameters based on a determination that the confidence indicated by the generated first classification result is less than that by the previous classification result.

10. The image-capture apparatus according to claim 1 , wherein the control circuitry is further configured to:
generate a first combination of values of imaging parameters for the imaging sensor based on the modified set of imaging parameters for the maximization of the confidence of the neural network model for the detection of the first object; and
control the memory to store the generated first combination of values of imaging parameters for the first object.

11. The image-capture apparatus according to claim 1 , wherein the control circuitry is further configured to:
receive a second user input for a selection of a second object of the plurality of objects; and
generate a second combination of values of imaging parameters for the imaging sensor to maximize the confidence of the neural network model for the detection of the second object; and
control the memory to store the generated second combination of values of imaging parameters for the second object.

12. The image-capture apparatus according to claim 1 , wherein the acquired imaging information comprises an uncompressed image frame of the first object or a lossless compressed image frame of the first object.

13. A method, comprising:
in an image-capture apparatus which includes an imaging sensor and a memory:
storing, by the memory, a neural network model trained to detect a plurality of objects in a field-of-view (FOV) of the imaging sensor;
controlling, based on a set of imaging parameters associated with the imaging sensor, the imaging sensor to acquire imaging information comprising a first object of the plurality of objects;
generating, by the neural network model, a first classification result for the first object based on the acquired imaging information,
wherein the generated first classification result indicates a confidence of the neural network model for a detection of the first object in the acquired imaging information;

modifying one or more first imaging parameters of the set of imaging parameters based on the generated first classification result; and
controlling, based on the modified set of imaging parameters, the imaging sensor to reacquire the imaging information to maximize the confidence of the neural network model for the detection of the first object in the reacquired imaging information.

14. The method according to claim 13 , further comprising updating the neural network model based on a training of the neural network model on the acquired imaging information,
wherein in the training, a set of neural weights of the neural network model is updated based on an output of the neural network model for the detection of the first object in the acquired imaging information.

15. The method according to claim 13 ,
receiving a user input for a selection of a file that comprises a set of neural network parameters of the neural network model; and
deploying the set of neural network parameters as the neural network model on the image-capture apparatus based on the selection

Assignees

• Sony Corp

Inventors

• Nishimura Hiroaki
• Georgis Nikolaos

Classifications

• G06V10/7788

  the supervisor being a human, e.g. interactive learning with a human teacher · CPC title

• G06V10/774

  Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title

• G06V20/52

  Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69) · CPC title

• G06V10/82

  using neural networks · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title