Imaging apparatus, imaging system, imaging method, and imaging program

Patent metadata

Field	Value
Publication number	US-11763554-B2
Application number	US-201917267500-A
Country	US
Kind code	B2
Filing date	Sep 2, 2019
Priority date	Aug 31, 2018
Publication date	Sep 19, 2023
Grant date	Sep 19, 2023

Abstract

An imaging apparatus includes: an imaging unit that has a pixel region in which a plurality of pixels is arranged and reads and outputs a pixel signal from the pixels included in the pixel region; a unit-of-readout controller that controls a unit of readout set as a part of the pixel region; a first unit-of-readout setting unit that sets a first unit of readout that is used for reading out the pixel signal from the pixel region for a recognition process that has learned training data for each of the units of readout; a second unit-of-readout setting unit that sets a second unit of readout that is used for reading out the pixel signal from the pixel region so as to output the pixel signal to a subsequent stage; and a mediation unit that performs mediation between the first unit of readout and the second unit of readout.

First claim

The invention claimed is:

1. An imaging apparatus comprising circuitry configured to:
read and output a pixel signal from pixels included in a pixel region in which a plurality of pixels is arranged;
control a unit of readout set as a part of the pixel region;
receive a first unit of readout that is used for reading out the pixel signal from the pixel region for a recognition process that has learned training data for each of the units of readout;
receive a second unit of readout that is used for reading out the pixel signal from the pixel region so as to output the pixel signal to a subsequent stage, wherein the first unit of readout and the second unit of readout are received at the same time; and
perform mediation between the first unit of readout and the second unit of readout,
wherein the unit of readout set as part of the pixel region is set by the performed mediation.

2. The imaging apparatus according to claim 1 ,
wherein the circuitry is configured to perform the mediation by a logical product of the first unit of readout and the second unit of readout.

3. The imaging apparatus according to claim 1 ,
wherein the circuitry is configured to perform the mediation based on a result of the recognition process.

4. The imaging apparatus according to claim 3 ,
wherein the circuitry is configured to select the second unit of readout in a case where the result of the recognition process indicates a recognition of a moving body.

5. The imaging apparatus according to claim 3 ,
wherein the circuitry is configured to select the second unit of readout in a case where the result of the recognition process indicates a recognition confidence of a threshold or less.

6. The imaging apparatus according to claim 1 ,
wherein the circuitry is configured to perform the mediation based on the pixel signal read out from the second unit of readout.

7. The imaging apparatus according to claim 6 ,
wherein the circuitry is configured to select the second unit of readout in a case where luminance based on the pixel signal exceeds a threshold.

8. The imaging apparatus according to claim 1 ,
wherein the circuitry is configured to perform the mediation based on external information supplied from outside of the imaging apparatus.

9. The imaging apparatus according to claim 8 ,
wherein the circuitry is configured to perform the mediation based on an operation mode supplied from the outside.

10. The imaging apparatus according to claim 8 ,
wherein the circuitry is configured to perform the mediation based on a detection output of another sensor device supplied from the outside.

11. An imaging system comprising:
an imaging apparatus having circuitry configured to:
read and output a pixel signal from pixels included in a pixel region in which a plurality of pixels is arranged,
control a unit of readout set as a part of the pixel region,
receive a first unit of readout that is used for reading out the pixel signal from the pixel region for a recognition process that has learned training data for each of the units of readout,
receive a second unit of readout that is used for reading out the pixel signal from the pixel region so as to output the pixel signal to a subsequent stage, wherein the first unit of readout and the second unit of readout are received at the same time, and
perform mediation between the first unit of readout and the second unit of readout, and
execute the recognition process,
wherein the unit of readout set as part of the pixel region is set by the performed mediation.

12. An imaging method executed by a processor, comprising:
reading and outputting a pixel signal from pixels included in a pixel region in which a plurality of pixels is arranged;
controlling a unit of readout set as a part of the pixel region;
receiving a first unit of readout that is used for reading out the pixel signal from the pixel region for a recognition process that has learned training data for each of the units of readout;
receiving a second unit of readout that is used for reading out the pixel signal from the pixel region so as to output the pixel signal to a subsequent stage, wherein the first unit of readout and the second unit of readout are received at the same time; and
performing mediation between the first unit of readout and the second unit of readout,
wherein the unit-of-readout set as part of the pixel region is set by the performed mediation.

13. The imaging method according to claim 12 , further comprising performing the mediation by a logical product of the first unit of readout and the second unit of readout.

14. The imaging method according to claim 12 , further comprising performing the mediation based on a result of the recognition process.

15. The imaging method according to claim 14 , further comprising selecting the second unit of readout in a case where the result of the recognition process indicates a recognition of a moving body.

16. The imaging method according to claim 14 , further comprising selecting the second unit of readout in a case where the result of the recognition process indicates a recognition confidence of a threshold or less.

17. The imaging method according to claim 12 , further comprising performing the mediation based on the pixel signal read out from the second unit of readout.

18. The imaging method according to claim 17 , further comprising selecting the second unit of readout in a case where luminance based on the pixel signal exceeds a threshold.

19. The imaging method according to claim 12 , further comprising performing the mediation based on information supplied to the processor from outside the processor.

20. A non-transitory, computer-readable medium storing instructions that, when executed by a processor in the computer, control the computer to implement a method comprising:
reading and outputting a pixel signal from pixels included in a pixel region in which a plurality of pixels is arranged;
controlling a unit of readout set as a part of the pixel region;
receiving a first unit of readout that is used for reading out the pixel signal from the pixel region for a recognition process that has learned training data for each of the units of readout;
receiving a second unit of readout that is used for reading out the pixel signal from the pixel region so as to output the pixel signal to a subsequent stage, wherein the first unit of readout and the second unit of readout are received at the same time; and
performing mediation between the first unit of readout and the second unit of readout,
wherein the unit-of-readout set as part of the pixel region is set by the performed mediation.

Assignees

• Sony Corp

Inventors

• Satoh Ryuta
• Aoki Suguru
• Yamamoto Keitaro
• Eki Ryoji

Classifications

• G06N3/0442

  characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU] · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/0895

  Weakly supervised learning, e.g. semi-supervised or self-supervised learning · CPC title

• H04N25/745 Primary

  Circuitry for generating timing or clock signals · CPC title