Imaging apparatus, imaging system, imaging method, and imaging program

Patent metadata

Field	Value
Publication number	US-12159455-B2
Application number	US-202318451144-A
Country	US
Kind code	B2
Filing date	Aug 17, 2023
Priority date	Aug 31, 2018
Publication date	Dec 3, 2024
Grant date	Dec 3, 2024

Abstract

An imaging apparatus includes: an imaging unit that has a pixel region in which a plurality of pixels is arranged and reads and outputs a pixel signal from the pixels included in the pixel region; a unit-of-readout controller that controls a unit of readout set as a part of the pixel region; a first unit-of-readout setting unit that sets a first unit of readout that is used for reading out the pixel signal from the pixel region for a recognition process that has learned training data for each of the units of readout; a second unit-of-readout setting unit that sets a second unit of readout that is used for reading out the pixel signal from the pixel region so as to output the pixel signal to a subsequent stage; and a mediation unit that performs mediation between the first unit of readout and the second unit of readout.

First claim

The invention claimed is:

1. An imaging apparatus comprising:
circuitry configured to read and output a pixel signal from pixels included in a pixel region in which a plurality of pixels is arranged;
control a unit of readout set as a part of the pixel region;
receive a first unit of readout that is used for reading out the pixel signal from the pixel region for a recognition process for each of the units of readout;
receive a second unit of readout that is used for reading out the pixel signal from the pixel region so as to output the pixel signal to a subsequent stage; and
perform mediation between the first unit of readout and the second unit of readout based on mediation information,
wherein the unit of readout set as part of the pixel region is set by the performed mediation, and
wherein the circuitry is configured to perform the mediation by a logical product of the first unit of readout and the second unit of readout.

2. The imaging apparatus according to claim 1 ,
wherein the mediation information includes results from a machine learning process.

3. The imaging apparatus according to claim 1 ,
wherein the circuitry is configured to perform the recognition process that has learned training data.

4. The imaging apparatus according to claim 1 ,
wherein the circuitry is further configured to perform the mediation based on a result of the recognition process.

5. The imaging apparatus according to claim 4 ,
wherein the circuitry is configured to select the second unit of readout under a condition that the result of the recognition process indicates a recognition of a moving body.

6. The imaging apparatus according to claim 4 ,
wherein the circuitry is configured to select the second unit of readout under a condition in which a result of the recognition process indicates a recognition confidence of a threshold or less.

7. The imaging apparatus according to claim 1 ,
wherein the circuitry is configured to perform the mediation based on the pixel signal read out from the second unit of readout.

8. The imaging apparatus according to claim 7 ,
wherein the circuitry is configured to select the second unit of readout under a condition in which luminance based on the pixel signal exceeds a threshold.

9. The imaging apparatus according to claim 1 ,
wherein the circuitry is configured to perform the mediation based on external information supplied from outside of the imaging apparatus.

10. The imaging apparatus according to claim 9 ,
wherein the circuitry is configured to perform the mediation based on an operation mode supplied from the outside.

11. The imaging apparatus according to claim 9 ,
wherein the circuitry is configured to perform the mediation based on a detection output of another sensor device supplied from the outside.

12. A method performed by an imaging apparatus that includes processing circuitry the method comprising:
reading and outputting, with the processing circuitry, a pixel signal from pixels included in a pixel region in which a plurality of pixels is arranged;
controlling with the processing circuitry a unit of readout set as a part of the pixel region;
receiving a first unit of readout that is used for reading out the pixel signal from the pixel region for a recognition process for each of the units of readout;
receiving a second unit of readout that is used for reading out the pixel signal from the pixel region so as to output the pixel signal to a subsequent stage; and
performing mediation between the first unit of readout and the second unit of readout based on mediation information,
wherein the unit of readout set as part of the pixel region is set by the performed mediation, and
wherein the performing mediation includes performing mediation using a logical product of the first unit of readout and the second unit of readout.

13. The method according to claim 12 ,
wherein the mediation information includes results from a machine learning process.

14. The method according to claim 12 ,
further comprising performing the recognition process with learned training data.

15. The method according to claim 12 ,
wherein the performing mediation further includes performing mediation based on a result of the recognition process.

16. The method according to claim 15 , further comprising:
selecting the second unit of readout under a condition that the result of the recognition process indicates a recognition of a moving body.

17. The method according to claim 15 , further comprising
selecting the second unit of readout under a condition in which a result of the recognition process indicates a recognition confidence of a threshold or less.

18. A non-transitory computer-readable memory comprising instructions that configure processing circuitry to perform an imaging process, the imaging process comprising:
reading and outputting, with the processing circuitry, a pixel signal from pixels included in a pixel region in which a plurality of pixels is arranged;
controlling with the processing circuitry a unit of readout set as a part of the pixel region;
receiving a first unit of readout that is used for reading out the pixel signal from the pixel region for a recognition process for each of the units of readout;
receiving a second unit of readout that is used for reading out the pixel signal from the pixel region so as to output the pixel signal to a subsequent stage; and
performing mediation between the first unit of readout and the second unit of readout based on mediation information,
wherein the unit of readout set as part of the pixel region is set by the performed mediation, and
wherein the performing mediation includes performing mediation using a logical product of the first unit of readout and the second unit of readout.

Assignees

• Sony Group Corp

Inventors

• Satoh Ryuta
• Aoki Suguru
• Yamamoto Keitaro
• Eki Ryoji

Classifications

• G06N3/0442

  characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU] · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/0895

  Weakly supervised learning, e.g. semi-supervised or self-supervised learning · CPC title

• H04N25/745 Primary

  Circuitry for generating timing or clock signals · CPC title