Image capturing apparatus and signal processing method

Patent metadata

Field	Value
Publication number	US-2025150706-A1
Application number	US-202318837011-A
Country	US
Kind code	A1
Filing date	Feb 2, 2023
Priority date	Feb 17, 2022
Publication date	May 8, 2025
Grant date	—

Abstract

The present technique pertains to an image capturing apparatus and a signal processing method that enable power consumption to be lowered for a sensor equipped with a DNN (Deep Neural Network). The image capturing apparatus performs a DNN (Deep Neural Network) process on image data that is generated by image capturing, outputs a trigger signal for controlling stopping or driving of the DNN process, and, on the basis of the trigger signal, controls stopping of the DNN process or driving of the DNN process and output to a subsequent stage. The present technique can be applied to an image capturing apparatus that is provided with an image sensor which is equipped with a DNN.

First claim

1 . An image capturing apparatus comprising:
a DNN (Deep Neural Network) processing unit that performs a DNN process on image data generated by image capturing; a trigger unit that outputs a trigger signal for controlling stopping or driving of the DNN process; and a control unit that, on a basis of the trigger signal, controls stopping or driving for the DNN process and output to a subsequent stage.

2 . The image capturing apparatus according to claim 1 , wherein,
in a case where the trigger signal indicates on, the control unit causes the DNN process to be operated and controls output of a result of the DNN process to the subsequent stage.

3 . The image capturing apparatus according to claim 2 , wherein,
in a case where the trigger signal indicates off, the control unit causes the DNN process to be stopped and causes output, to the subsequent stage, of a pre-stop processing result that indicates a result of the DNN process obtained before the DNN process is stopped.

4 . The image capturing apparatus according to claim 3 , wherein
the control unit causes the pre-stop processing result to be saved and, while the DNN process is stopped, causes output of the pre-stop processing result to be in synchronization with an output timing for the image data.

5 . The image capturing apparatus according to claim 2 , wherein,
in a case where the trigger signal indicates off, the control unit causes the DNN process to be stopped, and causes information pertaining to stopping of the DNN process to be outputted to the subsequent stage.

6 . The image capturing apparatus according to claim 1 , wherein
the control unit causes a timing of output to the subsequent stage to be in synchronization with an output timing for the image data.

7 . The image capturing apparatus according to claim 1 , wherein
the control unit causes a frame rate for the image capturing to decrease when stopping the DNN process, and the control unit returns the frame rate for image capturing to an original value when the DNN process is operated again.

8 . The image capturing apparatus according to claim 1 , wherein
the trigger unit outputs the trigger signal on a basis of a signal that indicates a characteristic of the image data.

9 . The image capturing apparatus according to claim 8 , wherein
the signal that indicates the characteristic of the image data is a signal that indicates whether or not a luminance value of the image data continues to exceed a predetermined luminance value.

10 . The image capturing apparatus according to claim 8 , wherein
the signal that indicates the characteristic of the image data is a signal that indicates whether or not a feature amount of the image data is separated by a predetermined amount from a feature amount of an image used when the DNN process is trained.

11 . The image capturing apparatus according to claim 8 , wherein
the signal that indicates the characteristic of the image data is a signal obtained from a signal processing unit that executes signal processing on the image data.

12 . The image capturing apparatus according to claim 1 , wherein
the trigger unit outputs the trigger signal on a basis of a signal that indicates a change in luminance of each pixel of an image sensor.

13 . The image capturing apparatus according to claim 12 , wherein
the signal that indicates the change in luminance of each pixel of the image sensor is a signal obtained from an external sensor.

14 . The image capturing apparatus according to claim 13 , wherein
the trigger unit receives, asynchronously with respect to a timing of the image capturing, the signal obtained from the external sensor.

15 . The image capturing apparatus according to claim 1 , wherein
the DNN process includes a process that uses a trained learning model to detect an object from the image data.

16 . A signal processing method comprising:
by an image capturing apparatus, performing a DNN (Deep Neural Network) process on image data generated by image capturing; outputting a trigger signal for controlling stopping or driving of the DNN process; and controlling, on a basis of the trigger signal, stopping or driving for the DNN process and output to a subsequent stage.

Assignees

• Sony Semiconductor Solutions Corp

Inventors

• Masuno Tomonori
• Ujiie Ichiro
• Hanada Seigo
• ONIKI Kazumasa
• Matsumoto Nana
• Yang Ying
• Yamada Kazuki

Classifications

• H04N23/80

  Camera processing pipelines; Components thereof · CPC title

• H04N25/47

  Image sensors with pixel address output; Event-driven image sensors; Selection of pixels to be read out based on image data · CPC title

• G06V10/82

  using neural networks · CPC title

• G06N3/04

  Architecture, e.g. interconnection topology · CPC title

• H04N25/76

  Addressed sensors, e.g. MOS or CMOS sensors · CPC title