Configurable and programmable image processor unit

Patent metadata

Field	Value
Publication number	US-11250537-B2
Application number	US-201916694335-A
Country	US
Kind code	B2
Filing date	Nov 25, 2019
Priority date	May 15, 2017
Publication date	Feb 15, 2022
Grant date	Feb 15, 2022

Abstract

An image processor unit is described. The image processor unit includes a plurality of inputs to receive at least one input image. The image processor unit includes a plurality of outputs to provide at least one output image. The image processor unit includes a network coupled to the plurality of inputs and the plurality of outputs. The network is to couple at least one of the inputs to at least one of the outputs. The image processor unit includes an image processor circuit coupled to the network. The network to route an input image that is received at one of the inputs to the image processor circuit. The image processor circuit is to execute image signal processing program code to generate a processed output image from the input image. The network is to route the processed output image to at least one of the outputs.

First claim

What is claimed is:

1. A device comprising:
one or more cameras;
one or more processing elements; and
an image processor comprising:
one or more stencil processors, and
an embedded processor that is configured to read a configuration register space of the image processor to control in which of a plurality of operating modes the image processor operates, wherein the plurality of operating modes include:
an image processing mode wherein the image processor is configured to receive raw image inputs from the one or more cameras and to perform one or more image processing algorithms to generate output images for consumption by the one or more processing elements, and
a bypass mode wherein the image processor is configured to receive the raw image inputs from the one or more cameras and to bypass performance of the one or more image processing algorithms to provide the raw image inputs to the one or more processing elements.

2. The device of claim 1 , wherein the one or more processing elements comprise an image signal processor.

3. The device of claim 2 , wherein in the image processing mode, the image processor is configured to provide the output images to the image signal processor for further processing.

4. The device of claim 2 , wherein in the image processing mode, the image processor is configured to generate output images and the one or more processing elements are configured to compute statistics from the raw image inputs.

5. The device of claim 4 , wherein the one or more processing elements computing statistics from the raw image inputs are configured to receive the raw image inputs directly from the image processor.

6. The device of claim 5 , wherein the one or more processing elements computing statistics from the raw image inputs are configured to receive the raw image inputs from a first camera of the one or more cameras, and wherein the image processor is configured to generate output images from a second camera of the one or more cameras.

7. The device of claim 1 , wherein in the bypass mode, the image processor is configured to operate as a multiplexor between multiple camera inputs and one or more image processor outputs.

8. The device of claim 1 , wherein in the image processing mode, the image processor is configured to generate both output images and statistics for consumption by the one or more processing elements.

9. The device of claim 1 , wherein the image processor comprises an embedded CPU.

10. The device of claim 9 , wherein the embedded CPU of the image processor is configured to execute a driver for one of the one or more cameras.

11. The device of claim 1 , wherein the image processor is configured to operate as a co-processor by receiving input data from a main CPU and generating output data for consumption by the main CPU.

12. A method performed by a device comprising one or more cameras and an image processor comprising one or more stencil processors and an embedded processor, the method comprising:
reading, by the embedded processor, a configuration register space of the image processor that controls in which of a plurality of operating modes the image processor operates; and
configuring the image processor to operate in a particular operating mode according to the configuration register space, wherein the plurality of operating modes include:
an image processing mode wherein the image processor is configured to receive raw image inputs from the one or more cameras and to perform one or more image processing algorithms to generate output images for consumption by one or more processing elements of the device; and
a bypass mode wherein the image processor is configured to receive the raw image inputs from the one or more cameras and to bypass performance of the one or more image processing algorithms to provide the raw image inputs directly to the one or more processing elements.

13. The method of claim 12 , wherein the one or more processing elements comprise an image signal processor.

14. The method of claim 13 , further comprising providing, by the image processor while operating in the image processing mode, the output images to the image signal processor for further processing.

15. The method of claim 13 , further comprising:
generating, by the image processor while operating in the image processing mode, output images; and
computing, by the one or more processing elements, statistics from the raw image inputs.

16. The method of claim 15 , wherein the one or more processing elements computing statistics from the raw image inputs receive the raw image inputs directly from the image processor.

17. The method of claim 16 , wherein the one or more processing elements computing statistics from the raw image inputs receive the raw image inputs from a first camera of the one or more cameras, and wherein the image processor is configured to generate output images from a second camera of the one or more cameras.

18. The method of claim 12 , wherein in the bypass mode, the image processor is configured to operate as a multiplexor between multiple camera inputs and one or more image processor outputs.

19. The method of claim 12 , further comprising generating, by the image processor while operating in the image processing mode, both output images and statistics for consumption by the one or more processing elements.

20. The method of claim 12 , wherein the image processor comprises an embedded CPU.

Assignees

• Google Llc

Inventors

• Basso Fabrizio
• Chang Edward
• Finchelstein Daniel
• Knight Timothy
• Mark William
• Meixner Albert
• Rabii Shahriar
• Redgrave Jason
• Reynders Masumi
• Shacham Ofer
• Stark Don
• Tomasko Michelle

Classifications

• H04N23/81

  for suppressing or minimising disturbance in the image signal generation · CPC title

• H04N23/951

  by using two or more images to influence resolution, frame rate or aspect ratio · CPC title

• H04N23/683

  performed by a processor, e.g. controlling the readout of an image memory · CPC title

• G06V40/16

  Human faces, e.g. facial parts, sketches or expressions · CPC title

• G06V20/00

  Scenes; Scene-specific elements (control of digital cameras H04N23/60) · CPC title