Image signal processor, method of operating the image signal processor, and application processor including the image signal processor

Patent metadata

Field	Value
Publication number	US-12020345-B2
Application number	US-202117534990-A
Country	US
Kind code	B2
Filing date	Nov 24, 2021
Priority date	Sep 21, 2018
Publication date	Jun 25, 2024
Grant date	Jun 25, 2024

Abstract

An image signal processor for generating a converted image based on a raw image includes processing circuitry configured to store data corresponding to a plurality of lines of a received image in a line buffer, perform an image processing operation by filtering the data stored in the line buffer based on at least one filter, and divide the raw image into a plurality of sub-images and request the plurality of sub-images from a memory in which the raw image is stored, such that the plurality of sub-images are sequentially received by the line buffer, a width of each of the plurality of sub-images being less than a width of the line buffer, and the plurality of sub-images being parallel to each other.

First claim

What is claimed is:

1. An application processor comprising:
an image signal processor configured to image-process a raw image generated by an image sensor and generate a converted image; and
an interface circuit configured to receive the raw image from the image sensor, store the raw image in a memory in a first operation mode, and transmit the raw image to the image signal processor in a second operation mode,
wherein, in the first operation mode, the image signal processor is configured to
divide the raw image stored in the memory into a plurality of sub-images, each of the plurality of sub-images including an overlap region having a size determined by at least one filter,
sequentially receive the plurality of sub-images from the memory,
sequentially image-process the plurality of sub-images using a line buffer to generate a plurality of converted sub-images, each converted sub-image including a converted overlap region, and
sequentially store the plurality of converted sub-images by overwriting a converted overlap region of a converted sub-image with a converted overlap region of a subsequent converted sub-image.

2. The application processor of claim 1 , wherein the application processor operates in the first operation mode when a width of the raw image is larger than a size of the line buffer and the application processor operates in the second operation mode when the width of the raw image is equal to or smaller than the size of the line buffer.

3. The application processor of claim 2 , wherein, the image signal processor is further configured to determine a number of the plurality of sub-images based on the width of the raw image, a width of the overlap region, and a width of the line buffer, the overlap region being a region shared between at least two adjacent sub-images of the plurality of sub-images.

4. The application processor of claim 2 , wherein at least two adjacent sub-images of the plurality of sub-images partially overlap each other.

5. The application processor of claim 1 , wherein heights of the plurality of sub-images of the raw image are the same.

6. The application processor of claim 5 , wherein, the image signal processor is further configured to store the plurality of converted sub-images in at least one of the memory or another memory in an order in which the plurality of converted sub-images are generated to merge the plurality of converted sub-images into a first converted image.

7. The application processor of claim 1 , wherein, the image signal processor is further configured to merge the plurality of converted sub-images together into a first converted image, the first converted image having a same size as the raw image.

8. The application processor of claim 1 , wherein the image signal processor is further configured to compress a first converted image and store the compressed first converted image in a non-volatile memory.

9. The application processor of claim 1 , wherein in the second operation mode, the image signal processor is further configured to receive the raw image from the interface circuit, image-process the raw image to generate a second converted image, scale the second converted image to reduce a size of the second converted image and generate a scaled image.

10. The application processor of claim 9 , wherein, the image signal processor is further configured to provide the scaled image to a display device.

11. An application processor comprising:
an image signal processor configured to
image-process a raw image provided from an image sensor and generate a converted image,
compress the converted image,
in a first operation mode,
divide a first raw image stored in a memory into a plurality of sub-images based on a width of the first raw image, a width of a line buffer, and a width of an overlap region between at least two adjacent sub-images of the plurality of sub-images,
sequentially receive the plurality of sub-images from the memory, and
sequentially image-process the plurality of sub-images using the line buffer to generate a plurality of converted sub-images, and

determine a number of the plurality of sub-images based on the width of the raw image, the width of the line buffer, and the width of the overlap region.

12. The application processor of claim 11 , wherein the width of the first raw image is greater than the width of the line buffer.

13. The application processor of claim 11 , wherein the width of the overlap region is determined based on a size of at least one filter.

14. The application processor of claim 11 , wherein the image signal processor is further configured to merge the plurality of converted sub-images into a first converted image, the first converted image having a same size as the first raw image.

15. The application processor of claim 11 , wherein, in a second operation mode, the image signal processor is further configured to receive a second raw image from the image sensor, and image-process the second raw image to generate a second converted image, wherein the width of the second raw image is equal to or less than the width of the line buffer.

16. The application processor of claim 15 , wherein the image signal processor is further configured to convert a size of the second converted image based on a resolution of a display device.

17. A method of operating an application processor for image-processing an image provided from an image sensor, the method comprising:
storing a first raw image received from the image sensor to a memory, a width of the first raw image being greater than a width of a line buffer;
dividing, by an image signal processor, the first raw image stored in the memory into a plurality of sub-images based on the width of the first raw image, the width of the line buffer, and a width of an overlap region between two adjacent sub-images, a width of each of the plurality of sub-images being less than the width of the line buffer;
receiving, by the image signal processor, the plurality of sub-images from the memory;
sequentially image-processing, by the image signal processor, the plurality of sub-images using the line buffer and at least one filter to generate a plurality of converted sub-images; and
merging, by the image signal processor, the plurality of converted sub-images into a first converted image having a same size as the first raw image,
wherein a number of the plurality of sub-images is determined based on the width of the first raw image, the width of the line buffer, and the width of the overlap region.

18. The method of claim 17 , further comprising:
encoding the first converted image to compress the first converted image; and
storing the compressed first converted image in a non-volatile memory.

19. The method of claim 17 , further comprising:
receiving, by the image signal processor, a second raw image from the image sensor, a width of the second raw image being equal to or less than the width of the line buffer; and
image-processing, by the image signal processor, the second raw image using the line buffer to generate a second converted image.

20. The method of claim 19 , further comprising:
scaling a size of the second converted image to generate a scaled image; and
providing the scaled image to a display device.

Assignees

• Samsung Electronics Co Ltd

Inventors

• Kim Dong-Hoon

Classifications

• G06V20/56

  exterior to a vehicle by using sensors mounted on the vehicle · CPC title

• G09G5/39

  Control of the bit-mapped memory · CPC title

• G09G2340/0492

  Change of orientation of the displayed image, e.g. upside-down, mirrored · CPC title

• G06T3/606

  by memory addressing or mapping · CPC title

• G06T1/20 Primary

  Processor architectures; Processor configuration, e.g. pipelining · CPC title