Imaging device and image processing method

Patent metadata

Field	Value
Publication number	US-2024005448-A1
Application number	US-202217901023-A
Country	US
Kind code	A1
Filing date	Sep 1, 2022
Priority date	Sep 3, 2021
Publication date	Jan 4, 2024
Grant date	—

Abstract

An image processing device is provided. The imaging device includes: an image sensor configured to generate an original image; and a processor including: a remosaic processing circuit configured to generate a first remosaic image by performing remosaic processing on a first region of the original image by using a first remosaic method, and generate a second remosaic image by performing remosaic processing on a second region of the original image by using the first remosaic method or a second remosaic method; and a merging circuit configured to generate a converted image having a Bayer pattern based on the first remosaic image and the second remosaic image.

First claim

What is claimed is:

1 . An imaging device comprising:
an image sensor configured to generate an original image; and a processor comprising:
a remosaic processing circuit configured to generate a first remosaic image by performing remosaic processing on a first region of the original image by using a first remosaic method, and generate a second remosaic image by performing remosaic processing on a second region of the original image by using the first remosaic method or a second remosaic method; and
a merging circuit configured to generate a converted image having a Bayer pattern based on the first remosaic image and the second remosaic image.

2 . The imaging device of claim 1 , wherein the first remosaic method comprises a method of performing the remosaic processing by using a machine learning model, and
wherein the second remosaic method comprises a method of performing the remosaic processing based on a pre-set reference equation.

3 . The imaging device of claim 2 , wherein the machine learning model comprises a neural network model.

4 . The imaging device of claim 2 , wherein the remosaic processing circuit is further configured to:
select one or more machine learning models from among a plurality of machine learning models, based on quality of the first remosaic image and a generation time of the first remosaic image; and perform the remosaic processing on the first region to generate the first remosaic image using the one or more machine learning models.

5 . The imaging device of claim 1 , wherein the first region is a pre-set central region of the original image, and
wherein the second region is an entire region of the original image.

6 . The imaging device of claim 1 , wherein the remosaic processing circuit is further configured to detect a seam in the original image and determine the first region based on the seam.

7 . The imaging device of claim 1 , wherein the merging circuit is further configured to generate the converted image by merging the first remosaic image to a corresponding region of the second remosaic image.

8 . The imaging device of claim 7 , wherein the merging circuit is further configured to generate the converted image by performing correction processing on boundary regions of the first remosaic image and the second remosaic image after merging the first remosaic image and the second remosaic image.

9 . The imaging device of claim 1 , wherein the processor further comprises a selecting circuit configured to output one of the second remosaic image and the converted image based on a selection signal generated by the image sensor.

10 . The imaging device of claim 9 , wherein according to the selection signal, the selecting circuit is configured to output the converted image based on illuminance of the original image being equal to or greater than a pre-set reference illuminance, and output the second remosaic image based on the illuminance of the original image being less than the pre-set reference illuminance.

11 . The imaging device of claim 9 , wherein according to the selection signal, the selecting circuit is configured to output the converted image based on the original image being an image captured with a zoom ratio less than a pre-set reference zoom ratio, and output the second remosaic image based on the original image being an image captured with a zoom ratio equal to or greater than the pre-set reference zoom ratio.

12 . An image processing method comprising:
generating an original image through an image sensor; generating, through a processor, a first remosaic image by performing remosaic processing on a first region of the original image by using a first remosaic method; generating, through the processor, a second remosaic image by performing remosaic processing on a second region of the original image by using the first remosaic method or a second remosaic method; and generating a converted image having a Bayer pattern based on the first remosaic image and the second remosaic image.

13 . The image processing method of claim 12 , wherein the first remosaic method comprises a method of performing the remosaic processing by using a machine learning model, and
wherein the second remosaic method comprises a method of performing the remosaic processing based on a pre-set reference equation.

14 . The image processing method of claim 12 , wherein the generating of the first remosaic image comprises:
selecting one or more machine learning models from among a plurality of machine learning models, based on quality of the first remosaic image and a generation time of the first remosaic image; and generating the first remosaic image by performing the remosaic processing on the first region of the original image by using the one or more machine learning models.

15 . The image processing method of claim 12 , further comprising outputting, through the processor, one of the second remosaic image and the converted image.

16 . The image processing method of claim 15 , wherein the outputting comprises:
outputting the converted image based on illuminance of the original image being equal to or greater than a pre-set reference illuminance; and outputting the second remosaic image based on the illuminance of the original image being less than the pre-set reference illuminance.

17 . The image processing method of claim 15 ,
wherein the outputting comprises: outputting the converted image when a zoom ratio of the original image being less than a pre-set reference zoom ratio; and outputting the second remosaic image based on the zoom ratio being equal to or greater than the pre-set reference zoom ratio.

18 . An imaging device comprising:
an image sensor configured to generate an original image; and an image processor configured to perform remosaic processing by using first remosaic method and second remosaic method on the original image to generate a converted image having a Bayer pattern.

19 . The imaging device of claim 18 , wherein the image processor is further configured to:
generate a first remosaic image by performing remosaic processing on a first region of the original image by using a first remosaic method; generate a second remosaic image by performing remosaic processing on a second region of the original image by using the first remosaic method or a second remosaic method; and generate the converted image based on the first remosaic image and the second remosaic image.

20 . The imaging device of claim 18 , wherein the first remosaic method comprises a method of performing the remosaic processing by using a machine learning model, and
wherein the second remosaic method comprises a method of performing the remosaic processing based on a pre-set reference equation.

Assignees

• Samsung Electronics Co Ltd

Inventors

• Lee Hunsoo
• Park Soyoun
• Jung Byungwook

Classifications

• G06T3/4015 Primary

  Image demosaicing, e.g. colour filter arrays [CFA] or Bayer patterns · CPC title

• G06T2200/32

  involving image mosaicing · CPC title

• G06T3/4046 Primary

  using neural networks · CPC title

• G06N20/20

  Ensemble learning · CPC title

• G06N3/08

  Learning methods · CPC title