System and method for single image super- resolution for smart device camera

Patent metadata

Field	Value
Publication number	US-12380535-B2
Application number	US-202318360536-A
Country	US
Kind code	B2
Filing date	Jul 27, 2023
Priority date	Jul 27, 2023
Publication date	Aug 5, 2025
Grant date	Aug 5, 2025

Abstract

A method includes extracting multiple shallow features from a low-resolution image using a shallow feature extractor that includes a quaternion convolutional network. The method also includes extracting multiple deep features from the multiple shallow features using a deep feature extractor that includes multiple quaternion residual distillation blocks (QRDBs), where each QRDB includes a quaternion self-attention module. The method further includes reconstructing the multiple deep features into a high-resolution image. Each QRDB may further include a quaternion gated deconvolutional feed forward network (QGDFN) configured to suppress one or more of the multiple deep features.

First claim

What is claimed is:

1. A method comprising:
extracting multiple shallow features from a low-resolution image using a shallow feature extractor that includes a quaternion convolutional network;
extracting multiple deep features from the multiple shallow features using a deep feature extractor that includes multiple quaternion residual distillation blocks (QRDBs), each QRDB comprising a quaternion self-attention module; and
reconstructing the multiple deep features into a high-resolution image.

2. The method of claim 1 , wherein the low-resolution image comprises a red-green-blue (RGB) image captured using a camera of a smartphone.

3. The method of claim 1 , wherein extracting the multiple shallow features from the low-resolution image comprises:
extracting red, green, blue, and grayscale channels from the low-resolution image; and
processing the red, green, blue, and grayscale channels using the quaternion convolutional network to generate multiple feature vectors representing the multiple shallow features.

4. The method of claim 1 , wherein the quaternion convolutional network comprises a leaky rectified linear unit (ReLU).

5. The method of claim 1 , wherein the QRDBs are arranged in series such that an output of a previous QRDB is an input of a next QRDB.

6. The method of claim 1 , wherein each QRDB further comprises a quaternion gated deconvolutional feed forward network (QGDFN) configured to suppress one or more of the multiple deep features.

7. The method of claim 1 , wherein reconstructing the multiple deep features into the high-resolution image comprises:
processing the multiple deep features using a second quaternion convolutional network;
performing a pixel shuffle on an output of the second quaternion convolutional network; and
processing an output of the pixel shuffle using a third quaternion convolutional network to generate the high-resolution image.

8. An electronic device comprising:
at least one processing device configured to:
extract multiple shallow features from a low-resolution image using a shallow feature extractor that includes a quaternion convolutional network;
extract multiple deep features from the multiple shallow features using a deep feature extractor that includes multiple quaternion residual distillation blocks (QRDBs), each QRDB comprising a quaternion self-attention module; and
reconstruct the multiple deep features into a high-resolution image.

9. The electronic device of claim 8 , wherein the low-resolution image comprises a red-green-blue (RGB) image captured using a camera of a smartphone.

10. The electronic device of claim 8 , wherein, to extract the multiple shallow features from the low-resolution image, the at least one processing device is configured to:
extract red, green, blue, and grayscale channels from the low-resolution image; and
process the red, green, blue, and grayscale channels using the quaternion convolutional network to generate multiple feature vectors representing the multiple shallow features.

11. The electronic device of claim 8 , wherein the quaternion convolutional network comprises a leaky rectified linear unit (ReLU).

12. The electronic device of claim 8 , wherein the QRDBs are arranged in series such that an output of a previous QRDB is an input of a next QRDB.

13. The electronic device of claim 8 , wherein each QRDB further comprises a quaternion gated deconvolutional feed forward network (QGDFN) configured to suppress one or more of the multiple deep features.

14. The electronic device of claim 8 , wherein, to reconstruct the multiple deep features into the high-resolution image, the at least one processing device is configured to:
process the multiple deep features using a second quaternion convolutional network;
perform a pixel shuffle on an output of the second quaternion convolutional network; and
process an output of the pixel shuffle using a third quaternion convolutional network to generate the high-resolution image.

15. A non-transitory machine-readable medium containing instructions that when executed cause at least one processor of an electronic device to:
extract multiple shallow features from a low-resolution image using a shallow feature extractor that includes a quaternion convolutional network;
extract multiple deep features from the multiple shallow features using a deep feature extractor that includes multiple quaternion residual distillation blocks (QRDBs), each QRDB comprising a quaternion self-attention module; and
reconstruct the multiple deep features into a high-resolution image.

16. The non-transitory machine-readable medium of claim 15 , wherein the instructions that when executed cause the at least one processor to extract the multiple shallow features from the low-resolution image comprise:
instructions that when executed cause the at least one processor to:
extract red, green, blue, and grayscale channels from the low-resolution image; and
process the red, green, blue, and grayscale channels using the quaternion convolutional network to generate multiple feature vectors representing the multiple shallow features.

17. The non-transitory machine-readable medium of claim 15 , wherein the quaternion convolutional network comprises a leaky rectified linear unit (ReLU).

18. The non-transitory machine-readable medium of claim 15 , wherein the QRDBs are arranged in series such that an output of a previous QRDB is an input of a next QRDB.

19. The non-transitory machine-readable medium of claim 15 , wherein each QRDB further comprises a quaternion gated deconvolutional feed forward network (QGDFN) configured to suppress one or more of the multiple deep features.

20. The non-transitory machine-readable medium of claim 15 , wherein the instructions that when executed cause the at least one processor to reconstruct the multiple deep features into the high-resolution image comprise:
instructions that when executed cause the at least one processor to:
process the multiple deep features using a second quaternion convolutional network;
perform a pixel shuffle on an output of the second quaternion convolutional network; and
process an output of the pixel shuffle using a third quaternion convolutional network to generate the high-resolution image.

Assignees

• Samsung Electronics Co Ltd

Inventors

• Jangid Devendra Kumar
• Gnanasambandam Abhiram
• Glotzbach John W
• Lee John Seokjun
• Sheikh Hamid R

Classifications

• G06T3/4046 Primary

  using neural networks · CPC title

• H04N23/815

  for controlling the resolution by using a single image · CPC title

• H04N23/84

  for processing colour signals · CPC title

• G06T3/4053 Primary

  based on super-resolution, i.e. the output image resolution being higher than the sensor resolution · CPC title