Electronic apparatus and control method thereof

Patent metadata

Field	Value
Publication number	US-11575882-B2
Application number	US-202017135247-A
Country	US
Kind code	B2
Filing date	Dec 28, 2020
Priority date	Dec 27, 2019
Publication date	Feb 7, 2023
Grant date	Feb 7, 2023

Abstract

An electronic apparatus includes a stacked display including a plurality of panels, and a processor configured to obtain first light field (LF) images of different viewpoints, input the obtained first LF images to an artificial intelligence model for converting an LF image into a layer stack, to obtain a plurality of layer stacks to which a plurality of shifting parameters indicating depth information in the first LF images are respectively applied, and control the stacked display to sequentially and repeatedly display, on the stacked display, the obtained plurality of layer stacks. The artificial intelligence model is trained by applying the plurality of shifting parameters that are obtained based on the depth information in the first LF images.

First claim

What is claimed is:

1. An electronic apparatus comprising:
a stacked display comprising a plurality of panels; and
a processor configured to:
obtain first light field (LF) images of different viewpoints,
obtain a plurality of shifting parameters based on depth information in the first LF images,
input the first LF images and the obtained plurality of shifting parameters to an artificial intelligence model for converting an LF image into a layer stack, to obtain a first plurality of layer stacks to which the plurality of shifting parameters are respectively applied, and
reconstruct second LF images using the first plurality of layer stacks and the plurality of shifting parameters,
train the artificial intelligence model based on the first LF images and the second LF images,
input the first LF images to the trained artificial intelligence model to obtain a second plurality of layer stacks, and
control the stacked display to sequentially and repeatedly display, on the stacked display, the second plurality of layer stacks.

2. The electronic apparatus as claimed in claim 1 , wherein the second plurality of layer stacks comprise a first layer stack to which a first shifting parameter indicating first depth information in the first LF images is applied, and a second layer stack to which a second shifting parameter indicating second depth information in the first LF images is applied, and
wherein the first layer stack is for displaying a region corresponding to the first depth information in the first LF images, and the second layer stack is for displaying a region corresponding to the second depth information in the first LF images.

3. The electronic apparatus as claimed in claim 2 , wherein the second plurality of layer stacks further comprise a third layer stack, and
wherein the processor is configured to sequentially and repeatedly display, on the stacked display, the first layer stack, the second layer stack, and the third layer stack.

4. The electronic apparatus as claimed in claim 1 , wherein the processor is configured to:
compare the first LF images and the second LF images to obtain a loss function, and
train the artificial intelligence model, based on the loss function.

5. The electronic apparatus as claimed in claim 4 , wherein the processor is configured to:
respectively apply the plurality of shifting parameters to the first plurality of layer stacks, to obtain a plurality of third LF images respectively with respect to the first plurality of layer stacks, and
reconstruct the second LF images based on the obtained plurality of third LF images.

6. The electronic apparatus as claimed in claim 4 , wherein the artificial intelligence model is implemented as one among a deep neural network (DNN) model, a non-negative tensor factorization (NTF) model, and a non-negative metric factorization (NMF) model.

7. The electronic apparatus as claimed in claim 6 , wherein, based on the artificial intelligence model being the DNN model, a weight of the DNN model is updated by the obtained loss function.

8. The electronic apparatus as claimed in claim 6 , wherein, based on the artificial intelligence model being one among the NTF model and the NMF model, a parameter of the artificial intelligence model is updated by the obtained loss function.

9. The electronic apparatus as claimed in claim 1 , wherein the depth information is obtained from the first LF images by a stereo matching technique, and the plurality of shifting parameters are obtained based on the obtained depth information.

10. The electronic apparatus as claimed in claim 1 , wherein a number of the plurality of shifting parameters is a same as a number of the first plurality of layer stacks.

11. A control method of an electronic apparatus, the control method comprising:
obtaining first light field (LF) images of different viewpoints;
obtaining a plurality of shifting parameters based on depth information in the first LF images;
inputting the first LF images and the plurality of shifting parameters to an artificial intelligence model for converting an LF image into a layer stack, to obtain a first plurality of layer stacks to which the plurality of shifting parameters are respectively applied; and
reconstructing second LF images using the first plurality of layer stacks and the plurality of shifting parameters;
training the artificial intelligence model based on the first LF images and the second LF images;
inputting the first LF images to the trained artificial intelligence model to obtain a second plurality of layer stacks; and
sequentially and repeatedly displaying, on a stacked display, the second plurality of layer stacks.

12. The control method as claimed in claim 11 , wherein the second plurality of layer stacks comprise a first layer stack to which a first shifting parameter indicating first depth information in the first LF images is applied, and a second layer stack to which a second shifting parameter indicating second depth information in the first LF images is applied, and
wherein the first layer stack is for displaying a region corresponding to the first depth information in the first LF images, and the second layer stack is for displaying a region corresponding to the second depth information in the first LF images.

13. The control method as claimed in claim 12 , wherein the second plurality of layer stacks further comprise a third layer stack, and
wherein the displaying comprises sequentially and repeatedly displaying, on the stacked display, the first layer stack, the second layer stack, and the third layer stack.

14. The control method as claimed in claim 11 , further comprising:
comparing the first LF images and the second LF images to obtain a loss function; and
training the artificial intelligence model, based on the loss function.

15. The control method as claimed in claim 14 , wherein the reconstructing of the second LF images further comprises:
respectively applying the plurality of shifting parameters to the first plurality of layer stacks, to obtain a plurality of third LF images respectively with respect to the first plurality of layer stacks; and
reconstructing the second LF images based on the obtained plurality of third LF images.

16. The control method as claimed in claim 14 , wherein the artificial intelligence model is implemented as one among a deep neural network (DNN) model, a non-negative tensor factorization (NTF) model, and a non-negative metric factorization (NMF) model.

17. The control method as claimed in claim 16 , wherein, based on the artificial intelligence model being the DNN model, a weight of the DNN model is updated by the loss function.

18. The control method as claimed in claim 16 , wherein, based on the artificial intelligence model being one among the NTF model and the NMF model, a parameter of the artificial intelligence model is updated by the obtained loss function.

19. The control method as claimed in claim 11 , wherein the depth information is obtained from the first LF images by a stereo matching technique, and the plurality of shifting parameters are obtained based on the obtained depth information.

20. The control method as claimed in claim 11 , wherein a number of the plurality of shifting parameters is a same as a number of the first plurality of layer stacks.

21. A non-transitory computer-readable storage medium storing instructions, when executed by a processor, cause the processor to perform the control method of claim 11 .

Assignees

• Samsung Electronics Co Ltd

Inventors

• Jin Bora
• Lee Yeoul
• Lee Jihye
• Lim Kyungmin
• Lee Jaesung

Classifications

• G06N3/0455

  Auto-encoder networks; Encoder-decoder networks · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/09

  Supervised learning · CPC title

• H04N13/122

  Improving the three-dimensional [3D] impression of stereoscopic images by modifying image signal contents, e.g. by filtering or adding monoscopic depth cues (H04N13/128 takes precedence) · CPC title

• H04N13/128

  Adjusting depth or disparity · CPC title