Method of outputting three-dimensional image and electronic device performing the method

Patent metadata

Field	Value
Publication number	US-11711502-B2
Application number	US-202117361936-A
Country	US
Kind code	B2
Filing date	Jun 29, 2021
Priority date	Dec 3, 2018
Publication date	Jul 25, 2023
Grant date	Jul 25, 2023

Abstract

A method and apparatus for outputting a three-dimensional (3D) image are provided. To output a 3D image, a stereo image is generated based on viewpoints of a user and rendered into a 3D image. Since the stereo image is generated based on the viewpoints of the user, the user views a different side of an object appearing in the 3D image depending on a viewpoint of the user.

First claim

What is claimed is:

1. A method performed by an electronic device for outputting a three-dimensional (3D) image, the method comprising:
obtaining eye position information of a user in a coordinate system of a display panel;
calculating virtual viewpoints of the user with respect to an optical system based on the eye position information and optical system information corresponding to the optical system;
transforming the coordinate system of the display panel into a graphic space coordinate system;
setting at least two viewpoints within the graphic space coordinate system based on the eye position information;
setting an object to be output within the graphic space coordinate system by transforming a pose of the object into the graphic space coordinate system;
generating a stereo image with respect to the object to be output on a virtual screen based on the at least two viewpoints set within the graphic space coordinate system; and
rendering a 3D image based on the virtual viewpoints and the stereo image to the user through the optical system,
wherein the graphic space coordinate system is a global coordinate system of a virtual 3D space corresponding to a real space of a system including the electronic device, and
wherein the generating the stereo image comprises:
setting a first plane and a second plane within a pyramidal structure with respect to each of the at least two viewpoints within the graphic space coordinate system;
generating frustum information based on the first plane and the second plane;
determining the pose of the object based on pose information of a system in which the user is present; and
generating the stereo image based on the frustum information and the object set within the graphic space coordinate system.

2. The method of claim 1 , wherein the generating the stereo image comprises:
calculating the frustum information with respect to each of the at least two viewpoints set within the graphic space coordinate system.

3. The method of claim 2 , further comprising:
setting the virtual screen based on the eye position information,
wherein the calculating of the frustum information comprises generating the frustum information based on the virtual screen.

4. The method of claim 3 , wherein the setting of the virtual screen comprises:
determining a difference between a first eye position and a target eye position;
determining a first weight with respect to first virtual screen information preset with respect to the first eye position based on the difference between the first eye position and the target eye position;
determining target virtual screen information with respect to the target eye position based on the first virtual screen information and the first weight; and
setting the virtual screen based on the target virtual screen information.

5. The method of claim 4 , wherein the first eye position is preset within an eye box set in a space within the system.

6. The method of claim 2 , further comprising:
smoothing the at least two viewpoints set within the graphic space coordinate system,
wherein the calculating of the frustum information comprises calculating the frustum information based on the smoothed at least two viewpoints.

7. The method of claim 1 , wherein the optical system comprises the display panel configured to output the 3D image, and
the virtual viewpoints are positioned in a rear portion of the display panel.

8. The method of claim 7 , wherein the rendering comprises:
determining virtual rays passing through a pixel on the display panel based on the virtual viewpoints;
determining intersection points between the virtual rays and an optical layer; and
assigning a pixel value to the pixel based on distances between optical elements of the optical layer and the intersection points.

9. The method of claim 8 , wherein the determining of the virtual rays comprises determining the virtual rays based on a first refractive index of a medium between the optical layer and the display panel, and a second refractive index of a medium outside of a display device including the optical layer and the display panel.

10. The method of claim 1 , wherein the optical system information comprises information about an optical characteristic of a concave mirror or a convex lens included in the optical system.

11. The method of claim 1 , wherein the system is a vehicle, and
the virtual screen is set outside of the vehicle.

12. The method of claim 11 , wherein the optical system comprises a windshield of the vehicle.

13. A non-transitory computer-readable storage medium storing instructions that, when executed by a processor, cause the processor to perform the method of claim 1 .

14. An electronic device for outputting a three-dimensional (3D) image, the electronic device comprising:
a memory configured to record a program to output a 3D image; and
a processor configured to execute the program to perform:
calculating virtual viewpoints of a user within a system with respect to an optical system based on eye position information of the user in a coordinate system of a display panel and optical system information corresponding to the optical system;
transforming the coordinate system of the display panel into a graphic space coordinate system;
setting at least two viewpoints within the graphic space coordinate system based on the eye position information;
setting an object to be output within the graphic space coordinate system by transforming a pose of the object into the graphic space coordinate system;
generating a stereo image with respect to the object to be output on a virtual screen based on the at least two viewpoints set within the graphic space coordinate system; and
rendering a 3D image based on the virtual viewpoints and the stereo image to the user through the optical system,
wherein the graphic space coordinate system is a global coordinate system of a virtual 3D space corresponding to a real space of a system including the electronic device, and
wherein the generating the stereo image comprises:
setting a first plane and a second plane within a pyramidal structure with respect to each of the at least two viewpoints within the graphic space coordinate system;
generating frustum information based on the first plane and the second plane;
determining the pose of the object based on pose information of a system in which the user is present; and
generating the stereo image based on the frustum information and the object set within the graphic space coordinate system.

15. The electronic device of claim 14 , wherein the generating the stereo image comprises:
calculating the frustum information with respect to each of the at least two viewpoints set within the graphic space coordinate system.

16. The electronic device of claim 14 , further comprising:
a camera configured to generate the eye position information of the user within the system.

17. The electronic device of claim 14 , further comprising:
a display device including the display panel configured to output the rendered 3D image.

18. The electronic device of claim 14 , wherein the system is a vehicle, and the electronic device is installed in the vehicle.

19. A method of outputting a three-dimensional (3D) image, the method comprising:
obtaining eye positions of a user with respect to a camera;
transforming coordinates of the eye positions into a coordinate system of a display panel;
transforming the coordinate system of the display panel into a graphic space coordinate system;
setting two

Assignees

• Samsung Electronics Co Ltd

Inventors

• Lee Seok
• Park Juyong
• Nam Dong Kyung

Classifications

• B60K2360/149

  Instrument input by detecting viewing direction not otherwise provided for · CPC title

• B60K35/211

  producing three-dimensional [3D] effects, e.g. stereoscopic images · CPC title

• B60K35/23 Primary

  Head-up displays [HUD] (optical aspects of head-up displays G02B27/01) · CPC title

• H04N13/117 Primary

  the virtual viewpoint locations being selected by the viewers or determined by viewer tracking · CPC title

• G02B27/0101

  characterised by optical features (G02B27/0172 takes precedence) · CPC title