Image processing method of generating an image based on a user viewpoint and image processing device

Patent metadata

Field	Value
Publication number	US-12054047-B2
Application number	US-202318305867-A
Country	US
Kind code	B2
Filing date	Apr 24, 2023
Priority date	Jul 28, 2017
Publication date	Aug 6, 2024
Grant date	Aug 6, 2024

Abstract

Provided are an image processing method and an image processing device. The image processing method includes generating an image based on viewpoint information of a user; rendering the image based on information about what is in front of the user; and outputting the rendered image using an optical element.

First claim

What is claimed is:

1. An image processing method comprising:
tracking first information including a first position of a left eye of a user and a second position of a right eye of the user;
generating second information associated with at least one of surroundings and an object present in front of the user;
rendering an image of a display panel so as to provide a first virtual image to be input to the left eye of the user and a second virtual image to be input to the right eye of the user on an imaging plane based on the first information and the second information; and
controlling the display panel to output the rendered image,
wherein a depth of a 3-Dimensional (3D) Augmented Reality (AR) object, within the depth range, represented by the first virtual image and the second virtual image is different from a depth of the imaging plane.

2. The method of claim 1 , further comprising simultaneously providing the first virtual image to be input to the left eye of the user and the second virtual image to be input to the right eye of the user on an imaging plane.

3. The method of claim 1 , further comprising:
allocating a first pixel value corresponding to the first virtual image to a first pixel of the display panel; and
allocating a second pixel value corresponding to the second virtual image to a second pixel of the display panel.

4. The method of claim 1 , wherein a disparity is present between the first virtual image and the second virtual image.

5. The image processing method of claim 1 , wherein the rendering the image comprises acquiring at least one of a shape, a position, and a depth of an object present in front of the user using a sensor.

6. The image processing method of claim 1 , wherein the generating the image comprises acquiring viewpoint information by directly detecting an eye of the user using one of an image camera and an infrared camera or by detecting the eye of the user as reflected on a windshield of a vehicle.

7. The image processing method of claim 6 , wherein the generating the image comprises:
determining a position of an eye of the user based on the viewpoint information; and
allocating an image to a plurality of sub-pixels corresponding to the position of the eye.

8. The image processing method of claim 7 , wherein the allocating the image comprises:
allocating an image to be input to a left eye of the user to a plurality of sub-pixels corresponding to a position of the left eye of the user; and
allocating an image to be input to a right eye of the user to a plurality of sub-pixels corresponding to a position of the right eye of the user.

9. The image processing method of claim 6 , wherein the generating the image comprises generating the image based on the viewpoint information and an optical transform.

10. The image processing method of claim 1 , wherein the optical element is at least one of a lenticular lens and a parallax barrier.

11. The image processing method of claim 1 , wherein the outputting the rendered image comprises enlarging the rendered image using a magnifying optical system.

12. The image processing method of claim 11 , wherein the magnifying optical system comprises at least one of an aspherical mirror and a plane mirror.

13. The image processing method of claim 1 , wherein the rendering the image comprises rendering the image so that a depth of the image is greater than a virtual image distance.

Assignees

• Samsung Electronics Co Ltd

Inventors

• Lee Jinho
• Nam Dong Kyung
• Park Juyong
• Cho Yang Ho
• Choi Kyuhwan
• Choi Yoonsun

Classifications

• G06T5/50 Primary

  using two or more images, e.g. averaging or subtraction · CPC title

• B60K35/235

  with means for detecting the driver's gaze direction or eye points · CPC title

• B60K35/233

  controlling the size or position in display areas of virtual images depending on the condition of the vehicle or the driver · CPC title

• B60K2360/785

  on or in relation to the windshield or windows · CPC title

• B60K2360/334

  Projection means · CPC title