Image processing method and system

Patent metadata

Field	Value
Publication number	US-2025148613-A1
Application number	US-202418923602-A
Country	US
Kind code	A1
Filing date	Oct 22, 2024
Priority date	Nov 3, 2023
Publication date	May 8, 2025
Grant date	—

Abstract

There is provided an image processing method. The method comprises obtaining a first velocity of one or more parts of a second image frame between a first image frame and the second image frame. The first image frame and the second image frame are frames for display at a head mounted display, HMD. The method further comprises receiving motion data indicative of motion of the HMD between a display time of the first image frame and a display time of the second image frame, adjusting the first velocity based on the received motion data to determine an adjusted velocity of the one or more parts of the second image frame between the first image frame and the second image frame, and performing one or more image processing operations on the second image frame based on the adjusted velocity of the one or more parts of the second image frame.

First claim

1 . An image processing method comprising:
obtaining a first velocity of one or more parts of a second image frame between a first image frame and the second image frame, wherein the first image frame and the second image frame are frames for display at a head mounted display, HMD; receiving motion data indicative of motion of the HMD between a display time of the first image frame and a display time of the second image frame; adjusting the first velocity based on the received motion data to determine an adjusted velocity of the one or more parts of the second image frame between the first image frame and the second image frame; and performing one or more image processing operations on the second image frame based on the adjusted velocity of the one or more parts of the second image frame.

2 . The method of claim 1 , wherein the one or more image processing operations comprise one or more selected from the list consisting of: motion-blurring, temporal anti-aliasing, and depth of field processing.

3 . The method of claim 1 , further comprising outputting the processed second image frame to the HMD for display.

4 . The method of claim 1 , wherein adjusting the first velocity based on the received motion data comprises subtracting a function of the motion data from the first velocity.

5 . The method of claim 4 , wherein, for each image part of the one or more parts of the second image frame, the function of the motion data is further based on a position of the image part.

6 . The method of claim 1 , wherein obtaining the first velocity comprises retrieving the first velocity from a memory buffer.

7 . The method of claim 6 , wherein adjusting the first velocity comprises directly adjusting the first velocity stored in the memory buffer.

8 . The method of claim 1 , wherein obtaining the first velocity comprises determining the first velocity based on motion of the respective parts of the second image frame between the first image frame and the second image frame.

9 . The method of claim 8 , wherein determining the first velocity comprises: generating an image comprising the one or more parts of the second image frame, using a position of the one or more parts of the second image frame and of a virtual camera from the first image frame; and determining a change in position of the one or more parts of the second image frame between the generated image and the second image frame.

10 . The method of claim 1 , wherein each part of the second image frame comprises a pixel of the second image frame.

11 . The method of claim 1 , further comprising storing the adjusted velocity of the one or more parts of the second image frame in a memory buffer.

12 . The method of claim 1 , further comprising, upon determining that the motion of the HMD is below a predetermined threshold, performing one or more image processing operations on the second image frame based on the first velocity, without adjusting the first velocity.

13 . The method of claim 1 , further comprising detecting the motion data using one or more sensors, wherein the one or more sensors comprise one or more selected from the list consisting of: one or more motion sensors, one or more cameras operable to capture images of the HMD, and one or more cameras mounted on the HMD and operable to capture images of an environment around the HMD.

14 . A non-transitory computer-readable medium comprising computer executable instructions adapted to cause a computer system to perform an image processing method comprising:
obtaining a first velocity of one or more parts of a second image frame between a first image frame and the second image frame, wherein the first image frame and the second image frame are frames for display at a head mounted display, HMD; receiving motion data indicative of motion of the HMD between a display time of the first image frame and a display time of the second image frame; adjusting the first velocity based on the received motion data to determine an adjusted velocity of the one or more parts of the second image frame between the first image frame and the second image frame; and performing one or more image processing operations on the second image frame based on the adjusted velocity of the one or more parts of the second image frame.

15 . The non-transitory computer-readable medium of claim 14 , wherein the one or more image processing operations comprise one or more selected from the list consisting of: motion-blurring, temporal anti-aliasing, and depth of field processing.

16 . The non-transitory computer-readable medium of claim 14 , wherein adjusting the first velocity based on the received motion data comprises subtracting a function of the motion data from the first velocity.

17 . The non-transitory computer-readable medium of claim 16 , wherein, for each image part of the one or more parts of the second image frame, the function of the motion data is further based on a position of the image part.

18 . The non-transitory computer-readable medium of claim 14 , wherein obtaining the first velocity comprises determining the first velocity based on motion of the respective parts of the second image frame between the first image frame and the second image frame.

19 . The non-transitory computer-readable medium of claim 18 , wherein determining the first velocity comprises: generating an image comprising the one or more parts of the second image frame, using a position of the one or more parts of the second image frame and of a virtual camera from the first image frame; and determining a change in position of the one or more parts of the second image frame between the generated image and the second image frame.

20 . An image processing system comprising:
a first velocity processor configured to obtain a first velocity of one or more parts of a second image frame between a first image frame and the second image frame, wherein the first image frame and the second image frame are frames for display at a head mounted display, HMD; an input processor configured to receive motion data indicative of motion of the HMD between a display time of the first image frame and a display time of the second image frame; an adjustment processor configured to adjust the first velocity based on the received motion data to determine an adjusted velocity of the one or more parts of the second image frame between the first image frame and the second image frame; and an image processor configured to perform one or more image processing operations on the second image frame based on the adjusted velocity of the one or more parts of the second image frame.

Assignees

• Sony Interactive Entertainment Inc

Inventors

• Gupta Rajeev
• Connor Patrick John

Classifications

• G06T5/70

  Denoising; Smoothing · CPC title

• G02B2027/014

  comprising information/image processing systems · CPC title

• G09G2320/0261

  in the context of movement of objects on the screen or movement of the observer relative to the screen · CPC title

• H04N13/344

  with head-mounted left-right displays · CPC title

• G09G5/37

  Details of the operation on graphic patterns (G09G5/38 takes precedence) · CPC title