Apparatus and method for head mountable displays

Patent metadata

Field	Value
Publication number	US-2025124664-A1
Application number	US-202418907634-A
Country	US
Kind code	A1
Filing date	Oct 7, 2024
Priority date	Oct 11, 2023
Publication date	Apr 17, 2025
Grant date	—

Abstract

An apparatus comprises receiving circuitry to receive tracking information indicative of tracked poses of a head mountable display “HMD”, rendering circuitry to render images for display having viewpoints corresponding to the tracked poses of the HMD, trajectory calculation circuitry to calculate a smoothed trajectory for position of the HMD in dependence on the tracking information, first image re-projection circuitry to re-project at least some of the rendered images according to a first re-projection operation to obtain first re-projected images for display by another display device, in which the first image re-projection circuitry is configured to re-project a given rendered image having a given viewpoint position for a different viewpoint position corresponding to the smoothed trajectory for the HMD, and output circuitry to output the first re-projected images for display by the another display device.

First claim

1 . Apparatus comprising:
receiving circuitry to receive tracking information indicative of tracked poses of a head mountable display “HMD”; rendering circuitry to render images for display having viewpoints corresponding to the tracked poses of the HMD; trajectory calculation circuitry to calculate a smoothed trajectory for position of the HMD in dependence on the tracking information; first image re-projection circuitry to re-project at least some of the rendered images according to a first re-projection operation to obtain first re-projected images for display by another display device, in which the first image re-projection circuitry is configured to re-project a given rendered image having a given viewpoint position for a different viewpoint position corresponding to the smoothed trajectory for the position of the HMD; and output circuitry to output the first re-projected images for display by the another display device.

2 . The apparatus according to claim 1 , wherein the trajectory calculation circuitry is configured to calculate the smoothed trajectory in dependence on a smoothing calculation using the received tracking information.

3 . The apparatus according to claim 2 , wherein the trajectory calculation circuitry is configured to calculate the smoothed trajectory in dependence on a moving average calculation using the received tracking information.

4 . The apparatus according to claim 2 , wherein the smoothed trajectory comprises a plurality of calculated positions for the HMD for a period of time and the trajectory calculation circuitry is configured to calculate a next position for the smoothed trajectory in dependence on a weighted average of a most recent calculated position for the smoothed trajectory and a current tracked position of the HMD with a greater weighting for the most recent calculated position.

5 . The apparatus according to claim 1 , wherein the trajectory calculation circuitry is configured to calculate an extrapolated trajectory for the position of the HMD in dependence on at least a portion of the smoothed trajectory, the extrapolated trajectory comprising one or more predicted positions for the HMD.

6 . The apparatus according to claim 4 , wherein the first image re-projection circuitry is configured to re-project the given rendered image having the given viewpoint position for the different viewpoint position, in which the different viewpoint position corresponds to one of:
a calculated position associated with the smoothed trajectory; and a predicted position for the HMD.

7 . The apparatus according to claim 1 , wherein the first image re-projection circuitry is configured to re-project the given rendered image having the given viewpoint position for the different viewpoint position corresponding to a calculated position on the smoothed trajectory for a respective time that is between a start time of a rendering operation for rendering the given rendered image and a time at which the first image re-projection operation is to be performed for the given rendered image.

8 . The apparatus according to claim 1 , wherein the trajectory calculation circuitry is configured to calculate a smoothed trajectory for the pose of the HMD in dependence on the tracking information, the smoothed trajectory for the pose of the HMD comprising a plurality of calculated poses for the HMD for a period of time, and the first image re-projection circuitry is configured to re-project the given rendered image having a given viewpoint pose for a different viewpoint pose corresponding to the smoothed trajectory for the pose of the HMD.

9 . The apparatus according to claim 1 , comprising second image re-projection circuitry configured to re-project at least some of the rendered images according to a second re-projection operation to obtain second re-projected images for display by the HMD, wherein the output circuitry is configured to output the second re-projected images for display by the HMD.

10 . The apparatus according to claim 9 , wherein the second image re-projection circuitry is configured to re-project the given rendered image according to the second re-projection operation to re-project the given rendered image for a current viewpoint corresponding to a current tracked pose of the HMD.

11 . The apparatus according to claim 1 , wherein the rendering circuitry is configured to render at least some of the images for display having a field of view greater than a display field of view associated with the HMD.

12 . The apparatus according to claim 1 , wherein the first image re-projection circuitry is configured to re-project one or more rendered images according to the first re-projection operation to obtain one or more first re-projected images for display by the another display device, in which one or more of the first re-projected images comprise a portion outside a field of view of one or more of the rendered images, and wherein the first image re-projection circuitry is configured to use image data from one or more image sources for the portion outside the field of view.

13 . The apparatus according to claim 1 , comprising detection circuitry to detect image-based motion vector information for at least some of the rendered images, and wherein the first image re-projection circuitry is configured to re-project the given rendered image for the different viewpoint in dependence on the image-based motion vector information.

14 . A computer implemented method comprising:
receiving tracking information indicative of tracked poses of a head mountable display “HMD”; rendering images for display having viewpoints corresponding to the tracked poses of the HMD; calculating a smoothed trajectory for position of the HMD in dependence on the tracking information; re-projecting at least some of the rendered images according to a first re-projection operation to obtain first re-projected images for display by another display device; in which re-projecting a given rendered image having a given viewpoint position according to the first re-projection operation comprises re-projecting the given rendered for a different viewpoint position corresponding to the smoothed trajectory for the position of the HMD; and outputting the first re-projected images for display by the another display device.

15 . A non-transitory computer-readable storage medium storing computer software which, when executed by a computer, causes the computer to a method comprising:
receiving tracking information indicative of tracked poses of a head mountable display “HMD”; rendering images for display having viewpoints corresponding to the tracked poses of the HMD; calculating a smoothed trajectory for position of the HMD in dependence on the tracking information; re-projecting at least some of the rendered images according to a first re-projection operation to obtain first re-projected images for display by another display device; in which re-projecting a given rendered image having a given viewpoint position according to the first re-projection operation comprises re-projecting the given rendered for a different viewpoint position corresponding to the smoothed trajectory for the position of the HMD; and outputting the first re-projected images for display by the another display device.

Assignees

• Sony Interactive Entertainment Inc

Inventors

• Green Lawrence Martin
• Leung Jun Yen
• Cerrato Maurizio

Classifications

• G02B27/017

  Head mounted · CPC title

• G02B27/0093

  with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking · CPC title

• G06T19/003 Primary

  Navigation within 3D models or images · CPC title

• A63F2300/8082

  Virtual reality · CPC title

• A63F13/86

  Watching games played by other players · CPC title