Apparatus and method for generating view images

Patent metadata

Field	Value
Publication number	US-11480786-B2
Application number	US-201816960575-A
Country	US
Kind code	B2
Filing date	Dec 29, 2018
Priority date	Jan 12, 2018
Publication date	Oct 25, 2022
Grant date	Oct 25, 2022

Abstract

An apparatus for generating view images for a scene comprises a store ( 101 ) which stores three dimensional scene data representing the scene from a viewing region. The three dimensional scene data may e.g. be images and depth maps captured from capture positions within the viewing region. A movement processor ( 105 ) receives motion data, such as head or eye tracking data, for a user and determines an observer viewing position and an observer viewing orientation from the motion data. A change processor ( 109 ) determines an orientation change measure for the observer viewing orientation and an adapter ( 111 ) is arranged to reduce a distance from the observer viewing position relative to the viewing region in response to the orientation change measure. An image generator ( 103 ) generates view images for the observer viewing position and the observer viewing orientation from the scene data.

First claim

The invention claimed is:

1. An apparatus for generating view images for a scene, the apparatus comprising:
a memory circuit,
wherein the memory circuit is arranged to store three dimensional scene data,
wherein the three dimensional scene data represents the scene from a viewing region, and
wherein the three dimensional scene data comprises an Omni Directional Stereo with depth representation of the scene at the viewing region;

a movement processor circuit,
wherein the movement processor circuit is arranged to receive motion data,
wherein the motion data is associated with a motion of a user,
wherein the movement processor circuit is arranged to determine an observer viewing position and an observer viewing orientation from the motion data;

a change processor circuit, wherein the change processor circuit is arranged to determine an orientation change measure for the observer viewing orientation;
an adapter circuit,
wherein the adapter circuit is arranged to reduce a distance between the observer viewing position and a center position for a view circle for the Omni Directional Stereo with depth representation at the viewing region in response to the orientation change measure,
wherein reducing the distance moves the observer viewing position closer to the viewing region; and
an image generator circuit,
wherein the image generator circuit is arranged to generate view images at the observer viewing orientation from the observer viewing position at the reduced distance using the three dimensional scene data.

2. The apparatus of claim 1 , wherein the adapter circuit is arranged to reduce the distance only when the orientation change measure exceeds a threshold.

3. The apparatus of claim 1 , wherein the adapter circuit is arranged to gradually reduce the distance.

4. The apparatus of claim 1 , wherein a rate of reduction of the distance is a monotonically increasing function of the orientation change measure.

5. The apparatus of claim 4 , wherein the adapter circuit is arranged to impose an upper limit on the rate of reduction.

6. The apparatus of claim 1 , wherein the motion data comprises head motion tracking data.

7. The apparatus of claim 1 , wherein the motion data comprises eye-pupil tracking data.

8. The apparatus of claim 1 ,
wherein the motion data comprises head motion tracking data,
wherein the motion data comprises eye-pupil tracking data,
wherein the change processor circuit is arranged to determine the orientation change measure in response to relative movements indicated by the head motion tracking data and the eye-pupil tracking data.

9. The apparatus of claim 1 , wherein the adapter circuit is arranged to reduce the distance by modifying coordinates for the three dimensional scene data.

10. The apparatus of claim 1 ,
wherein the three dimensional scene data is referenced to a first reference coordinate system,
wherein the observer viewing position is referenced to a second reference coordinate system,
wherein adapter circuit is arranged to modify a first reference coordinate and a second reference coordinate for the three dimensional scene data by changing a mapping from the first reference coordinate system to the second reference coordinate system.

11. The apparatus of claim 10 , wherein the image generator circuit is arranged to generate view images for views not on the view circle by performing viewpoint shifting based on views on the view circle and the depth.

12. The apparatus of claim 1 ,
wherein the apparatus is arranged to execute a virtual reality or augmented reality application,
wherein the view images are view images for an observer of the virtual reality or augmented reality application.

13. A method of generating view images for a scene, the method comprising:
storing a three dimensional scene data,
wherein the three dimensional scene data comprises an Omni Directional Stereo with depth representation of the scene,
wherein the three dimensional scene data represents the scene from a viewing region;
receiving motion data for a user;
determining an observer viewing position and an observer viewing orientation from the motion data;
determining an orientation change measure for the observer viewing orientation;
reducing a distance between the observer viewing position and a center position for a view circle for the Omni Directional Stereo with depth representation in response to the orientation change measure to move the observer viewing position closer to the viewing region; and
generating view images at the observer viewing orientation from the observer viewing position at the reduced distance using the three dimensional scene data.

14. A non-transitory computer readable medium storing instructions for generating view images for a scene, wherein when executed on a processor, cause the processor to:
store a three dimensional scene data,
wherein the three dimensional scene data comprises an Omni Directional Stereo with depth representation of the scene,
wherein the three dimensional scene data represents the scene from a viewing region;
receive motion data for a user;
determine an observer viewing position and an observer viewing orientation from the motion data;
determine an orientation change measure for the observer viewing orientation;
reduce a distance between the observer viewing position and a center position for a view circle for the Omni Directional Stereo with depth representation in response to the orientation change measure to move the observer viewing position closer to the viewing region; and
generate view images at the observer viewing orientation from the observer viewing position at the reduced distance using the three dimensional scene data.

15. The method of claim 13 , further comprising reducing the distance only when the orientation change measure exceeds a threshold.

16. The method of claim 13 , further comprising gradually reducing the distance.

17. The method of claim 13 , wherein a rate of reduction of the distance is a monotonically increasing function of the orientation change measure.

18. The method of claim 17 , further comprising imposing an upper limit on the rate of reduction.

19. The method of claim 13 , wherein the motion data comprises head motion tracking data.

20. The method of claim 13 , wherein the motion data comprises eye-pupil tracking data.

Assignees

• Koninklijke Philips Nv

Inventors

• Bruls Wilhelmus Hendrikus Alfonsus
• VAREKAMP Christian

Classifications

• G02B30/20

  by providing first and second parallax images to an observer's left and right eyes · CPC title

• H04N13/366

  using viewer tracking · CPC title

• G02B27/0093 Primary

  with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking · CPC title

• H04N13/279

  the virtual viewpoint locations being selected by the viewers or determined by tracking · CPC title

• G06F3/013

  Eye tracking input arrangements (G06F3/015 takes precedence) · CPC title