Using prediction information with light fields

Patent metadata

Field	Value
Publication number	US-11755103-B2
Application number	US-202217722034-A
Country	US
Kind code	B2
Filing date	Apr 15, 2022
Priority date	Aug 30, 2019
Publication date	Sep 12, 2023
Grant date	Sep 12, 2023

Abstract

An example process executed by a processor of a computing device to communicate a first light field data to a user device, wherein the first light field data is limited to a first volume of space that represents a first plurality of views for display on the user device; receive a sensor data associated with the user device; predict a behavior based at least in part from the received sensor data; generate a second light field data based at least in part on the predicted behavior, wherein the second light field data is limited to a second volume of space that is different from the first volume of space and represents a second plurality of views for display on the user device; and communicate the second light field data to the user device.

First claim

What is claimed is:

1. A computer-implemented method comprising:
receiving, at a computing device, data acquired from a user device displaying virtual reality, augmented reality, or mixed reality content based on light field data using a display thereof, the data indicative of a current state of the user device, and wherein:
the data indicative of the current state of the user device includes data indicative of a display field of view of the display and sensor data associated with the user device, the sensor data including position sensor data and/or motion sensor data, and
the light field data includes light field data points defining a first three-dimensional volume;
predicting, by the computing device, a behavior based at least in part on the received sensor data;
generating, by the computing device based at least in part on the predicted behavior, further light field data for use in displaying the virtual reality, augmented reality, or mixed reality content at the user device, the further light field data including light field data points defining a second three-dimensional volume corresponding to a visual content greater than the display field of view of the user device; and
sending the further light field data to the user device.

2. The method of claim 1 , wherein the user device includes a head-mounted device and wherein the sensor data represents a motion of the head-mounted device.

3. The method of claim 1 , wherein the user device includes a wearable device and the sensor data represents a motion of the wearable device.

4. The method of claim 1 , wherein the user device includes a hand-held device and the sensor data represents a motion of the hand-held device.

5. The method of claim 1 , further comprising:
prior to receiving the data indicative of the current state of the user device, providing the first light field data to the user device, the first light field data including a first light field spatial volume,
wherein a second light field spatial volume is adjusted in size based on the sensor data associated with the user device.

6. The method of claim 1 , wherein the further light field data includes data for enabling the user device to display:
a first field of view for a user in a first viewing perspective and a second field of view for the user in a second viewing perspective different from the first viewing perspective.

7. The method of claim 1 , wherein the first or second three-dimensional volume of light field data points describes the light flowing in a plurality of directions through a plurality of light field data points.

8. The method of claim 1 , wherein the first or second three-dimensional volume of light field data points is a partial volume with respect to a determined direction of motion of the user device.

9. The method of claim 1 , wherein the light field data is based on a data model for light characterization in a volume of space.

10. A computer system comprising:
a processor; and
a memory storing instructions that, when executed by the processor, cause the computer system to:
receive data acquired from a user device displaying virtual reality, augmented reality, or mixed reality content based on light field data using a display thereof, the data indicative of the current state of the user device and including data indicative of a display field of view of the display and sensor data associated with the user device, the sensor data including position sensor data and/or motion sensor data, and
wherein the light field data includes light field data points defining a first three-dimensional volume;
predict a behavior based at least in part on the received sensor data;
generate based at least in part on the predicted behavior, further light field data for use in displaying the virtual reality, augmented reality, or mixed reality content at the user device, the further light field data including light field data points defining a second three-dimensional volume corresponding to a visual content greater than the display field of view of the user device; and
send the further light field data to the user device.

11. The computer system of claim 10 , wherein the sensor data represents a motion of at least one head-mounted device, a wearable device, or a hand-held device associated with the user device.

12. The computer system of claim 10 , wherein the instructions stored in the memory, when executed by the processor, further cause the computer system to:
prior to receiving the data indicative of the current state of the user device, provide the first light field data to the user device, the first light field data including a first light field spatial volume,
wherein a second light field spatial volume is adjusted in size based on the sensor data associated with the user device.

13. The computer system of claim 10 , wherein the further light field data includes data for enabling the user device to display:
a first field of view for a user in a first viewing perspective and a second field of view for the user in a second viewing perspective different from the first viewing perspective.

14. The computer system of claim 10 , wherein the first or second three-dimensional volume of light field data points describes the light flowing in a plurality of directions through a plurality of light field data points.

15. The computer system of claim 10 , wherein the first or second three-dimensional volume of light field data points is a partial volume with respect to a determined direction of motion of the user device.

16. The computer system of claim 10 , wherein the light field data is based on a data model for light characterization in a volume of space.

17. A non-transitory computer-readable storage medium storing instructions that, when executed by a processor of a computer system, cause the computer system to:
receive data acquired from a user device displaying virtual reality, augmented reality, or mixed reality content based on light field data using a display thereof, the data indicative of the current state of the user device and including data indicative of a display field of view of the display and sensor data associated with the user device, the sensor data including position sensor data and/or motion sensor data, and
wherein the light field data includes light field data points defining a first three-dimensional volume;
predict a behavior based at least in part on the received sensor data;
generate based at least in part on the predicted behavior, further light field data for use in displaying the virtual reality, augmented reality, or mixed reality content at the user device, the further light field data including light field data points defining a second three-dimensional volume corresponding to a visual content greater than the display field of view of the user device; and
send the further light field data to the user device.

18. The non-transitory computer-readable storage medium of claim 17 , wherein the sensor data represents a motion of at least one head-mounted device, a wearable device, or a hand-held device associated with the user device.

19. The non-transitory computer-readable storage medium of claim 17 , wherein the instructions, when executed, further cause the computer system to:
prior to receiving the data indicative of the current state of the user device, provide the first light field data to the user device, the first light field data including a first light field spatial volume,
wherein a second light field spatial volume is adjusted in size based on the sensor da

Assignees

• Shopify Inc

Inventors

• Wade Jonathan

Classifications

• G06Q30/0641 Primary

  utilising user interfaces specially adapted for shopping · CPC title

• G06F3/012 Primary

  Head tracking input arrangements · CPC title

• G06N20/00

  Machine learning · CPC title

• G06F3/011

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title

• G06T15/08

  Volume rendering · CPC title