Creation and user interactions with three-dimensional wallpaper on computing devices

Patent metadata

Field	Value
Publication number	US-11240481-B2
Application number	US-201916704592-A
Country	US
Kind code	B2
Filing date	Dec 5, 2019
Priority date	Dec 26, 2018
Publication date	Feb 1, 2022
Grant date	Feb 1, 2022

Abstract

A wallpaper system presents a first wallpaper image of a wallpaper video to a user and receives, via a user input device, one or both of: (i) a spatial user input selection, and (ii) a time user input selection from the user to apply to the wallpaper video. In response to detecting one or both of: (i) the spatial user input selection, and (ii) the time user input selection, the wallpaper system determines one or both of: (i) a respective spatial movement parameter within a wallpaper video associated with the spatial user input selection, and (ii) a respective time coordinate within the wallpaper video associated with the time user input selection. Wallpaper system presents, via the image display, a second wallpaper image associated with one or both of: (i) the respective spatial movement parameter, and (ii) the respective time coordinate.

First claim

What is claimed is:

1. A wallpaper system comprising:
an image display for presenting a wallpaper video including a sequence of wallpaper images, wherein:
the wallpaper images are two-dimensional (2D) and based on raw images or processed raw images,
each of the wallpaper images is associated with a respective time coordinate on a time (T) axis for a presentation time and a respective spatial movement parameter of a respective original image of an original video, wherein upon selection of the original video as wallpaper by the user, a wallpaper image matrix is created and persistently stored in the memory, including respective sets of wallpaper images of the wallpaper video, such that:
each respective set of wallpaper images provides an appearance of a spatial movement or rotation around the respective original image of the original video, and
each wallpaper image within the respective set of wallpaper images corresponds to a different spatial movement parameter within the respective original image;

an image display driver coupled to the image display to control the image display to present the wallpaper video;
a user input device to receive from a user: (i) a spatial user input selection that manipulates a spatial movement parameter of the wallpaper images, and (ii) a time user input selection to apply to the wallpaper video to move the wallpaper video forwards or backwards in time;
a memory;
a processor coupled to the image display driver, the user input device, and the memory; and
wallpaper programming in the memory, wherein execution of the wallpaper programming by the processor configures the wallpaper system to perform functions, including functions to:
present, via the image display, a first wallpaper image of the wallpaper video to the user;
receive, via the user input device, from the user: (i) the spatial user input selection, and (ii) the time user input selection to apply to the wallpaper video; and
in response to receiving: (i) the spatial user input selection, and (ii) the time user input selection:
determine: (i) the respective spatial movement parameter associated with the spatial user input selection, and (ii) the respective time coordinate associated with the time user input selection; and
present, via the image display, a second wallpaper image at an angular orientation determined by the respective spatial movement parameter and at a point in time of the wallpaper video determined by the respective time coordinate.

2. The wallpaper system of claim 1 , wherein:
a mobile device includes the user input device;
the user input device includes:
a touch sensor including an input surface and a sensor array that is coupled to the input surface to receive at least one finger contact inputted from a user;
a sensing circuit integrated into or connected to the touch sensor and connected to the processor, the sensing circuit configured to measure voltage to track the at least one finger contact on the input surface;

the function of receiving, via the user input device, from the user: (i) the spatial user input selection, and (ii) the time user input selection to apply to the wallpaper video includes:
receiving on the input surface of the touch sensor the at least one finger contact inputted from the user;
tracking, via the sensing circuit, the at least one finger contact on the input surface; and
detecting: (i) the spatial user input selection, and (ii) the time user input selection to apply to the wallpaper video on the input surface of the touch sensor based on the at least one finger contact from the user.

3. The wallpaper system of claim 2 , wherein:
the time user input selection is detected; and
the time user input selection includes a vertical swipe type of finger gesture on the touch sensor.

4. The wallpaper system of claim 1 , wherein:
a mobile device includes the user input device;
the user input device includes a movement tracker coupled to the processor to track movement of the mobile device, the movement tracker including:
(i) at least one accelerometer to measure acceleration of the mobile device,
(ii) at least one gyroscope to measure rotation of the mobile device, or
(iii) an inertial measurement unit (IMU) having the at least one accelerometer and the at least one gyroscope; and

the function of receiving, via the user input device, from the user: (i) the spatial user input selection, and (ii) the time user input selection from the user to apply to the wallpaper video includes:
tracking, via the movement tracker, movement of the mobile device by:
(i) measuring, via the at least one accelerometer, the acceleration of the mobile device,
(ii) measuring, via the at least one gyroscope, the rotation of the mobile device, or
(iii) measuring, via the inertial measurement unit, both the acceleration and the rotation of the mobile device; and

detecting: (i) the spatial user input selection, and (ii) the time user input selection to apply to the wallpaper video by detecting at least one variation of the tracked movement over a time period.

5. The wallpaper system of claim 4 , wherein:
the spatial user input selection is detected; and
the spatial user input selection includes horizontal tilting of the mobile device.

6. The wallpaper system of claim 1 , further comprising:
an eyewear device including:
a frame;
a temple connected to a lateral side of the frame; and
a depth-capturing camera, wherein the depth-capturing camera is supported by at least one of the frame or the temple and includes:
(i) at least two visible light cameras with overlapping fields of view, or (ii) a least one visible light camera and a depth sensor.

7. The wallpaper system of claim 6 , wherein:
the depth-capturing camera includes the at least two visible light cameras with overlapping fields of view;
the at least two visible light cameras include a left visible light camera to capture a left raw image and a right visible light camera to capture a right raw image;
execution of the wallpaper programming by the processor configures the wallpaper system to perform functions, including functions to generate, the second wallpaper image by:
calculating: (i) a left image disparity map between a left pixel matrix of pixels and a right pixel matrix of pixels, and (ii) a right image disparity map between the right pixel matrix and the left pixel matrix, such that:
the left raw image or a left processed image include the left pixel matrix, and
the right raw image or a right processed image include the right pixel matrix;

determining the respective spatial movement parameter of the left pixel matrix and the right pixel matrix along an X axis for horizontal position movement;
filling up a left interpolated pixel matrix by moving pixels in the left pixel matrix along the X axis based on the respective spatial movement parameter;
filling up a right interpolated pixel matrix by moving pixels in the right pixel matrix along the X axis based on the spatial movement parameter; and
blending together the left interpolated pixel matrix and the right interpolated pixel matrix to create the second wallpaper image.

8. The wallpaper system of claim 7 , wherein:
the right interpolated pixel matrix is filled up by moving pixels in the right pixel matrix along the X axis further based on a complement of the spatial movement parameter.

9. The wallpaper system of claim 1 , wherein:
the depth-capturing camera includes the at least two visible light cameras with overlapping fields of view;
the at least two visible light cameras include a left visible light camera to capture a left raw image and a right visible light camera to capture a right raw image;
the function of creating, the wallpaper

Assignees

• Snap Inc

Inventors

• Katz Sagi
• Melamed Guy

Classifications

• G02B2027/0178

  Eyeglass type (eyeglass details G02C) · CPC title

• G02B27/017

  Head mounted · CPC title

• G06F3/04815

  Interaction with a metaphor-based environment or interaction object displayed as three-dimensional [3D], e.g. changing the user viewpoint with respect to the environment or object · CPC title

• G01P15/14

  by making use of gyroscopes (gyroscopes per se G01C19/00) · CPC title

• G06F3/04883

  for inputting data by handwriting, e.g. gesture or text · CPC title