Device and method for interactive video presentation

Patent metadata

Field	Value
Publication number	US-11418848-B2
Application number	US-202017008326-A
Country	US
Kind code	B2
Filing date	Aug 31, 2020
Priority date	Aug 31, 2020
Publication date	Aug 16, 2022
Grant date	Aug 16, 2022

Abstract

A method for interactive video presentation includes: obtaining, by an electronic device, video data corresponding to a story; presenting, through a display interface, a portion of the video data corresponding to a selected storyline path of the story; receiving, by the input interface, a user request for switching between a two-dimensional (2D) video streaming mode and a three-dimensional (3D) exploration mode; and in response to the user request being switching from the 2D video streaming mode to the 3D exploration mode: acquiring, by the processor, 3D video scenes with exploration options for an avatar, the 3D video scenes matched to a current story status and currently presented video data; and presenting, through the display interface, the 3D video scenes with the exploration options.

First claim

What is claimed is:

1. A method for generating and presenting a video by an electronic device having a processor, comprising:
obtaining, by the electronic device, video data corresponding to a story;
presenting, through a display interface, a portion of the video data corresponding to a selected storyline path of the story;
receiving, by the input interface, a user request for switching between a two-dimensional (2D) video streaming mode and a three-dimensional (3D) exploration mode; and
in response to the user request being switching from the 2D video streaming mode to the 3D exploration mode: acquiring, by the processor, 3D video scenes with exploration options for an avatar, the 3D video scenes matched to a current story status and currently presented video data; and presenting, through the display interface, the 3D video scenes with the exploration options.

2. The method according to claim 1 , wherein acquiring 3D video scenes with exploration options includes:
generating a movement path for the avatar in a 3D space; and
acquiring 3D environment scene changes according to the movement path.

3. The method according to claim 2 , wherein acquiring 3D video scenes with exploration options further includes:
receiving user instructions to operate the avatar in the 3D space; and
generating the movement path according to the user instructions.

4. The method according to claim 3 , wherein acquiring 3D video scenes with exploration options further includes:
generating and presenting animation contents corresponding to interactions between an object in the 3D space and the avatar.

5. The method according to claim 4 , wherein the interactions between the object and the avatar includes a dialog, and the object expresses, during the dialog, one or more of: a lawful opinion, a chaotic opinion, a wise opinion, an educational opinion, or a random opinion.

6. The method according to claim 5 , wherein the 3D video scenes with exploration options are generated using neural network processing based on trained models of character responses.

7. The method according to claim 1 , wherein:
the story is hyper-story including a plurality of storyline paths, and further including a plurality of character statuses corresponding to a plurality of story characters at a plurality of time points of the story; and
the method further includes:
receiving, via an input interface, a user input of a preferred story development; and
identifying, by the processor, the selected storyline path from the plurality of storyline paths of the hyper-story according to the preferred story development.

8. The method according to claim 7 , wherein:
the user input for the preferred story development includes a voice input or a hand gesture input; and
the method further includes interpreting the user input to determine the preferred story development, including:
performing a natural language processing to convert the user input to a wish command; or
performing a hand gesture analysis to determine a user selection from a menu displayed by the displayed interface.

9. The method according to claim 7 , wherein identifying the selected storyline path according to the preferred story development includes:
obtaining, by the processor, a directed acyclic graph (DAG) matrix having a size of M×N to represent a DAG space of the hyper-story, M being a number of all combinations of character statuses for the plurality of story characters, N being a number of the time points, and each vertex of the matrix representing a combination of character status at a time point;
identifying, by the processor, a starting vertex in the matrix corresponding to a current time point and a current combination of character statuses;
performing, by the processor, a minimization process of a wish-realization-distance (WRD) function to identify a connection path connecting a plurality of vertexes in the DAG space, the connection path starting at the starting vertex and ending at an outcome vertex, wherein the minimization process dynamically measures and minimizes a distance between the preferred story development and a realization process represented by the connection path while taking into account one or more of: a matching degree of wish and realization, a story and video quality of the realization process, a latency between wish making and realization, and potential user exploration opportunities during the process.

10. The method according to claim 9 , wherein the minimization process of the WRD function includes:
calculating, by the processor, a matching degree between the preferred story development and the outcome of the connection path;
calculating, by the processor, a latency factor accounting for a minimum time duration of all paths between the starting vertex and the outcome vertex,
calculating, by the processor, an opportunity factor accounting for a maximum total number of scenes of all paths between the starting vertex and the outcome vertex;
calculating, by the processor, a quality factor accounting for a minimum quality of all steps in the connection path;
calculating, by the processor, the WRD function as a weighted sum of the matching degree, the latency factor, and the opportunity factor, wherein a first weighting coefficient for the matching degree<=0, a second weighting coefficient for the latency factor>=0, and a third weighting coefficient for the opportunity factor>=0; and
identifying the connection path by finding a path corresponding to a minimum value of the WRD function while maintaining the quality factor above a threshold.

11. The method according to claim 9 , further comprising:
before presenting the portion of the video data corresponding to the selected storyline path, notifying, by the display interface, the latency for the realization and a confidence level of the realization.

12. A device for performing computer-aided diagnosis (CAD) based on a medical image, comprising:
a memory, storing computer-executable instructions; and
a processor, coupled with the memory and, when the computer-executable instructions being executed, configured to:
obtain video data corresponding to a story;
present, via a display interface, a portion of the video data corresponding to a selected storyline path of the story;
receive a user request for switching between a two-dimensional (2D) video streaming mode and a three-dimensional (3D) exploration mode; and
in response to the user request being switching from the 2D video streaming mode to the 3D exploration mode: acquire 3D video scenes with exploration options for an avatar, the 3D video scenes matched to a current story status and currently presented video data; and presenting, via the display interface, the 3D video scenes with the exploration options.

13. The device according to claim 12 , wherein the processor is further configured to:
generate a movement path for the avatar in a 3D space; and
acquire 3D environment scene changes according to the movement path.

14. The device according to claim 13 , wherein the processor is further configured to:
receive user instructions to operate the avatar in the 3D space; and
generate the movement path according to the user instructions.

15. The device according to claim 14 , wherein the processor is further configured to:
generate and present animation contents corresponding to interactions between an object in the 3D space and the avatar.

16. The device according to claim 11 , wherein:
the story is hyper-story including a plurality of storyline paths, and further including a plurality of character statuses co

Assignees

• Tcl Res America Inc

Inventors

• Wang Haohong

Classifications

• G06N3/092

  Reinforcement learning · CPC title

• H04N21/47217

  for controlling playback functions for recorded or on-demand content, e.g. using progress bars, mode or play-point indicators or bookmarks (specific graphical features in visual interfaces H04N21/4312) · CPC title

• H04N21/485 Primary

  End-user interface for client configuration · CPC title

• H04N21/8146

  involving graphical data, e.g. 3D object, 2D graphics · CPC title

• A63F13/00

  Video games, i.e. games using an electronically generated display having two or more dimensions · CPC title