Generative video data stories

Patent metadata

Field	Value
Publication number	US-12537028-B2
Application number	US-202418664395-A
Country	US
Kind code	B2
Filing date	May 15, 2024
Priority date	May 15, 2024
Publication date	Jan 27, 2026
Grant date	Jan 27, 2026

Abstract

Methods, systems, and computer-readable storage media for a video generation platform that automatically generates videos, also referred to herein as stories, based on story templates, story data, and story metadata. The video generation platform provides interfaces for third-party systems to render videos and publish the videos as a story for defined channels and recipients.

First claim

What is claimed is:

1 . A computer-implemented method for programmatic generation of videos, the method being executed by one or more processors and comprising:
receiving user input comprising a description of a story that a user seeks to convey in a video; transmitting a first prompt to a large language model (LLM) system, the first prompt comprising at least a portion of the user input; receiving, from the LLM system, storyboard code representative of a set of scenes; in response to the storyboard code, for each scene in the set of scenes:
transmitting a second prompt to the LLM system, the second prompt comprising at least a portion of the storyboard code and a predefined set of layout templates, and
receiving, from the LLM system, identification of a layout template of the predefined set of layout templates for a respective scene;
for one or more layout templates in a set of layout templates identified by the LLM system, populating at least one placeholder with a data value; and displaying, in a user interface (UI), a storyboard for the video, the storyboard comprising graphical display of one or more scenes of the set of scenes.

2 . The method of claim 1 , wherein the storyboard code comprises, for each scene, a description of a visualization of the respective scene, a layout template being identified by the LLM system at least partially in response to the description of the visualization.

3 . The method of claim 1 , wherein each layout template in the predefined set of layout templates is associated with at least one description, one or more layout templates being identified by the LLM system at least partially in response to the description.

4 . The method of claim 1 , wherein populating at least one placeholder with a data value is executed by the LLM system in response to a third prompt at least partially comprising the user input.

5 . The method of claim 1 , further comprising:
receiving, from the user, revisions to an abstract displayed with a scene of the storyboard and in response, providing a modified scene using the LLM system responsive to the revisions.

6 . The method of claim 1 , wherein each layout template is provided in Javascript Object Notation (JSON).

7 . The method of claim 1 , further comprising:
receiving a request comprising a template identifier and a data payload; retrieving, from a data store, the story template based on the template identifier, the story template being of a first format and defining content elements and data to be depicted in a video; populating one or more data values of the data payload into the story template; providing, based on the story template, code in a second format that is different from the first format; generating the video based on the code; and transmitting the video to one or more users.

8 . A non-transitory computer-readable storage medium coupled to one or more processors and having instructions stored thereon which, when executed by the one or more processors, cause the one or more processors to perform operations for programmatic generation of videos, the operations comprising:
receiving user input comprising a description of a story that a user seeks to convey in a video; transmitting a first prompt to a large language model (LLM) system, the first prompt comprising at least a portion of the user input; receiving, from the LLM system, storyboard code representative of a set of scenes; in response to the storyboard code, for each scene in the set of scenes:
transmitting a second prompt to the LLM system, the second prompt comprising at least a portion of the storyboard code and a predefined set of layout templates, and
receiving, from the LLM system, identification of a layout template of the predefined set of layout templates for a respective scene;
for one or more layout templates in a set of layout templates identified by the LLM system, populating at least one placeholder with a data value; and displaying, in a user interface (UI), a storyboard for the video, the storyboard comprising graphical display of one or more scenes of the set of scenes.

9 . The non-transitory computer-readable storage medium of claim 8 , wherein the storyboard code comprises, for each scene, a description of a visualization of the respective scene, a layout template being identified by the LLM system at least partially in response to the description of the visualization.

10 . The non-transitory computer-readable storage medium of claim 8 , wherein each layout template in the predefined set of layout templates is associated with at least one description, one or more layout templates being identified by the LLM system at least partially in response to the description.

11 . The non-transitory computer-readable storage medium of claim 8 , wherein populating at least one placeholder with a data value is executed by the LLM system in response to a third prompt at least partially comprising the user input.

12 . The non-transitory computer-readable storage medium of claim 8 , wherein operations further comprise:
receiving, from the user, revisions to an abstract displayed with a scene of the storyboard and in response, providing a modified scene using the LLM system responsive to the revisions.

13 . The non-transitory computer-readable storage medium of claim 8 , wherein each layout template is provided in Javascript Object Notation (JSON).

14 . The non-transitory computer-readable storage medium of claim 8 , wherein operations further comprise:
receiving a request comprising a template identifier and a data payload; retrieving, from a data store, the story template based on the template identifier, the story template being of a first format and defining content elements and data to be depicted in a video; populating one or more data values of the data payload into the story template; providing, based on the story template, code in a second format that is different from the first format; generating the video based on the code; and transmitting the video to one or more users.

15 . A system, comprising:
a computing device; and a computer-readable storage device coupled to the computing device and having instructions stored thereon which, when executed by the computing device, cause the computing device to perform operations for programmatic generation of videos, the operations comprising:
receiving user input comprising a description of a story that a user seeks to convey in a video;
transmitting a first prompt to a large language model (LLM) system, the first prompt comprising at least a portion of the user input;
receiving, from the LLM system, storyboard code representative of a set of scenes;
in response to the storyboard code, for each scene in the set of scenes:
transmitting a second prompt to the LLM system, the second prompt comprising at least a portion of the storyboard code and a predefined set of layout templates, and
receiving, from the LLM system, identification of a layout template of the predefined set of layout templates for a respective scene;

for one or more layout templates in a set of layout templates identified by the LLM system, populating at least one placeholder with a data value; and
displaying, in a user interface (UI), a storyboard for the video, the storyboard comprising graphical display of one or more scenes of the set of scenes.

16 . The system of claim 15 , wherein the storyboard code comprises, for each scene, a description of a visualization of the respective scene, a layout template being identified by the LLM system at least partiall

Assignees

• Sap Se

Inventors

• Wittmann Michael
• Miers Henrik
• Keilbach Pascal
• Pusch Joshua
• Uflacker Matthias

Classifications

• G06F40/40

  Processing or translation of natural language (natural language analysis G06F40/20; semantic analysis G06F40/30) · CPC title

• G11B27/036 Primary

  Insert-editing · CPC title

• G11B27/34

  Indicating arrangements  {(indicating means incorporated in magazine or cassette G11B23/046 and G11B23/0875; indicating measured values in general G01D)} · CPC title

• G06F40/56

  Natural language generation · CPC title

• G06F40/186

  Templates · CPC title