Generating artwork tutorials

Patent metadata

Field	Value
Publication number	US-12079908-B2
Application number	US-202117547942-A
Country	US
Kind code	B2
Filing date	Dec 10, 2021
Priority date	Jan 18, 2021
Publication date	Sep 3, 2024
Grant date	Sep 3, 2024

Abstract

A computer-implemented method and an apparatus for generating an action-to-feature mapping of a visual artwork may include initializing a system to create a visual artwork in, and obtaining an input stream of data recording a creation of a visual artwork from the system, and identifying, based on the input stream of data, at least one time series of utensil actions capable of reproducing the visual artwork, and obtaining at least one image of the visual artwork, and generating the action-to-feature mapping based on the at least one time series of utensil actions and on the at least one image of the visual artwork, thereby recognizing at least one artistic feature of the visual artwork. The action-to-feature mapping may include at least one pair of at least one utensil action of the at least one time series of utensil actions and the at least one artistic feature.

First claim

The invention claimed is:

1. A computer-implemented method for generating and running a creative tutorial algorithm for an action-to-feature mapping of a visual artwork, comprising:
initializing a system to create the visual artwork by obtaining data defining a creative objective and obtaining information about at least one targeted artistic style from a plurality of predetermined artistic styles;
generating, via the creative tutorial algorithm, one or more instructions on how to reproduce the creative objective based on the at least one artistic style;
obtaining an input stream of data recording a creation of the visual artwork from the system;
identifying, based on the input stream of data, at least one time series of utensil actions capable of reproducing the visual artwork;
obtaining at least one image of the visual artwork; and
generating the action-to-feature mapping based on the at least one time series of utensil actions and on the at least one image of the visual artwork, thereby recognizing at least one artistic feature of the visual artwork,
wherein the action-to-feature mapping comprises at least one pair of the at least one artistic feature and at least one utensil action of the at least one time series of utensil actions, wherein the creative tutorial algorithm compares the at least one artistic feature to the creative objective based on the targeted artistic style, and wherein the system comprises at least one smart utensil, and wherein the at least one smart utensil comprises a smart pen, a smart wrist device for recording the creation of the visual artwork, a knife for creating the visual artwork, a spatula, a palette knife for creating the visual artwork, a chisel for modeling the visual artwork, at least one hand of a user for molding the visual artwork, or a combination thereof.

2. The computer-implemented method of claim 1 , wherein the at least one smart utensil comprises a smart pen.

3. The computer-implemented method of claim 1 , wherein the system comprises a medium the visual artwork is created on or in, and wherein initializing the system to create the visual artwork comprises applying an initialization algorithm configured to establish a coordinate system of a portion of the medium.

4. The computer-implemented method of claim 1 , wherein obtaining the input stream of the data recording the creation of the visual artwork from the system comprises:
recording at least one time series of utensil track data of the smart utensil as the creation of the visual artwork progresses until completion, wherein the utensil track data comprise motion data of the smart utensil; and
identifying the at least one time series of utensil track data as the input stream of the data.

5. The computer-implemented method of claim 4 , wherein the utensil track data comprise one or more utensil track data items of: a position of the smart utensil, a direction of motion of the smart utensil, an orientation of the smart utensil, a speed of the motion of the smart utensil, an acceleration of the motion of the smart utensil, a force acting on the smart utensil, a mode of handling the smart utensil, and a mode of use of the smart utensil.

6. The computer-implemented method of claim 5 , wherein identifying, based on the input stream of data, the at least one time series of utensil actions capable of reproducing the visual artwork comprises:
applying a utensil action identification algorithm configured to identify the at least one utensil action in the input stream of data.

7. The computer-implemented method of claim 6 , wherein identifying the at least one utensil action in the input stream comprises:
gathering successive utensil track data in the time series of utensil track data until at least one utensil track data item substantially changes to define the at least one utensil action,
wherein the at least one utensil track data item substantially changes, if a difference or an absolute difference of the at least one utensil track data item sampled at two successive sample points exceeds a first threshold.

8. The computer-implemented method of claim 7 , wherein the at least one utensil track data item is the position of the smart utensil.

9. The computer-implemented method of claim 1 , wherein obtaining the at least one image of the visual artwork comprises applying an image capturing algorithm configured for:
prompting at least one user interaction via a user interface of the system guiding a user of the system to capture the at least one image of the visual artwork;
displaying the at least one captured image of the visual artwork via the user interface comprising a graphical output; and
receiving a confirmation indicating that the at least one image is completed via the user interaction.

10. The computer-implemented method of claim 1 , further comprising:
applying an artistic style algorithm configured to recognize at least one artistic style for the at least one image of the visual artwork.

11. The computer-implemented method of claim 10 , comprising applying an artwork meta data algorithm configured to recognize at least one item of artwork meta data for the at least one image of the visual artwork.

12. The computer-implemented method of claim 11 , comprising storing one or more of:
the action-to-feature mapping;
the at least one artistic feature;
the at least one artistic style of the at least one image of the visual artwork;
the at least one image of the visual artwork;
the artwork meta data; and
the at least one time series of utensil actions,
in an artistic style database.

13. The computer-implemented method of claim 1 , wherein generating the action-to-feature mapping based on the at least one time series of utensil actions and on the at least one image of the visual artwork comprises:
applying at least one computer vision algorithm configured to recognize the at least one artistic feature of the visual artwork or of the at least one image of the visual artwork.

14. A system for generating and running a creative tutorial algorithm for an action-to-feature mapping of a visual artwork, comprising one or more servers configured to perform:
initializing the system to create the visual artwork by obtaining data defining a creative objective and obtaining information about at least one targeted artistic style from a plurality of predetermined artistic styles;
generating, via the creative tutorial algorithm, one or more instructions on how to reproduce the creative objective based on the at least one artistic style;
obtaining an input stream of data recording a creation of the visual artwork from the system;
identifying, based on the input stream of data, at least one time series of utensil actions capable of reproducing the visual artwork;
obtaining at least one image of the visual artwork; and
generating the action-to-feature mapping based on the at least one time series of utensil actions and on the at least one image of the visual artwork, thereby recognizing at least one artistic feature of the visual artwork,
wherein the action-to-feature mapping comprises at least one pair of the at least one artistic feature and at least one utensil action of the at least one time series of utensil actions, wherein the creative tutorial algorithm compares the at least one artistic feature to the creative objective based on the targeted artistic style, and wherein the system comprises at least one smart utensil, and wherein the at least one smart utensil comprises a smart pen, a smart wrist device for recording the creation of the visual artwork, a knife for creating the visual artwork, a spatula, a palette knife f

Assignees

• SOCIéTé BIC

Inventors

• Duffy David
• Elliott-Bowman Bernadette

Classifications

• G06T11/10 Primary

  Texturing; Colouring; Generation of textures or colours (retouching, inpainting or scratch removal G06T5/77) · CPC title

• G06F3/0346

  with detection of the device orientation or free movement in a three-dimensional [3D] space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors · CPC title

• G06F3/016

  Input arrangements with force or tactile feedback as computer generated output to the user · CPC title

• G06V10/70

  using pattern recognition or machine learning (optical pattern recognition or electronic computations therefor G06V10/88) · CPC title

• G09B11/10

  Teaching painting · CPC title