Stroke operation prediction for three-dimensional digital content

Patent metadata

Field	Value
Publication number	US-10627910-B2
Application number	US-201715438276-A
Country	US
Kind code	B2
Filing date	Feb 21, 2017
Priority date	Feb 21, 2017
Publication date	Apr 21, 2020
Grant date	Apr 21, 2020

Abstract

Stroke operation prediction techniques and systems for three-dimensional digital content are described. In one example, stroke operation data is received that describes a stroke operation input via a user interface as part of the three-dimensional digital content. A cycle is generated that defines a closed path within the three-dimensional digital content based on the input stroke operation and at least one other stroke operation in the user interface. A surface is constructed based on the generated cycle. A predicted stroke operation is generated based at least in part on the constructed surface. The predicted stroke operation is then output in real time in the user interface as part of the three-dimensional digital content as the stroke operation data is received.

First claim

What is claimed is:

1. In a digital medium environment to generate three-dimensional digital content, a method implemented by at least one computing device configured to be worn by a user, the method comprising:
receiving, by the at least one computing device, stroke operation data describing an input stroke operation received via a user interface with respect to the three-dimensional digital content, the input stroke operation detected by a sensor of the at least one computing device as a hand gesture of the user, and the three-dimensional digital content defining at least a portion of an augmented or virtual reality environment;
generating, by the at least one computing device, a cycle that defines a closed path within the three-dimensional digital content based on the input stroke operation and at least one other stroke operation in the user interface;
constructing, by the at least one computing device, a surface based on the generated cycle;
generating, by the at least one computing device, a predicted stroke operation based at least in part on the constructed surface; and
outputting, by an output device associated with the at least one computing device, the predicted stroke operation in real time in the user interface as part of the three-dimensional digital content as the stroke operation data is received, the predicted stroke operation being user selectable within a set amount of time to cause the predicted stroke operation to be included as part of the three-dimensional digital content.

2. The method as described in claim 1 , wherein:
the stroke operation data includes a plurality of operation data samples; and
the generating of the cycle and the constructing of the surface are performed in real time as the plurality of operation data samples are received.

3. The method as described in claim 2 , wherein each of the plurality of operation data samples includes a spatial parameter, an appearance parameter, and a temporal parameter.

4. The method as described in claim 1 , wherein the generating of the cycle is based on the input stroke operation and at least one constraint used to select the at least one other stroke operation in the user interface from a plurality of other stroke operations in the user interface.

5. The method as described in claim 1 , wherein the generating of the cycle is based on the input stroke operation and at least one constraint used to identify the at least one other stroke operation in the user interface, the at least one constraint defining a threshold spatial or temporal distance between the input stroke operation and the at least one other stroke operation within the three-dimensional digital content.

6. The method as described in claim 1 , wherein the generating of the cycle is based on the input stroke operation and at least one constraint used to identify the at least one other stroke operation in the user interface, the at least one constraint defining a direction between the input stroke operation and the at least one other stroke operation within the three-dimensional digital content.

7. The method as described in claim 6 , wherein the direction is defined through a relationship of a surface normal of the input stroke operation and a surface normal of the at least one other stroke operation.

8. The method as described in claim 1 , wherein the generating of the cycle is based on the input stroke operation and at least one constraint used to identify the at least one other stroke operation in the user interface, the at least one constraint defining a minimum amount of the input stroke operation to be used as part of the generated cycle as defined between the input stroke operation and the at least one other stroke operation within the three-dimensional digital content.

9. The method as described in claim 1 , wherein the generating of the cycle is based on the input stroke operation and at least one constraint used to identify the at least one other stroke operation in the user interface, the at least one constraint specifying that the at least one other stroke operation does not loop back to a starting point at which the at least one other stroke operation intersects the input stroke operation.

10. The method as described in claim 1 , wherein the generating of the cycle includes constructing the at least one other stroke operation automatically and without user intervention to form the cycle, the at least one other stroke operation not included in the user interface before the constructing of the at least one other stroke operation.

11. The method as described in claim 1 , wherein the generating of the cycle includes generating a plurality of candidate cycles based on the input stroke operation and selecting the cycle from the plurality of candidate cycles.

12. The method as described in claim 1 , wherein the constructing of the surface includes generating mesh vertices of the surface based on the generated cycle using bilinear interpolation.

13. The method as described in claim 1 , wherein the generating of the predicted stroke operation includes:
projecting the stroke operation data into a two-dimensional stroke operation projection based on the constructed surface and depth information defining a distance of the two-dimensional stroke operation projection to the constructed surface;
calculating a two-dimensional stroke operation prediction based on the two-dimensional stroke operation projection; and
projecting the two-dimensional stroke operation projection into three dimensions to generate the predicted stroke operation based on the distance to the constructed surface.

14. The method as described in claim 1 , wherein the generating of the predicted stroke operation is based on high-level structure exhibited by existing stroke operations in the user interface.

15. In a digital medium environment to generate three-dimensional digital content, a system comprising:
a cycle generation module implemented at least partially in hardware of a computing device configured to be worn by a user to generate a cycle that defines a closed path within the three-dimensional digital content based on an input stroke operation, at least one other stroke operation in a user interface, and at least one constraint, the input stroke operation detected by a sensor of the computing device as a gesture of the user that defines a line in three dimensions in the user interface, and the three-dimensional digital content defining at least a portion of an augmented or virtual reality environment;
a surface construction module implemented at least partially in the hardware of the computing device to construct a surface based on the generated cycle; and
a stroke prediction module implemented at least partially in the hardware of the computing device to generate a predicted stroke operation based at least in part on the constructed surface, the predicted stroke operation being user selectable to cause the predicted stroke operation to be included as part of the three-dimensional digital content.

16. The system as described in claim 15 , wherein the at least one constraint defines:
a threshold temporal or spatial distance between the input stroke operation and the at least one other stroke operation within the three-dimensional digital content;
a direction between the input stroke operation and the at least one other stroke operation within the three-dimensional digital content; or
a minimum amount of the input stroke operation to be used as part of the generated cycle as defined between the input stroke operation and the at least one other stroke operation within the three-dimensiona

Assignees

• Adobe Inc

Inventors

• Lu Jingwan
• Diverdi Stephen J
• Kim Byungmoon
• Xing Jun

Classifications

• G06F3/011 Primary

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title

• G06T17/30

  Polynomial surface description · CPC title

• G06F3/0488

  using a touch-screen or digitiser, e.g. input of commands through traced gestures · CPC title

• G06F3/017 Primary

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title

• G06T19/00 Primary

  Manipulating three-dimensional [3D] models or images for computer graphics · CPC title