Providing drawing assistance using feature detection and semantic labeling

Patent metadata

Field	Value
Publication number	US-10235897-B2
Application number	US-201615294123-A
Country	US
Kind code	B2
Filing date	Oct 14, 2016
Priority date	Feb 3, 2014
Publication date	Mar 19, 2019
Grant date	Mar 19, 2019

Abstract

Methods for providing drawing assistance to a user sketching an image include geometrically correcting adjusting user strokes to improve their placement and appearance. In particular, one or more guidance maps indicate where the user “should” draw lines. As a user draws a stroke, the stroke is geometrically corrected by moving the stroke toward a portion of the guidance maps corresponding to the feature of the image the user is intending to draw based feature labels. To further improve the user drawn lines, parametric adjustments are optionally made to the geometrically-corrected stroke to emphasize “correctly” drawn lines and de-emphasize “incorrectly” drawn lines.

First claim

We claim:

1. A non-transitory computer readable medium storing instructions thereon for providing drawing assistance to users sketching designs that, when executed by at least one processor, cause a computer system to:
generate a guidance map for a design;
provide the design on via a touch screen;
detect a stroke drawn by a user on the touch screen over the design;
identify a portion of the guidance map corresponding to the stroke;
generate a geometrically-corrected stroke by moving the stroke drawn by the user toward a position of the identified portion of the guidance map; and
render the geometrically-corrected stroke on the touch screen in place of the stroke drawn by the user.

2. The non-transitory computer readable medium as recited in claim 1 , wherein the instructions, when executed by the at least one processor, cause the computer system to generate the guidance map by:
applying a feature detection algorithm to the design to identify one or more features in the design; and
associating feature labels with the identified one or more features in the guidance map.

3. The non-transitory computer readable medium as recited in claim 2 , wherein the instructions, when executed by the at least one processor, cause the computer system to identify the portion of the guidance map corresponding to the stroke by identifying the portion of the guidance map near to the stroke that has an associated feature label that corresponds to the stroke.

4. The non-transitory computer readable medium as recited in claim 3 , wherein the instructions, when executed by the at least one processor, cause the computer system to generate the guidance map by generating an edge guidance map by applying an edge detection filter to the design.

5. The non-transitory computer readable medium as recited in claim 4 , wherein the instructions, when executed by the at least one processor, cause the computer system to generate the guidance map further by generating a feature guidance map by applying a feature detection algorithm to the design to identify one or more features in the design.

6. The non-transitory computer readable medium as recited in claim 5 , further comprising instructions that, when executed by the at least one processor, cause the computer system to use the feature guidance map to supplement the edge guidance map by using feature lines of the feature guidance map for missing portions of the edge guidance map.

7. The non-transitory computer readable medium as recited in claim 6 , wherein the instructions, when executed by the at least one processor, cause the computer system to generate the geometrically-corrected stroke by:
determining a feature line of the feature guidance map near to the stroke that has the associated feature label that corresponds to the stroke;
determining an edge of the edge guidance map near to the stroke that has the associated feature label that corresponds to the stroke;
determining which of the edge or the feature line provides is closer to the stroke; and
moving the stroke toward the determined edge or feature line that is closer to the stroke.

8. The non-transitory computer readable medium as recited in claim 4 , wherein the edge detection filter comprises an extended Difference-of-Gaussians filter.

9. The non-transitory computer readable medium as recited in claim 5 , wherein:
the design comprises an image of a face; and
applying a feature detection algorithm to the image comprises applying a face detection algorithm to the image.

10. The non-transitory computer readable medium as recited in claim 1 , further comprising instructions that, when executed by the at least one processor, cause the computer system to determine deviations of the geometrically-corrected stroke from the identified portion of the guidance map by determining differences in position and curvature between the geometrically-corrected stroke and the identified portion of the guidance map.

11. The non-transitory computer readable medium as recited in claim 10 , further comprising instructions that, when executed by the at least one processor, cause the computer system to perform one or more of:
de-emphasizing portions of the geometrically-corrected stroke that deviate from the identified portion of the guidance map; or
emphasizing portions of the geometrically-corrected stroke that align with the identified portion of the guidance map.

12. The non-transitory computer readable medium as recited in claim 1 , further comprising instructions that, when executed by the at least one processor, cause the computer system to render the geometrically-corrected stroke as the user draws the stroke.

13. The non-transitory computer readable medium as recited in claim 1 , further comprising instructions that, when executed by the at least one processor, cause the computer system to:
determine a variation of the stroke from the identified portion of the guidance map; and
at least partially maintain the variation when geometrically correcting the stroke.

14. A system of providing drawing assistance to users sketching a design comprising:
a computing device comprising a touch screen;
at least one non-transitory computer readable storage medium storing instructions thereon that, when executed by the computing device, cause the system to:
generate a guidance map for an image;
provide the design on via the touch screen;
detecting a stroke drawn by a user on the touch screen over the image;
identify a portion of the guidance map corresponding to the stroke;
generate a geometrically-corrected stroke by moving the stroke drawn by the user toward a position of the identified portion of the guidance map; and
render the geometrically-corrected stroke on the touch screen in place of the stroke drawn by the user.

15. The system as recited in claim 14 , wherein the instructions, when executed by the computing device, cause the system to generate the guidance map for the image by generating one or more of:
a feature guidance map of the image by applying a feature detection algorithm to the image that identifies positions of one or more features of the image; or
an edge guidance map of the image by applying an edge detection filter to the image.

16. The system as recited in claim 15 , wherein the identified portion of the guidance map comprises one or more of:
a line of the feature guidance map near to the stroke that has an associated feature label that matches a feature label of the stroke; or
an edge of the edge guidance map near to the stroke that has an associated feature label that matches a feature label of the stroke.

17. The system as recited in claim 16 , wherein the instructions, when executed by the computing device, further cause the system to:
determine a first distance of the stroke from the line of the feature guidance map;
determine a second distance of the stroke from the edge of the guidance map; and
give weight to the line of the feature guidance map and the edge of the edge guidance map based on the determined first and second distances when generating the geometrically-corrected stroke.

18. A computer-implemented method of providing drawing assistance to users sketching designs comprising:
detecting one or more features in an image;
identifying input points defining a stroke drawn by the user on a touch screen;
identifying guidance points of a guidance map near to the input points of the stroke;
determining geometrically-corrected points for the input points based on the identified guidance points, the geometrically-corrected points m

Assignees

• Adobe Inc

Inventors

• Winnemoeller Holger
• Xie Jun
• Li Wilmot Wei-Mau
• Hertzmann Aaron Phillip

Classifications

• G06T11/23

  using straight lines or curves · CPC title

• G06T11/10 Primary

  Texturing; Colouring; Generation of textures or colours (retouching, inpainting or scratch removal G06T5/77) · CPC title

• G06T7/13

  Edge detection · CPC title

• G06T11/203

  Physics · mapped topic

• G09B5/02

  with visual presentation of the material to be studied, e.g. using film strip · CPC title