Systems and methods for identifying a design template matching a search query

Patent metadata

Field	Value
Publication number	US-12299028-B2
Application number	US-202418600121-A
Country	US
Kind code	B2
Filing date	Mar 8, 2024
Priority date	Mar 16, 2023
Publication date	May 13, 2025
Grant date	May 13, 2025

Abstract

Method and system for identifying design templates that match an input query are disclosed. The method includes: receiving a design search query; performing a template search based on the design search query, the template search returning a first template design, the first template design including a target image; processing the design search query to generate an image search query; performing an image search based on the image search query, the image search returning a candidate image; and generating a new design. The new design is based on the first template design and includes the candidate image instead of the target image.

First claim

The invention claimed is:

1. A computer-implemented method including:
receiving a design search query;
performing a template search based on the design search query, the template search returning a first design template, the first design template including a target image;
processing the design search query to generate an image search query, wherein the image search query includes one or more visual tokens based on the design search query and excludes non-visual tokens in the design search query, wherein generating the image search query includes providing the design search query to a machine learning model that is trained to ignore the non-visual tokens in the design search query and output the image search query including the one or more visual tokens;
performing an image search based on the image search query, the image search returning a candidate image;
generating a new design, wherein:
the new design is based on the first design template; and
the new design includes the candidate image instead of the target image.

2. The computer-implemented method of claim 1 , wherein prior to performing the image search, the method includes:
determining that the image search is not required based on the image search query and the first design template; and
in response to determining that the image search is not required:
foregoing generating the new design; and
returning the first design template.

3. The computer-implemented method of claim 2 , wherein determining that the image search is not required includes:
determining that one or more terms of the image search query appear in metadata associated with the first design template.

4. The computer-implemented method of claim 2 , wherein determining that the image search is not required includes:
determining that a relevance score of the first design template exceeds a threshold relevance score.

5. The computer-implemented method of claim 1 , wherein prior to generating the new design, the method includes:
determining that generation of a new design is not required based on the candidate image and the first design template; and
in response to determining that generation of the new design is not required:
foregoing generating the new design; and
returning the first design template.

6. The computer-implemented method of claim 5 , wherein determining that generation of a new design is not required includes:
determining that a confidence score of the candidate image is below a threshold confidence score.

7. The computer-implemented method of claim 5 , wherein determining that generation of a new design is not required includes:
determining that the candidate image has been used in a design less than a threshold number of times.

8. The computer-implemented method of claim 5 , wherein determining that generation of a new design is not required includes:
determining that one or more structural attributes of the target image do not match corresponding one or more structural attributes of the candidate image.

9. A computer-implemented method for identifying one or more design templates matching a design search query, including:
receiving the design search query, the design search query provided by a user;
performing a template search based on the design search query, the template search returning a set of design templates, each of the set of design templates including a target image;
processing the design search query to generate an image search query, wherein the image search query includes one or more visual tokens based on the design search query and excludes non-visual tokens in the design search query, wherein generating the image search query includes providing the design search query to a machine learning model that is trained to ignore the non-visual tokens in the design search query and output the image search query including the one or more visual tokens;
performing an image search based on the image search query, the image search returning a set of candidate images;
generating a set of design templates, the set of design templates including one or more new design templates, the one or more new design templates being based on one or more design templates from the set of design templates and including one or more candidate images from the set of candidate images instead of their corresponding target images.

10. The computer-implemented method of claim 9 , wherein prior to performing the image search, the method includes:
determining that the image search is not required based on determining that metadata of a threshold number of the design templates in the set of design templates includes one or more keywords of the image search query.

11. The computer-implemented method of claim 9 , wherein prior to performing the image search, the method includes:
determining that the image search is not required based on determining that a relevance score of a threshold number of the design templates in the set of design templates exceeds a threshold relevance score.

12. The computer-implemented method of claim 9 , wherein prior to generating the one or more new design templates, the method includes:
for each design template in the set of design templates, determining whether generation of one or more new design templates is required based on the set of candidate images and the design template; and
in response to determining that generation of the new design is not required:
foregoing generating the new design; and
returning the design template.

13. The computer-implemented method of claim 9 , further including:
determining a confidence score for each of the candidate images in the set of candidate images; and
ranking the candidate images in the set of candidate images based on the determined confidence scores.

14. The computer-implemented method of claim 9 , wherein generating the set of design templates includes:
identifying at least a subset of design templates from the set of design templates as candidates for new design templates;
for each candidate design template in the subset of design templates:
selecting a candidate image from the set of candidate images; and
replacing a target image with the selected candidate image.

15. The computer-implemented method of claim 14 , wherein identifying the subset of design templates includes at least one of:
determining that one or more terms of the image search query do not appear in metadata associated with the subset of design templates; or
determining that relevance scores of the subset of design templates is below a threshold score.

16. The computer-implemented method of claim 15 , wherein selecting the candidate image from the set of candidate images is based on matching one or more structural attributes of the target image with a candidate image in the set of candidate images.

17. A non-transitory machine readable medium storing instructions executable by a processing unit to cause the processing unit to:
receive a design search query;
perform a template search based on the design search query, the template search returning a first design template, the first design template including a target image;
process the design search query to generate an image search query, wherein the image search query includes one or more visual tokens based on the design search query and excludes non-visual tokens in the design search query, wherein generating the image search query includes providing the design search query to a machine learning model that is trained to ignore the non-visual tokens in the design search query and output the image sear

Assignees

• Canva Pty Ltd

Inventors

• Lacko Jarrah
• Wu Danny
• Radford William
• Yu Jin

Classifications

• G06F16/24578

  using ranking · CPC title

• G06F16/532 Primary

  Query formulation, e.g. graphical querying · CPC title

• G06F16/58

  Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually · CPC title

• G06F40/174

  Form filling; Merging · CPC title

• G06F40/103

  Formatting, i.e. changing of presentation of documents (automatic justification G06F40/189; automatic line break hyphenation G06F40/191) · CPC title