Transforming data to create layouts

Patent metadata

Field	Value
Publication number	US-10896284-B2
Application number	US-201816017982-A
Country	US
Kind code	B2
Filing date	Jun 25, 2018
Priority date	Jul 18, 2012
Publication date	Jan 19, 2021
Grant date	Jan 19, 2021

Abstract

Concepts and technologies are described herein for transforming data to create layouts. In accordance with the concepts and technologies disclosed herein, a computing device can execute a layout generator. The computing device can be configured to receive or retrieve data that includes visual content such as an image and other content such as text. The computing device can analyze the visual content to identify a salient region and/or an invariant region within the visual content. The computing device can access designs for laying out visual content and other content based upon an identified salient region and/or invariant region. The computing device can evaluate the designs to determine if a design matches the data. The computing device can load the data into the selected design to generate the layout.

First claim

We claim:

1. A computer-implemented method for transforming data to generate a layout, the computer-implemented method comprising:
obtaining data comprising an image and a content item;
analyzing the image to identify at least one selected from a group consisting of
a salient region of the image, the salient region comprising a subject of the image, and
an invariant region of the image, the invariant region comprising a region of the image wherein a variation of at least one feature is below a threshold to achieve a contrast between the content item and the image;
automatically generating a design to generate the layout based at least partially on at least one rule, the design comprising a viewport frame and at least one of a salient region placement frame and a content placement frame, the at least one rule specifying at least one selected from a group consisting of an aspect ratio of the viewport frame, a size range of the viewport frame, a location of the salient region placement frame, a size of the salient region placement frame, a location of the content placement frame, and a size of the content placement frame;
generating the layout based on the design;
prompting a user for feedback on the generated design;
receiving feedback from the user for the generated design; and
generating a new design to generate a new layout based at least partially on the feedback,
wherein prompting the user for the feedback on the generated design includes prompting the user to answer a series of binary questions related to at least one selected from a group consisting of the layout, text placement, image position, element location, and image size of the generated design.

2. The computer-implemented method of claim 1 , wherein generating the layout based on the design includes performing at least on selected from a group consisting of
adjusting a position of the image within the viewport frame such that the salient region of the image is located within the viewport frame at a position corresponding to the salient region placement frame, and
adjusting a position of the image within the viewport frame such that the invariant region of the image is located within the viewport frame at a position corresponding to the content placement frame.

3. The computer-implemented method of claim 1 , further comprising comparing the data to a plurality of designs to provide a comparison of the data and wherein automatically generating the design includes automatically generating the design in response to failing to select one of the plurality of designs based at least partially on the comparison of the data.

4. The computer-implemented method of claim 3 , further comprising, in response to failing to select one of the plurality of designs based at least partially on the comparison of the data, performing at least one manipulation to the image.

5. The computer-implemented method of claim 4 , wherein performing the at least one manipulation to the image includes performing at least one selected from a group consisting of zooming the image, panning the image, scrolling the image, cropping the image, extending the image, infilling the image, and seam carving the image.

6. The computer-implemented method of claim 1 , wherein analyzing the image to identify the salient region is based at least partially on at least one selected from a group consisting of
analyzing color differences at or within the salient region,
performing facial recognition processing on the image, and
analyzing blurring or sharpness at or within the salient region.

7. A computer storage medium having computer readable instructions stored thereupon that, when executed by a computer, cause the computer to:
obtain data comprising visual content and a content item;
analyze the visual content to identify at least one selected from a group consisting of
a salient region of the visual content, the salient region comprising a subject of the visual content, and
an invariant region of the visual content, the invariant region comprising a region of the visual content wherein a variation of at least one feature is below a threshold to achieve a contrast between the content item and the visual content;

automatically generate a design to generate a layout based at least partially on at least one rule, the design comprising a viewport frame and at least one of a salient region placement frame and a content placement frame, the at least one rule specifying at least one selected from a group consisting of an aspect ratio of the viewport frame, a size of the viewport frame, a location of the salient region placement frame, a size of the salient region placement frame, a location of the content placement frame, and a size of the content placement frame;
generate the layout based on the design;
prompt a user for feedback on the generated design;
receive feedback from the user for the generated design; and
generate a new design to generate a new layout based at least partially on the feedback,
wherein prompting the user for the feedback on the generated design includes prompting the user to provide a ranking for at least one selected from a group consisting of the layout, text placement, image position, element location, and image size of the generated design.

8. The computer storage medium of claim 7 , wherein generating the layout based on the design includes performing at least on selected from a group consisting of
adjusting a position of the visual content within the viewport frame such that the salient region of the visual content is located within the viewport frame at a position corresponding to the salient region placement frame, and
adjusting a position of the visual content within the viewport frame such that the invariant region of the visual content is located within the viewport frame at a position corresponding to the content placement frame.

9. The computer storage medium of claim 7 , wherein the computer readable instructions further cause the computer to compare the data to a plurality of designs to provide a comparison of the data and wherein automatically generating the design includes automatically generating the design in response to failing to select one of the plurality of designs based at least partially on the comparison of the data.

10. The computer storage medium of claim 9 , wherein the computer readable instructions further cause the computer to, in response to failing to select one of the plurality of designs based at least partially on the comparison of the data, performing at least one manipulation to the visual content.

11. The computer storage medium of claim 9 , wherein the at least one manipulation includes at least one selected from a group consisting of zooming the visual content, panning the visual content, scrolling the visual content, cropping the visual content, extending the visual content, infilling the visual content, and seam carving the visual content.

12. The computer storage medium of claim 7 , wherein analyzing the visual content to identify the salient region is based at least partially on at least one selected from a group consisting of
analyzing color differences at or within the salient region,
performing facial recognition processing on the visual content, and
analyzing blurring or sharpness at or within the salient region.

13. An apparatus comprising:
a processor;
a memory communicatively coupled to the processor; and
an application program which executes in the processor from the memory and which, when executed by the processor, causes the processor to:
obtain data comprising visual content and a content item;
analyze the visual content to ident

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Lee David Benjamin
• Freier Nathan George
• Tumanov Ilya

Classifications

• G11B27/34 Primary

  Indicating arrangements  {(indicating means incorporated in magazine or cassette G11B23/046 and G11B23/0875; indicating measured values in general G01D)} · CPC title

• G11B27/28

  by using information signals recorded by the same method as the main recording {(G11B27/22 takes precedence)} · CPC title

• G06T11/60

  Creating or editing images; Combining images with text · CPC title

• G06F40/106 Primary

  Display of layout of documents; Previewing · CPC title

• G06F3/04845

  for image manipulation, e.g. dragging, rotation, expansion or change of colour · CPC title