Transforming data to create layouts

Patent metadata

Field	Value
Publication number	US-9595298-B2
Application number	US-201213552416-A
Country	US
Kind code	B2
Filing date	Jul 18, 2012
Priority date	Jul 18, 2012
Publication date	Mar 14, 2017
Grant date	Mar 14, 2017

Abstract

Concepts and technologies are described herein for transforming data to create layouts. In accordance with the concepts and technologies disclosed herein, a computing device can execute a layout generator. The computing device can be configured to receive or retrieve data that includes visual content such as an image and other content such as text. The computing device can analyze the visual content to identify a salient region and/or an invariant region within the visual content. The computing device can access designs for laying out visual content and other content based upon an identified salient region and/or invariant region. The computing device can evaluate the designs to determine if a design matches the data. The computing device can load the data into the selected design to generate the layout.

First claim

We claim:

1. A computer-implemented method for transforming data to generate a layout, the computer-implemented method comprising performing computer-implemented operations for:
executing a layout generator at a computing device to obtain the data, the data comprising an image and at least one of text or another element of content;
analyzing the image, using the layout generator, to identify:
a salient region of the image, the salient region comprising a subject of the image; and
an invariant region of the image, at least one of color variation, luminance variation, sharpness variation, blur variation, or pattern variation of the invariant region being below a threshold to achieve a contrast between the text and the image;

comparing the data to a plurality of designs;
selecting a design of the plurality of designs to generate the layout, the design comprising a viewport frame, a salient region placement frame, and a text placement frame, and the selecting being based at least in part on:
determining that the salient region corresponds to the salient region placement frame; and
determining that the invariant region corresponds to the text placement frame; and

generating the layout based at least in part on:
placing the image such that at least a first portion of the salient region is located in a first position corresponding to the salient region placement frame; and
placing at least one of the text or other element of content such that at least a second portion of the at least one of the text or other element of content is located in a second position corresponding to the text placement frame.

2. The method of claim 1 , wherein determining that the salient region corresponds to the salient region placement frame comprises determining that the image can be moved or cropped such that at least the first portion of the salient region fits within the salient region placement frame.

3. The method of claim 1 , wherein selecting the design further comprises:
determining a score representative of a degree to which each design of the plurality of designs relates to the data;
ranking, based at least in part on the score, the plurality of designs; and
determining that the design is a highest ranked design of the plurality of designs.

4. The method of claim 1 , wherein the viewport frame defines viewable limits of the layout.

5. The method of claim 1 , wherein analyzing the image to identify the salient region is based at least in part on analyzing color differences at or within the salient region.

6. The method of claim 1 , wherein analyzing the image to identify of the salient region is based at least in part on performing facial recognition processing on the image.

7. The method of claim 1 , wherein analyzing the image to identify the salient region is based at least in part on analyzing blurring or sharpness at or within the salient region.

8. The method of claim 1 , further comprising, prior to determining that the salient region corresponds to the salient region placement frame, performing at least one manipulation to the image.

9. The method of claim 8 , wherein the at least one manipulation comprises zooming, panning, scrolling, cropping, extending, infilling, or seam carving of the image.

10. A computer storage medium having computer readable instructions stored thereupon that, when executed by a computer, cause the computer to:
execute a layout generator to obtain data comprising visual content and text;
analyze the image to identify:
a salient region of the image, the salient region comprising a subject of the image; and
an invariant region of the image, at least one of color variation, luminance variation, sharpness variation, blur variation, or pattern variation of the invariant region being below a threshold to achieve a contrast between the text and the image;

compare the data to a plurality of designs to provide a comparison of the data;
select, based at least partially upon the comparison of the data, a design of the plurality of designs, the design defining a layout and including a viewport frame, a salient region placement frame, and a text placement frame;
adjust a position of the image relative to the viewport frame to locate the salient region at a first location within the layout corresponding to the salient region placement frame;
overlay the text at a location within the layout that corresponds to a second location of the text placement frame within the design; and
generate the layout, the layout comprising at least a first portion of the image and at least a second portion of the text.

11. The computer storage medium of claim 10 , wherein the computer executable instructions further cause the computer to:
compare the salient region to first locations respectively corresponding to salient region placement frames in individual designs of the plurality of designs;
compare the invariant region to second locations respectively corresponding to text placement frames in the individual designs of the plurality of designs;
determine that the first location matches the salient region;
determine that the second location overlaps the invariant region; and
select the design based at least in part on determining that the first location matches the salient region and that the second location overlaps the invariant region.

12. The computer storage medium of claim 10 , wherein the computer executable instructions further cause the computer to:
prior to selecting the design, resize the image to generate a scaled version of the image; and
select the design based at least in part on the scaled version of the image causing the salient region to correspond to the salient region placement frame and the invariant region to correspond to the text placement frame.

13. The computer storage medium of claim 10 , wherein analyzing the image to identify the salient region is based at least in part on analyzing color distributions at or within the salient region.

14. The computer storage medium of claim 10 , wherein analyzing the image to identify the salient region is based at least in part on performing object recognition.

15. The method of claim 10 , wherein the invariant region excludes the salient region.

16. An apparatus comprising:
a processor;
a memory communicatively coupled to the processor; and
an application program which executes in the processor from the memory and which, when executed by the processor, causes the processor to:
execute a layout generator to obtain data comprising visual content and text;
analyze the image to identify a salient region of the image and an invariant region of the image, the salient region comprising a subject of the image and the invariant portion comprising a first portion of the image at which at least one of color variation, luminance variation, sharpness variation, blur variation, or pattern variation is below a threshold;
compare the data to a plurality of designs to provide a comparison of the data;
select, based at least partially upon the comparison of the data, a design defining a layout, the design comprising a viewport frame defining visible limits of the layout, a salient region placement frame corresponding to a first location at which the salient region of the image is to be placed, and a text placement frame corresponding to a second location at which the text is to be placed within the layout;
load the data into the design based, at least partially, upon the salient region, the invariant region, the salient region placement frame, and the text placement frame; and
generate the layout, the layout compris

Assignees

• Lee David Benjamin
• Freier Nathan George
• Tumanov Ilya
• Microsoft Technology Licensing Llc

Inventors

• Lee David Benjamin
• Freier Nathan George
• Tumanov Ilya

Classifications

• G11B27/28 Primary

  by using information signals recorded by the same method as the main recording {(G11B27/22 takes precedence)} · CPC title

• G06F3/04845

  for image manipulation, e.g. dragging, rotation, expansion or change of colour · CPC title

• G06F40/106 Primary

  Display of layout of documents; Previewing · CPC title

• G11B27/34 Primary

  Indicating arrangements  {(indicating means incorporated in magazine or cassette G11B23/046 and G11B23/0875; indicating measured values in general G01D)} · CPC title

• G06T11/60

  Creating or editing images; Combining images with text · CPC title