Intelligent image search results summarization and browsing

Patent metadata

Field	Value
Publication number	US-10521692-B2
Application number	US-201414324375-A
Country	US
Kind code	B2
Filing date	Jul 7, 2014
Priority date	Feb 8, 2010
Publication date	Dec 31, 2019
Grant date	Dec 31, 2019

Abstract

Techniques for intelligent image search results summarization and browsing scheme are described. Images having visual attributes are evaluated for similarities based in part on their visual attributes. At least one preference score indicating a probability of an image to be selected into a summary is calculated for each image. Images are selected based on the similarity of the selected images to the other images and the preference scores of the selected images. A summary of the plurality of images is generated including the selected one individual image.

First claim

What is claimed is:

1. A method, comprising:
receiving a plurality of images from one or more image providers in response to receiving a search term from a client, each of the plurality of images having at least one visual attribute;
determining a local scaling parameter for each image in the plurality of images by evaluating similarities between the images in the plurality of images based, at least in part, on the at least one visual attribute of each image in the plurality of images;
selecting a particular image from the plurality of images based, at least in part, on the local scaling parameter associated with the particular image;
comparing the local scaling parameter of the particular image to the local scaling parameter of each image in the plurality of images to determine a probability, for each image in the plurality of images, that each image will be grouped with the particular image, wherein the probability is based, at least in part, on a strength of similarity between the particular image and each individual image of the plurality of images and a determined preference score of each individual image of the plurality of images, the preference score including an image quality of each individual image in the plurality of images and a relevance score of each individual image in the plurality of images;
generating a summary of the plurality of images, the summary including the particular image;
for each image in the plurality of images, adjusting the probability that at least a portion of the image corresponds to the search term and will be grouped with the particular image based, at least in part, on the selecting of the particular image; and
providing the summary of the plurality of images and the plurality of images to the client.

2. The method of claim 1 , wherein the client displays the summary of the plurality of images in a first portion of a display and the plurality of images in a second portion of the display.

3. The method of claim 1 , wherein the relevance score is calculated based, at least in part, on sequence order of the images in the plurality of images.

4. The method of claim 1 , wherein the image quality is calculated based, at least in part, on a visual quality of individual ones of the plurality of images.

5. The method of claim 1 , wherein the at least one visual attribute includes at least one of a color entropy, a brightness, a blur, a block, a dynamic range, an intensity contrast, an image width, or an image height.

6. The method of claim 1 , wherein the at least one visual attribute includes at least one of a color, a shape, or a texture.

7. The method of claim 1 , wherein the client concurrently displays the summary of the plurality of images and the plurality of images.

8. The method of claim 1 , wherein the summary of the plurality of images displays a browsing path associated with similar images of the plurality of images in response to a user selection.

9. The method of claim 1 , wherein grouping of each image and the particular image is associated with a category.

10. A device, comprising:
one or more processors; and
one or more computer-readable media storing instructions that, when executed by the one or more processors, cause the one or more processors to:
receive, in response to execution of a search term, a plurality of images from one or more image providers;
determine a local scaling parameter for each individual image of the plurality of images by evaluating similarities between each individual image of the plurality of images based, at least in part, on at least one visual attribute of each of the individual images of the plurality of images;
select a particular image from the plurality of images based, at least in part, on the local scaling parameter associated with the particular image and a determined quality score that indicates a visual quality of the particular image;
compare the local scaling parameter of the particular image to the local scaling parameter of each image in the plurality of images to determine a probability that each individual image in the plurality of images will be grouped with the particular image, wherein the probability is based, at least in part, on a strength of similarity between the particular image and each individual image of the plurality of images and a determined preference score of each individual image of the plurality of images, the preference score including an image quality of each individual image of the plurality of images and a relevance score of each individual image of the plurality of images;
generate a summary of the plurality of images, including the particular image; and
adjust the probability of each individual image in the plurality of images that each individual image is associated with the search term and will be grouped with the particular image for inclusion in the summary of the plurality of images.

11. The device of claim 10 , wherein the relevance score is based, at least in part, on a sequence associated with the plurality of images and wherein the selecting of the particular image is based, at least in part, on the relevance score.

12. The device of claim 10 , further comprising instructions for displaying the plurality of images concurrently with the particular image.

13. The device of claim 10 , further comprising instructions for providing the summary of the plurality of images and the plurality of images to a client device, wherein the client device displays at least one of the plurality of images concurrently with the summary of the plurality of images.

14. The device of claim 10 , wherein the summary of the plurality of images displays a browsing path associated with similar images of the plurality of images in response to a user selection.

15. A hardware computer readable storage device comprising instructions that, when executed by one or more processors, cause the one or more processors to:
provide a user input to a server;
receive a plurality of images from the server in response to the user input;
determine a plurality of local scaling parameters for individual images of the plurality of images by evaluating similarities between the individual images of the plurality of images using at least one visual attribute of the plurality of images;
select one or more summary images representative of the plurality of images based, at least in part, on the plurality of local scaling parameters of the individual images of the plurality of images;
compare individual ones of the plurality of local scaling parameters to determine a preference score for each image of the plurality of images, the preference score including an image quality score and a relevance score, the preference score indicating a probability, for each image of the plurality of images, that each image will be grouped with the one or more summary images;
adjust the probability of each individual image of the plurality of images that each individual image corresponds to the user input and will be grouped with the one or more summary images; and
concurrently display the one or more summary images and at least one individual image of the plurality of images on a display.

16. The hardware computer readable storage device of claim 15 , wherein the plurality of local scaling parameters of the individual images of the plurality of images includes an image quality score based, at least in part, on a visual quality associated with the individual images of the plurality of images and wherein the selecting of the one or more summary images is based, at least in part, on the image quality score.

17. T

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Wang Jingdong
• Hua Xian-Sheng

Classifications

• G06F2218/12

  Classification; Matching · CPC title

• G06V20/30 Primary

  in albums, collections or shared content, e.g. social network photos or video · CPC title

• G06F16/54

  Browsing; Visualisation therefor · CPC title

• G06T17/05

  Geographic models · CPC title

• G06F16/583

  using metadata automatically derived from the content · CPC title