Identifying image comments from similar images

Patent metadata

Field	Value
Publication number	US-10732783-B2
Application number	US-201514981816-A
Country	US
Kind code	B2
Filing date	Dec 28, 2015
Priority date	Dec 28, 2015
Publication date	Aug 4, 2020
Grant date	Aug 4, 2020

Abstract

An image chat application generates comments to images based on features of the images. In one example, the image chat application searches through a repository of stored image-comment pairs to identify a stored image that is similar to the image, and generates a comment to the image based on an identified stored image-comment pair. In another example, the image chat application may identify and tag particular objects that dominate an image, and may generate a comment to the image based on characteristics of those particular objects. In this second example, the image chat application further generates a comment to the image based on comments previously associated with the identified tag.

First claim

What is claimed is:

1. A system comprising:
one or more processors;
a computer-readable medium coupled to the one or more processors, including one or more modules that are executable by the one or more processors to:
receive data indicating an image;
extract, by the one or more processors, a first set of feature representations from a first object of the image and a second set of feature representations from a second object of the image;
determine a similarity between the image and individual stored images of a dataset of stored images, based at least in part on the first set of feature representations of the image and the second set of feature representations of the image;
identify a particular stored image from the dataset of stored images, based at least in part on the similarity;
identify a first comment, based at least in part on the particular stored image and the first set of feature representations of the image;
identify a second comment, based at least in part on the particular stored image and the second set of feature representations of the image;
generate, autonomously in response to identifying the first comment and the second comment, a comment by combining a portion of the first comment and a portion of the second comment; and
cause a display of the comment on a user interface of a client device, the comment being displayed as a response to a display of the image.

2. The system of claim 1 , wherein the first set of feature representations from the image corresponds to the first object being a first dominant object of the image and the second set of feature representations from the image corresponds to the second object being a second dominant object of the image.

3. The system of claim 1 , wherein the client device is associated with a user or a user group, and wherein the individual stored images of the dataset of stored images include at least one tag that identifies a dominant object of the individual stored images, the at least one tag being based at least in part on particular tags that are commonly associated with the user or the user group.

4. The system of claim 1 , wherein the one or more modules are further executable by the one or more processors to:
determine that the image is displayed on a web-based social media platform, based at least in part on the data indicating the image; and
retrieve the image from the web-based social media platform.

5. A system comprising:
one or more processors;
a computer-readable medium coupled to the one or more processors, including one or more modules that are executable by the one or more processors to:
receive data indicating an image;
extract, by the one or more processors, a dominant object from the image;
compare the dominant object with a dataset of tagged images to select a first tagged image from the dataset of tagged images similar to the image;
identify a first tag that is associated with the first tagged image;
compare the dominant object with additional datasets of tagged images, the additional datasets associated with the identified first tag;
identify, from the comparison of the dominant object with the additional datasets, a second tagged image that is similar to the dominant object from the image;
identify a second tag that is associated with the second tagged image;
generate a comment using the second tag to describe the dominant object; and
cause a display of the comment on a user interface of a client device, the comment being displayed as a response to a display of the image.

6. The system of claim 5 , wherein the one or more modules are executable to use a dataset index to identify at least one association of the additional datasets with the first tag that is associated with the first tagged image.

7. The system of claim 6 , wherein the one or more modules are further executable to use the dataset index to identify refinements of the first tag that is associated with the first tagged image.

8. One or more computer-readable storage media including one or more modules that are executable by one or more processors to cause a computing system to:
receive data indicating an image;
determine a similarity between the image and individual stored images of a dataset of stored images, based at least in part on a first set of feature representations of the image including a first object and a second set of feature representations of the image including a second object;
identify a particular stored image from the dataset of stored images, based at least in part on the similarity;
identify a first particular comment from a plurality of comments associated with the particular stored image and the first set of feature representations of the image;
identify a second particular comment from the plurality of comments, based at least in part on the particular stored image and the second set of feature representations of the image;
generate, autonomously in response to the identifications of the first particular comment and the second particular comment, a particular comment by combining portions of the first particular comment and portions of the second particular comment; and
cause a display of the particular comment on a user interface of a client device, the particular comment being displayed as a response to a display of the image.

9. The one or more computer-readable storage media of claim 8 , wherein the plurality of comments is based at least in part on one or more of:
comments previously posted by users on a social media platform; or
stock comments.

10. The one or more computer-readable storage media of claim 8 , wherein the one or more modules are further executable by the one or more processors to select the first particular comment randomly from the plurality of comments.

11. The one or more computer-readable storage media of claim 8 , wherein the one or more modules are further executable by the one or more processors to identify a tag associated with the particular stored image, and the first particular comment is identified from the plurality of comments based at least in part on the tag.

12. A computer-implemented method, comprising:
receiving, via a client device, data indicating an image;
extracting, from the image, a first set of feature representations having a first feature vector associated with a first object of the image and a second set of feature representations having a second feature vector associated with a second object of the image;
determining a similarity between the image and individual stored images of a dataset of stored images, based at least in part on the first feature vector and the second feature vector;
identifying a particular stored image from the dataset of stored images, based at least in part on the similarity;
identifying a plurality of comments based at least in part on the particular stored image;
identifying a first comment from the plurality of comments, based at least in part on the particular stored image and the first feature vector;
identifying a second comment from the plurality of comments, based at least in part on the particular stored image and the second feature vector;
generating, autonomously in response to identifying the first comment and the second comment, a comment by combining a portion of the first comment and a portion of the second comment; and
causing a display of the comment on a user interface of the client device as a response to a display of the image.

13. The computer-implemented method of claim 12 , wherein the plurality of comments is associated with the individual stored images or is associated with one or more tags, the one or more tags being associated with

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Yang Kuiyuan
• Ma Wei-Ying
• Liu Chunyan
• Li Di
• Xu Pengfei
• Rui Yong
• Wang Baoxun

Classifications

• H04L51/04 Primary

  Real-time or near real-time messaging, e.g. instant messaging [IM] · CPC title

• H04L51/52

  for supporting social networking services · CPC title

• G06V20/20

  in augmented reality scenes · CPC title

• G06F16/58

  Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually · CPC title

• H04L51/10

  Multimedia information · CPC title