Grouping and ranking images based on facial recognition data

Patent metadata

Field	Value
Publication number	US-9773156-B2
Application number	US-201414264619-A
Country	US
Kind code	B2
Filing date	Apr 29, 2014
Priority date	Apr 29, 2014
Publication date	Sep 26, 2017
Grant date	Sep 26, 2017

Abstract

Technologies for grouping images, and ranking the images and the groupings, based on entities shown in the images. Images may be grouped based on faces shown in the images. Different images with faces that indicate the same entity (e.g., Adam) may be automatically grouped together. Different images with faces that indicate the same multiple entities (e.g., the people in my family) may also be automatically grouped together. Such automatic grouping may be based on facial recognition technologies. Further, images and groups of images may be automatically ranked based on the faces shown and entities represented. Such rankings may also be influenced by adjacent data that indicates family and friends and the like, and that can be used to identify such entities in the images.

First claim

The invention claimed is:

1. A method performed on a computing device, the method comprising:
determining, by the computing device for each image in a set of images, facial recognition data that, for each face detected by the computing device in the each image, comprises a face identifier that uniquely identifies the each face, a set of facial feature descriptors, a face score that is based on an open/closed state of the each face's eyes and mouth and that indicates an overall quality of the each face, and a face signature that, across the images in the set, uniquely identifies an entity that the each face represents; and
grouping, by the computing device based at least on the face signatures and the face scores, images of the set into one or more groups, where each group comprises one or more images that each show a detected face that represents a same entity as that represented by any other detected face shown in any image in the each group.

2. The method of claim 1 where the grouping is further based on the face scores.

3. The method of claim 1 where a face with a face score below a particular threshold is not included in the grouping.

4. The method of claim 1 where at least one of the groups is a multi-entity group.

5. The method of claim 1 further comprising ranking, based on the face scores, the images in the set or the groups.

6. The method of claim 1 further comprising ranking the groups according to a number of entities of the groups.

7. The method of claim 1 further comprising ranking the groups according to entities of the groups matching adjacent data that indicates family and friends.

8. A system comprising a computing device and at least one program module that together are configured for performed actions comprising:
determining, by the computing device for each image in a set of images, facial recognition data that, for each face detected by the computing device in the each image, comprises a face identifier that uniquely identifies the each face, a set of facial feature descriptors, a face score that is based on an open/closed state of the each face's eyes and mouth and that indicates an overall quality of the face, and a face signature that, across the images in the set, uniquely identifies an entity that the each face represents; and
grouping, by the computing device based at least on the face signatures and the face scores, images of the set into one or more groups, where each group comprises one or more images that each show a detected face that represents a same entity as that represented by any other detected face shown in any image in the each group.

9. The system of claim 8 where the grouping is further based on the face scores.

10. The system of claim 8 where a face with a face score below a particular threshold is not included in the grouping.

11. The system of claim 8 where at least one of the groups is a multi-entity group.

12. The system of claim 8 , the actions further comprising ranking, based on the face scores, the images in the set or the groups.

13. The system of claim 8 , the actions further comprising ranking the groups according to a number of entities of the groups.

14. The system of claim 8 , the actions further comprising ranking the groups according to entities of the groups matching adjacent data that indicates family and friends.

15. At least one computer-readable media storing computer-executable instructions that, when executed by a computing device, cause the computing device to perform actions comprising:
determining, by the computing device for each image in a set of images, facial recognition data that, for each face detected by the computing device in the image, comprises a face identifier that uniquely identifies the each face, a set of facial feature descriptors, a face score that is based on an open/closed state of the each face's eyes and mouth and that indicates an overall quality of the each face, and a face signature that, across the images in the set, uniquely identifies an entity that the each face represents; and
grouping, by the computing device based at least on the face signatures and the face scores, images of the set into one or more groups, where each group comprises one or more images that each show a detected face that represents a same entity as that represented by any other detected face shown in any image in the each group.

16. The at least one computer-readable media of claim 15 where the grouping is further based on the face scores, or where a face with a face score below a particular threshold is not included in the grouping.

17. The at least one computer-readable media of claim 15 where at least one of the groups is a multi-entity group.

18. The at least one computer-readable media of claim 15 , the actions further comprising ranking, based on the face scores, the images in the set or the groups.

19. The at least one computer-readable media of claim 15 , the actions further comprising ranking the groups according to a number of entities of the groups.

20. The at least one computer-readable media of claim 15 , the actions further comprising ranking the groups according to entities of the groups matching adjacent data that indicates family and friends.

Assignees

• Microsoft Corp
• Microsoft Technology Licensing Llc

Inventors

• Chan Chunkit Jacky
• Lee David
• Light Allison

Classifications

• G06F16/583

  using metadata automatically derived from the content · CPC title

• G06V20/30 Primary

  in albums, collections or shared content, e.g. social network photos or video · CPC title

• G06K9/00677

  Physics · mapped topic

• G06K9/00221 Primary

  Physics · mapped topic

• G06F17/30247

  Physics · mapped topic