Dynamic image analysis and cropping

Patent metadata

Field	Value
Publication number	US-11462009-B2
Application number	US-201916421146-A
Country	US
Kind code	B2
Filing date	May 23, 2019
Priority date	Jun 1, 2018
Publication date	Oct 4, 2022
Grant date	Oct 4, 2022

Abstract

The disclosed techniques generally relate to devices and methods for generating cropped images in which one or more features of interest are preserved or emphasized. In one implementation, such features of interest may include facial features. In accordance with certain implementations, location and extent of such features may be identified and used in conjunction with information regarding display characteristics so as to generate cropped images preserving or emphasizing the feature of interest when displayed on a given device.

First claim

What is claimed is:

1. A tangible, non-transitory machine-readable medium, comprising machine-readable instructions that, when executed by one or more processors, cause the one or more processors to:
receive a request from an electronic device, wherein the request is for media content associated with an image;
identify the image based upon the request;
identify a respective device crop code associated with the electronic device;
request a cropped image using an indication of the image and the respective device crop code;
receive the cropped image, wherein the cropped image comprises a version of the image that is specifically cropped for display on the electronic device associated with the respective device crop code; wherein cropping regions of the cropped image are configured to preserve a particular facial feature of a plurality of facial features of one or more faces depicted in the image from being cropped and are determined based upon:
a priority of the particular facial feature over other facial features of the plurality of facial features;
a non-linear edge of a display of the electronic device that will display the cropped image;
an obstructed display region of the display where the image will be at least partially obstructed by display of additional content on the display; or
a combination thereof; and

provide the cropped image to the electronic device.

2. The machine-readable medium of claim 1 , wherein the cropping regions of the cropped image are determined based upon the obstructed display region of the display and wherein the additional content on the display that will at least partially obstruct the image comprises graphical elements that will be displayed concurrently with the cropped image.

3. The machine-readable medium of claim 1 , wherein the cropping regions of the cropped image are determined based upon a shift of the image that facilitates the priority of the particular facial feature.

4. The machine-readable medium of claim 3 , wherein the shift of the image causes depiction of the prioritized particular facial feature of the plurality of facial features that would not be visible without the shift.

5. The machine-readable medium of claim 3 , in response to a plurality of faces depicted in the image, wherein the priority of the particular facial feature results in an overall reduction of the plurality of faces depicted in the image.

6. The machine-readable medium of claim 1 , wherein the request for media content comprises one or more of a request for music content, video content, e-book content or news content having the content associated with the image.

7. A method of cropping images, comprising:
receiving a request from an electronic device, wherein the request is for media content associated with an image;
identifying the image based upon the request;
identifying and requesting a cropped image specific to the electronic device;
executing a first facial recognition routine on the image to identify one or more facial feature boundaries encompassing one or more facial features of a plurality of facial features of one or more faces of the image, wherein the one or more facial feature boundaries prioritizes a facial feature over other facial features of the plurality of facial features;
executing a second facial recognition routine to identify one or more face boundaries that identify the one or more faces of the image, the one or more face boundaries encompassing at least a portion of the facial feature boundaries;
executing a third facial recognition routine to identify a face group boundary that identifies a region around two or more of the one or more faces of the image, the face group boundary encompassing at least a portion of the one or more face boundaries;
generating the cropped image, by:
identifying a cropping region based upon the one or more facial feature boundaries, the one or more face boundaries, and the face group boundary, wherein the cropping region is configured to preserve a particular facial feature of the plurality of facial features of the one or more faces depicted in the image from being cropped; and
cropping the image based upon the cropping region; and

providing the cropped image to the electronic device.

8. The method of claim 7 , wherein the cropped image is cropped for display on the electronic device associated with a respective device crop code.

9. The method of claim 7 , wherein the image comprises one of an artist image, an album cover, a book cover, a news photo, or a stock photo.

10. The method of claim 7 , wherein the particular facial feature comprises an eye, a nose, or a mouth.

11. The method of claim 7 , wherein the face group boundary is used to define the cropping region when multiple faces should be displayed in the cropped image.

12. The method of claim 7 , wherein the one or more face boundaries are used to define the cropping region when at least an entire face should be displayed, wherein the entire face comprises the particular facial feature and additional features.

13. The method of claim 7 , wherein the cropping region is determined based upon an emphasis of the particular facial feature over other facial features of the plurality of facial features when the emphasized particular facial feature should be displayed.

14. The method of claim 13 , wherein the cropping region is based upon:
a non-linear edge of a display of the electronic device that will display the cropped image;
an obstructed display region of the display where the image will be at least partially obstructed by display of additional content on the display; or
a combination thereof.

15. The method of claim 14 , wherein the cropping region of the cropped image is determined based upon the obstructed display region of the display and wherein the additional content on the display that will at least partially obstruct the image comprises graphical elements that will be displayed concurrently with the cropped image.

16. The method of claim 13 , wherein the cropping region of the cropped image is determined based upon a shift of the image that facilitates the emphasis of the particular facial feature of the plurality of facial features, or both.

17. A computer for providing a cropped image to an electronic device, the computer comprising:
a memory; and
a processor, wherein the memory includes instructions executable by the processor to cause the computer to perform operations comprising:
receiving a request from the electronic device, wherein the request is for media content associated with an image;
identifying the image based upon the request;
identifying a respective device crop code associated with the electronic device;
requesting the cropped image using an indication of the image and the respective device crop code;
receiving the cropped image, wherein the cropped image comprises a version of the image that is specifically cropped for display on electronic devices associated with the respective device crop code; wherein cropping regions of the cropped image are configured to preserve a particular facial feature of a plurality of facial features of one or more faces depicted in the image from being cropped and are determined based upon:
a priority of the particular facial feature over other facial features of the plurality of facial features;
a non-linear edge of a display of the electronic device that will display the cropped image;
an obstructed display region of the display where the image will be at least partially obstructed by display of additional content on the display; or
a combination ther

Assignees

• Apple Inc

Inventors

• Agarwal Ashish
• Peng Fei
• Zhang Wenyu
• Thorn Matthew H
• Harold Sean Michael
• Friedman Douglas Isaac
• Alarcón Leticia Marie

Classifications

• G06V40/172

  Classification, e.g. identification · CPC title

• G06V40/171

  Local features and components; Facial parts (eye characteristics G06V40/18); Occluding parts, e.g. glasses; Geometrical relationships · CPC title

• G06V20/20 Primary

  in augmented reality scenes · CPC title

• G06T7/11

  Region-based segmentation · CPC title

• G06T2207/30201

  Face · CPC title