Providing focal length adjustments

Patent metadata

Field	Value
Publication number	US-9786030-B1
Application number	US-201414305449-A
Country	US
Kind code	B1
Filing date	Jun 16, 2014
Priority date	Jun 16, 2014
Publication date	Oct 10, 2017
Grant date	Oct 10, 2017

Abstract

Implementations generally relate to providing focal length adjustments in photos. In some implementations, a method includes determining a face model of a face in a photo. The method further includes simulating a change in the focal length based on the face model.

First claim

What is claimed is:

1. A computer-implemented method comprising:
determining a face model of a face in a photo;
determining a focal length associated with the face based on depth information, wherein the depth information is based on depth cues provided by matching the face with one or more reference images associated with a known user of a social network system to determine that the face in the photo is a known object;
providing a slider to a user via a user interface to adjust the focal length;
receiving user input that indicates a movement of the slider;
simulating a change in the focal length based on the face model and the user input to produce a new photo, wherein simulating the change in the focal length comprises normalizing the face, adjusting a size of a facial feature after the normalizing, and undoing the normalizing after adjusting the size of the facial feature, wherein adjusting the size of the facial feature comprises at least one of:
moving face landmarks corresponding to the facial feature closer to each other to decrease the size of the facial feature corresponding to increase in the focal length; and
moving the face landmarks corresponding to the facial feature away from each other to increase the size of the facial feature corresponding to decrease in the focal length; and

causing the new photo to be displayed in the user interface.

2. The method of claim 1 , wherein the face model comprises the face landmarks, wherein each face landmark is associated with one or more floating point coordinates.

3. The method of claim 2 , further comprising determining head pose information.

4. The method of claim 2 , further comprising wherein the one or more floating point coordinates include spatial coordinates and depth coordinates.

5. The method of claim 1 , wherein the simulating of the change in the focal length further comprises parametrized face warping.

6. The method of claim 5 , further comprising restricting the parametrized face warping to the face and head.

7. The method of claim 1 , wherein the simulating of the change in the focal length further comprises changing a projection of the face.

8. The method of claim 1 , wherein a rate of change of the focal length is one of: directly related to the movement of the slider or bears a non-linear relationship to the movement of the slider.

9. The computer-implemented method of claim 1 , further comprising computing the depth information from at least one of defocus and shading.

10. A computer-implemented method comprising:
determining a face model of a face in a photo;
determining a focal length associated with the face based on depth information, wherein the depth information is based on depth cues provided by matching the face with one or more reference images by extracting features from the face in the photo and comparing the features with features of the one or more reference images to determine that the extracted features match the features of the one or more reference images;
providing a control that is displayed via a user interface, the control effective to adjust the focal length;
receiving user input via the control;
simulating a change in the focal length based on the face model and the user input to produce a new photo, wherein the simulating of the change in the focal length includes:
normalizing the face,
after normalizing the face, performing parametrized face warping, changing a projection of the face, and adjusting a size of a facial feature, and
undoing the normalizing, wherein adjusting the size of the facial feature comprises at least one of:
moving face landmarks corresponding to the facial feature closer to each other to decrease the size of the facial feature corresponding to increase in the focal length; and
moving face landmarks corresponding to the facial feature away from each other to increase the size of the facial feature corresponding to decrease in the focal length; and

causing the new photo to be displayed in the user interface.

11. The method of claim 10 , wherein simulating the change in the focal length further comprises face warping based on interpolation using thin-plate splines.

12. The computer-implemented method of claim 10 , further comprising computing the depth information from at least one of defocus and shading.

13. A system comprising:
one or more processors; and
logic encoded in one or more tangible media for execution by the one or more processors and when executed operable to perform operations comprising:
determining a face model of a face in a photo, wherein the face model includes one or more face landmarks;
computing depth information, wherein the depth information is based on depth cues provided by matching the face with one or more reference images of a known user to determine that the face has features that match features of the one or more reference images;
determining a focal length associated with the face based on the depth information;
simulating a change in the focal length based on the face model to produce a new photo, wherein the simulating includes normalizing the face, changing a projection of the face by moving one or more of the face landmarks to different positions to adjust a size of a facial feature after the normalizing, and undoing the normalizing after changing the projection of the face, wherein the moving comprises at least one of:
moving face landmarks corresponding to the facial feature closer to each other to decrease the size of the facial feature corresponding to increase in the focal length; and
moving the face landmarks corresponding to the facial feature away from each other to increase the size of the facial feature corresponding to decrease in the focal length; and

causing the new photo to be displayed.

14. The system of claim 13 , wherein the logic when executed is further operable to perform operations comprising determining head pose information.

15. The system of claim 13 , wherein, to simulate the change in the focal length, the logic when executed is further operable to perform operations comprising parametrized face warping.

16. The system of claim 13 , wherein the operation of simulating the change further comprises normalizing the face.

17. The system of claim 13 , wherein the logic when executed is further operable to perform operations comprising changing head pose information.

18. The system of claim 13 , wherein simulating the change in the focal length further comprises processing one or more face landmarks by at least one of:
applying an orthogonal matrix;
applying a user-defined pose matrix; and
adjusting a head pose.

Assignees

• Google Inc

Inventors

• Binder Thomas
• Böhme Martin Christopher

Classifications

• G06K9/00228

  Physics · mapped topic

• G06T3/0093 Primary

  Physics · mapped topic

• G06V40/165

  using facial parts and geometric relationships · CPC title

• G06T3/18 Primary

  Image warping, e.g. rearranging pixels individually · CPC title