Multiple orientation detection

Patent metadata

Field	Value
Publication number	US-10347218-B2
Application number	US-201715647034-A
Country	US
Kind code	B2
Filing date	Jul 11, 2017
Priority date	Jul 12, 2016
Publication date	Jul 9, 2019
Grant date	Jul 9, 2019

Abstract

Methods, systems, computer-readable media, and apparatuses for image processing and utilization are presented. In some embodiments, an image containing at a face of a user may be obtained using a mobile device. An orientation of the face of the user within the image may be determined using the mobile device. The orientation of the face of the user may be determined using multiple stages: (a) a rotation stage for controlling a rotation applied to a portion of the image, to generate a portion of rotated image, and (b) an orientation stage for controlling an orientation applied to orientation-specific feature detection performed on the portion of rotated image. The determined orientation of the face of the user may be utilized as a control input to modify a display rotation of the mobile device.

First claim

What is claimed is:

1. A method for image processing and utilization, comprising:
obtaining, using a mobile device, an image containing a face of a user of the mobile device;
determining, using the mobile device, an orientation of the face of the user within the image, wherein the orientation of the face of the user within the image is determined using multiple stages, the multiple stages comprising:
(a) a rotation stage for controlling a rotation along a rotation axis applied to a portion of the image, to generate a portion of rotated image; and
(b) an orientation stage for operating a feature detector at a plurality of orientation offsets along the rotation axis, including a first orientation offset and a second orientation offset, using the portion of rotated image,

wherein for the first orientation offset, the first orientation offset is provided as an input to the detector to instruct the detector to perform feature detection at the first orientation offset on the portion of rotated image;
wherein for the second orientation offset, the second orientation offset is provided as an input to the detector to instruct the detector to perform feature detection at the second orientation offset on the portion of rotated image; and
utilizing the determined orientation of the face of the user as a control input to modify a display rotation of the mobile device.

2. The method of claim 1 , wherein the portion of rotated image is stored in a rotation buffer.

3. The method of claim 1 , wherein the orientation-specific feature detection is performed using a computer vision (CV) computation unit.

4. The method of claim 1 , wherein the orientation of the face of the user within the image is determined based on multiple modes, the multiple modes comprising:
(a) a detection mode for detecting an initial orientation of the face of the user within an initial image; and
(b) a tracking mode for tracking the orientation of the face of the user within a subsequent image, using the detected initial orientation.

5. The method of claim 4 ,
wherein in the detection mode, the initial orientation of the face of the user within the image is detected by performing feature detection at a first plurality of hypothesized angles; and
wherein in the tracking mode, the orientation of the face of the user within the subsequent image is tracked by performing feature detection at a second plurality of hypothesized angles, the second plurality being fewer than the first plurality.

6. The method of claim 4 ,
wherein in the tracking mode, the orientation of the face of the user within the subsequent image is tracked upon detection of a trigger condition associated with a non-image sensor.

7. The method of claim 6 , wherein the non-image sensor comprises an accelerometer.

8. An apparatus for image processing and utilization, comprising:
an image sensor configured to obtain an image containing at least a face of a user of a mobile device;
a rotation buffer configured to store a portion of rotated image resulting from applying a rotation along a rotation axis to a portion of the image;
a computer vision (CV) computation unit configured to determine an orientation of the face of the user within the image, wherein the CV computation unit is configured to
perform feature detection at a plurality of orientation offsets along the rotation axis, including a first orientation offset and a second orientation offset, using the portion of rotated image,

wherein for the first orientation offset, the first orientation offset is provided as an input to the CV computation unit to instruct the CV computation unit to perform feature detection at the first orientation offset on the portion of rotated image;
wherein for the second orientation offset, the second orientation offset is provided as an input to the CV computation unit to instruct the CV computation unit to perform feature detection at the second orientation offset on the portion of rotated image; and
a controller configured to receive and organize the determined orientation of the face of the user as a control input to modify a display rotation of the mobile device.

9. The apparatus of claim 8 , wherein the CV computation unit is configured to determine the orientation of the face of the user on multiple modes, the multiple modes comprising:
(a) a detection mode for detecting an initial orientation of the face of the user within an initial image; and
(b) a tracking mode for tracking the orientation of the face of the user within a subsequent image, using the detected initial orientation.

10. The apparatus of claim 9 ,
wherein in the detection mode, the CV computation unit is configured to detect the initial orientation of the face of the user within the image by performing feature detection at a first plurality of hypothesized angles; and
wherein in the tracking mode, the CV computation unit is configured to track the orientation of the face of the user within the subsequent image by performing feature detection at a second plurality of hypothesized angles, the second plurality being fewer than the first plurality.

11. The apparatus of claim 9 ,
wherein in the tracking mode, the CV computation unit is configured to track the orientation of the face of the user within the subsequent image based upon detection of a trigger condition associated with a non-image sensor.

12. The apparatus of claim 11 , wherein the non-image sensor comprises an accelerometer.

13. A non-transitory computer-readable medium having instructions embedded thereon for image processing and utilization, the instructions, when executed by one or more processing units, cause the one or more processing units to perform:
obtaining, using a mobile device, an image containing a face of a user of the mobile device;
determining, using the mobile device, an orientation of the face of the user within the image, wherein the orientation of the face of the user within the image is determined using multiple stages, the multiple stages comprising:
(a) a rotation stage for controlling a rotation along a rotation axis applied to a portion of the image, to generate a portion of rotated image; and
(b) an orientation stage for operating a feature detector at a plurality of orientation offsets along the rotation axis, including a first orientation offset and a second orientation offset, using the portion of rotated image,

wherein for the first orientation offset, the first orientation offset is provided as an input to the detector to instruct the detector to perform feature detection at the first orientation offset on the portion of rotated image;
wherein for the second orientation offset, the second orientation offset is provided as an input to the detector to instruct the detector to perform feature detection at the second orientation offset on the portion of rotated image; and
utilizing the determined orientation of the face of the user as a control input to modify a display rotation of the mobile device.

14. The non-transitory computer-readable medium of claim 13 , wherein the portion of rotated image is stored in a rotation buffer.

15. The non-transitory computer-readable medium of claim 13 , wherein the orientation-specific feature detection is performed using a computer vision (CV) computation unit.

16. The non-transitory computer-readable medium of claim 13 , wherein the orientation of the face of the user within the image is determined based on multiple modes, the multiple modes comprising:
(a) a detection mode for detecting an initial orientation of the face of the user within an initial image;

Assignees

• Qualcomm Inc

Inventors

• Park Edwin Chongwoo
• Krishnamoorthi Raghuraman
• Rangan Venkat
• Rasquinha Nelson

Classifications

• G06T3/606

  by memory addressing or mapping · CPC title

• G09G2354/00

  Aspects of interface with display user · CPC title

• G09G2360/12

  Frame memory handling · CPC title

• G09G2340/0492

  Change of orientation of the displayed image, e.g. upside-down, mirrored · CPC title

• G09G2360/08

  Power processing, i.e. workload management for processors involved in display operations, such as CPUs or GPUs · CPC title