Gyroscope conditioning and gyro-camera alignment

Patent metadata

Field	Value
Publication number	US-9013617-B2
Application number	US-201213651055-A
Country	US
Kind code	B2
Filing date	Oct 12, 2012
Priority date	Oct 12, 2012
Publication date	Apr 21, 2015
Grant date	Apr 21, 2015

Abstract

An apparatus and method for generating parameters for an application, such as an augmented reality application (AR app), using camera pose and gyroscope rotation is disclosed. The parameters are estimated based on pose from images and rotation from a gyroscope (e.g., using least-squares estimation with QR factorization or a Kalman filter). The parameters indicate rotation, scale and/or non-orthogonality parameters and optionally gyroscope bias errors. In addition, the scale and non-orthogonality parameters may be used for conditioning raw gyroscope measurements to compensate for scale and non-orthogonality.

First claim

What is claimed is:

1. A method in a mobile device for generating parameters for an application of the mobile device, the method comprising:
receiving a sequence of images from a camera;
applying a computer vision algorithm to the sequence of images to generate pose angular information;
estimating gyroscope information from a sequence of gyroscope measurements; and
processing the pose angular information and the gyroscope information to derive at least two of: (1) rotation information, (2) scale information, and (3) non-orthogonality information comprising:
determining a translation matrix from the sequence of images; and
applying the pose angular information, the translation matrix and the gyroscope information to a Kalman filter to compute at least two of: (1) the rotation information; (2) the scale information; and (3) the non-orthogonality information, for use with the application.

2. The method of claim 1 , wherein the gyroscope information and the pose angular information are synchronized.

3. The method of claim 1 , further comprising providing the sequence of gyroscope measurements by conditioning a sequence of raw gyroscope measurements using at least one of the scale information or the non-orthogonality information.

4. The method of claim 1 , further comprising:
applying at least one of the scale information and the non-orthogonality information to the application.

5. The method of claim 1 , wherein processing the pose angular information and the gyroscope information comprises:
computing a composite matrix from the pose angular information and the gyroscope information; and
decomposing the composite matrix into a rotation matrix and scale and non-orthogonality matrix;
wherein the scale information and the non-orthogonality information comprises the scale and non-orthogonality matrix.

6. The method of claim 5 , wherein computing the composite matrix comprises applying a least-squares estimation to the pose angular information and the gyroscope information.

7. The method of claim 5 , wherein decomposing the composite matrix comprises applying a QR factorization to the composite matrix.

8. The method of claim 1 , wherein the pose angular information comprises a sequence of pose angular vectors.

9. The method of claim 1 , wherein the sequence of gyroscope measurements comprises a buffered sequence of gyroscope measurements, and wherein estimating the gyroscope information from the sequence of gyroscope measurements comprises:
obtaining a sequence of raw gyroscope measurements from a gyroscope; and
buffering the sequence of raw gyroscope measurements to obtain the buffered sequence of gyroscope measurements.

10. The method of claim 9 , wherein the gyroscope information comprises a sequence of gyroscope vectors.

11. The method of claim 1 , wherein the application comprises an augmented reality application.

12. The method of claim 1 , wherein processing the pose angular information and the gyroscope information further comprises deriving an estimated gyroscope bias error.

13. A mobile device for generating parameters for an application of the mobile device, the mobile device comprising:
a camera configured to provide a sequence of images;
a gyroscope configured to generate a sequence of gyroscope measurements; and
a processor coupled to the camera and the gyroscope, wherein the processor comprises:
a computer vision module coupled to receive the sequence of images from the camera, wherein the computer vision module generates pose angular information based on the sequence of images;
an angular velocity module to generate gyroscope information from the sequence of gyroscope measurements; and
a parameter estimation module coupled to receive the pose angular information from the computer vision module and coupled to receive the gyroscope information from the angular velocity module, wherein the parameter estimation module derives at least two of rotation information, scale information, and non-orthogonality information comprising:
determining a translation matrix from the sequence of images; and
applying the pose angular information, the translation matrix and the gyroscope information to a Kalman filter to compute at least two of the rotation information, the scale information and the non-orthogonality information, for use with the application of the mobile device.

14. The mobile device of claim 13 , further comprising a gyroscope buffer module coupled to the parameter estimation module to receive the scale information and the non-orthogonality information, wherein the gyroscope buffer module is configured to condition a sequence of raw gyroscope measurements using the scale information and the non-orthogonality information.

15. The mobile device of claim 13 , wherein the gyroscope is coupled to the parameter estimation module to receive the scale information and the non-orthogonality information, wherein the gyroscope is further to configured to calibrate a sequence of raw gyroscope measurements using the scale information and the non-orthogonality information.

16. The mobile device of claim 13 , wherein the parameter estimation module further derives rotation information.

17. The mobile device of claim 13 , wherein the parameter estimation module derives the scale information and the non-orthogonality information by:
computing a composite matrix from the pose angular information and the gyroscope information; and
decomposing the composite matrix into a rotation matrix and scale and non-orthogonality matrix, wherein the scale information and the non-orthogonality information comprises the scale and non-orthogonality matrix.

18. The mobile device of claim 17 , wherein the parameter estimation module computes the composite matrix by applying a least-squares estimation to the pose angular information and the gyroscope information.

19. The mobile device of claim 13 , wherein the scale information and the non-orthogonality information derived by the parameter estimation module is input to an augmented reality (AR) application.

20. A mobile device for generating parameters for an application of the mobile device, the mobile device comprising:
means for receiving a sequence of images from a camera;
means for applying a computer vision algorithm to the sequence of images to generate pose angular information;
means for estimating gyroscope information from a sequence of gyroscope measurements; and
means for processing the pose angular information and the gyroscope information to derive at least two of rotation information, scale information, and non-orthogonality information comprising:
means for determining a translation matrix from the sequence of images; and
means for applying the pose angular information, the translation matrix and the gyroscope information to a Kalman filter to compute at least two of the rotation information, the scale information and the non-orthogonality information, for use with the application of the mobile device.

21. The mobile device of claim 20 , wherein the scale information, the scale information and the non-orthogonality information comprises a scale and non-orthogonality matrix.

22. The mobile device of claim 20 , wherein the means for processing the pose angular information and the gyroscope information to derive at least two of the rotation information, the scale information and the non-orthogonality information comprises:
means for processing a sequence of pose angular vectors and a sequence of gyroscope vec

Assignees

• Qualcomm Inc

Inventors

• Ramachandran Mahesh
• Brunner Christopher
• Chari Murali R
• Diaz Spindola Serafin

Classifications

• G06T2207/10016

  Video; Image sequence · CPC title

• G06T7/0042

  Physics · mapped topic

• G01C25/00 Primary

  Manufacturing, calibrating, cleaning, or repairing instruments or devices referred to in the other groups of this subclass (testing, calibrating or compensating compasses G01C17/38) · CPC title

• G06T2207/30244

  Camera pose · CPC title

• G06T7/73

  using feature-based methods · CPC title