Eyewear having unsynchronized rolling shutter cameras

Patent metadata

Field	Value
Publication number	US-11726327-B2
Application number	US-202017021023-A
Country	US
Kind code	B2
Filing date	Sep 15, 2020
Priority date	Sep 15, 2020
Publication date	Aug 15, 2023
Grant date	Aug 15, 2023

Abstract

Eyewear having unsynchronized rolling shutter (RS) cameras such that images produced by each camera are unaligned, and a state that includes velocity and gravity orientation in the eyewear's reference system is calculated. The state is not limited to these two parameters as other parameters such as the acceleration bias, the gyroscope bias, or both may be included. Math solvers are used such that processing time to calculate the velocity and gravity orientation are acceptable. Arranging the RS cameras in an unsynchronized configuration allows estimating the motion of the eyewear from just one stereo image pair and removes the requirement of possessing more images.

First claim

What is claimed is:

1. Eyewear, comprising:
a frame;
a pair of cameras each coupled to the frame and configured to capture a respective first camera image and a second camera image, wherein the cameras are unsynchronized; and
an electronic processor configured to:
receive the first image and the second image from the unsynchronized cameras;
process the first image and the second image based on a characteristic of the first image relative to the second image; and
calculate a velocity of the eyewear as a function of the characteristic.

2. The eyewear of claim 1 , wherein the electronic processor also calculates gravity of the eyewear.

3. The eyewear of claim 1 , wherein the characteristic is selected from a group of: the first image is rotated relative to the second image, the first image is shifted in time relative to the second image, and the first image is spatially shifted relative to the second image.

4. The eyewear of claim 1 , wherein the processor is configured to use a linear equation to calculate the velocity of the eyewear.

5. The eyewear of claim 1 , wherein the cameras comprise rolling shutter cameras and the first image and the second image include a common object point.

6. The eyewear of claim 5 , wherein the processor is configured to process a time difference between the pair of cameras imaging the object point.

7. The eyewear of claim 1 , wherein the eyewear comprises an inertial measurement unit (IMU) configured to generate IMU readings, wherein the processor is configured to process the IMU readings to determine the velocity.

8. A method of use of eyewear having a frame, a pair of cameras each coupled to the frame and configured to capture a respective first camera image and a second camera image, wherein the pair of cameras are unsynchronized, and an electronic processor:
receiving the first image and the second image from the unsynchronized cameras;
processing the first image and the second image based on a characteristic of the first image relative to the second image; and
calculating a velocity of the eyewear as a function of the characteristic.

9. The method of claim 8 , wherein the electronic processor calculates gravity of the eyewear.

10. The method of claim 8 , wherein the characteristic is selected from a group of: the first image is rotated relative to the second image, the first image is shifted in time relative to the second image, and the first image is spatially shifted relative to the second image.

11. The method of claim 8 , wherein the cameras comprise rolling shutter cameras and processor uses a linear equation to calculate the velocity of the eyewear.

12. The method of claim 8 , wherein the first image and the second image include a common object point.

13. The method of claim 12 , wherein the processor processes a time difference between the pair of cameras imaging the object point.

14. The method of claim 8 , wherein the eyewear comprises an inertial measurement unit (IMU) configured to generate IMU readings, wherein the processor processes the IMU readings to determine the velocity.

15. A non-transitory computer-readable medium storing program code which, when executed, is operative to cause a processor of eyewear having a frame, a pair of unsynchronized cameras each coupled to the frame and configured to capture a respective first camera image and a second camera image to perform the steps of:
receive the first image and the second image from the unsynchronized cameras;
process the first image and the second image based on a characteristic of the first image relative to the second image; and
calculate a velocity of the eyewear as a function of the characteristic.

16. The non-transitory computer-readable medium as specified in claim 15 further comprising code to calculate gravity of the eyewear.

17. The non-transitory computer readable medium of claim 15 , wherein the characteristic is selected from a group of: the first image is rotated relative to the second image, the first image is shifted in time relative to the second image, and the first image is spatially shifted relative to the second image.

18. The non-transitory computer-readable medium of claim 15 , further comprising code to calculate the velocity of the eyewear using a linear equation.

19. The non-transitory computer-readable medium of claim 15 , wherein the first image and the second image include a common object point.

20. The non-transitory computer-readable medium of claim 19 , further comprising code to process a time difference between the pair of cameras imaging the object point.

Assignees

• Micusik Branislav
• Evangelidis Georgios
• Snap Inc

Inventors

• Micusik Branislav
• Evangelidis Georgios

Classifications

• G02B27/0172 Primary

  characterised by optical features · CPC title

• G02B27/0176

  characterised by mechanical features · CPC title

• G06V10/42

  Global feature extraction by analysis of the whole pattern, e.g. using frequency domain transformations or autocorrelation · CPC title

• G02B2027/014

  comprising information/image processing systems · CPC title

• G02B2027/0138

  comprising image capture systems, e.g. camera · CPC title