Flexible eyewear device with dual cameras for generating stereoscopic images

Patent metadata

Field	Value
Publication number	US-11575872-B2
Application number	US-202117521001-A
Country	US
Kind code	B2
Filing date	Nov 8, 2021
Priority date	Dec 20, 2018
Publication date	Feb 7, 2023
Grant date	Feb 7, 2023

Abstract

Three-dimensional image calibration and presentation for eyewear including a pair of image capture devices is described. Calibration and presentation includes obtaining a calibration offset to accommodate flexure in the support structure for the eyewear, adjusting a three-dimensional rendering offset by the obtained calibration offset, and presenting the stereoscopic images using the three-dimension rendering offset.

First claim

What is claimed is:

1. Image capture eyewear, including:
a support structure;
dual cameras connected to the support structure to capture stereoscopic images of a scene;
a processor;
a memory accessible to the processor; and
programming in the memory, wherein execution of the programming by the processor configures the image capture eyewear to perform functions, including functions to:
determine an amount of flexure of the support structure;

select a calibration offset from a plurality of predetermined calibration offsets responsive to the determined amount of flexure to accommodate flexure in the support structure;
adjust a three-dimensional rendering offset between two captured stereoscopic images of the scene from the dual cameras using the selected calibration offset; and
present the adjusted two captured stereoscopic images of the scene from the dual cameras as a three-dimensional image using the adjusted three-dimensional rendering offset.

2. The eyewear of claim 1 , wherein the dual cameras are positioned on the support structure of the eyewear a predetermined distance apart to capture stereoscopic images from different fields of view.

3. The eyewear of claim 1 , wherein the support structure includes a frame and temples extending from the frame, the dual cameras are supported by the frame, and the frame includes a rigid bar embedded therein for increasing the stiffness of the frame.

4. The eyewear of claim 1 , wherein the programming in the memory further obtains the calibration offset by calculating the calibration offset as a difference between an actual offset and a previously determined offset for one or more features in a known scene determined with eyewear not experiencing any flexure, where the actual offset is a number of pixels between the stereoscopic images for the one or more features in the known scene in respective stereoscopic images captured by the dual cameras.

5. The eyewear of claim 1 , wherein the programming in the memory further estimates the calibration offset based on a value generated by a strain gauge in the support structure.

6. The eyewear of claim 1 , wherein the programming in the memory for obtaining the calibration offset determines a distance to a known object in the image.

7. The eyewear of claim 1 , wherein the programming in the memory for presenting images includes blending the stereoscopic images using the adjusted offset.

8. The eyewear of claim 1 , wherein the functions to select a calibration offset includes a function to:
select the calibration offset from the plurality of predetermined calibration offsets responsive to the determined amount of flexure to accommodate flexure in the support structure, the plurality of predetermined calibration offsets determined by capturing stereoscopic images of a known scene, obtaining a distance to the known scene, identifying an actual pixel offset between the stereoscopic images for one or more image features, determining the predetermined calibration offsets by comparing the actual pixel offsets to a previously determined offset for the one or more features, and storing the predetermined calibration offsets.

9. A calibration method for image capture eyewear, the method comprising the steps of:
calculating a plurality of predetermined calibration offsets;
obtaining stereoscopic images from dual cameras supported by a support structure of an image capture eyewear device;
determining an amount of flexure of the support structure;
selecting a calibration offset from the plurality of predetermined calibration offsets responsive to the determined amount of flexure to accommodate flexure in the support structure between the dual cameras;
adjusting a three-dimensional rendering offset between two obtained stereoscopic images from the dual cameras using the selected calibration offset;
rendering the two obtained stereoscopic images from the dual cameras as a three-dimensional image using the adjusted three-dimensional rendering offset, wherein the adjusted three-dimensional rendering offset is adjusted by the selected calibration offset; and
presenting the three-dimensional image on a display.

10. The method of claim 9 , wherein calculating the plurality of predetermined calibration offsets comprises calculating each calibration offset as a difference between an actual offset and a previously determined offset for one or more features in a known scene determined with eyewear not experiencing any flexure, where the actual offset is a number of pixels between the stereoscopic images for the one or more features in the known scene in respective stereoscopic images captured by the dual cameras.

11. The method of claim 9 , wherein calculating the plurality of predetermined calibration offsets comprises estimating each calibration offset based on a value generated by a strain gauge in the support structure.

12. A non-transitory computer readable medium comprising instructions which, when executed by one or more processors, cause the one or more processors to perform operations comprising:
obtaining stereoscopic images from dual cameras supported by a support structure of an image capture eyewear device;
determining an amount of flexure of the support structure;
selecting a calibration offset from a plurality of predetermined calibration offsets responsive to the determined amount of flexure to accommodate flexure in the support structure between the dual cameras;
adjusting a three-dimensional rendering offset between two obtained stereoscopic images from the dual cameras using the selected calibration offset;
rendering the two obtained stereoscopic images from the dual cameras as a three-dimensional image using the adjusted three-dimensional rendering offset, wherein the adjusted three-dimensional rendering offset is adjusted by the selected calibration offset; and
presenting the three-dimensional image on a display.

13. The medium of claim 12 , further comprising instructions which, when executed by one or more processors, cause the one or more processors to perform additional operations comprising:
blending the stereoscopic images using the adjusted offset to create the three-dimensional image for presentation on the display.

14. The medium of claim 12 , wherein the image capture eyewear device includes a first display and a second display, the medium further including instructions which, when executed by one or more processors, cause the one or more processors to perform additional operations comprising:
presenting a left image component of the three-dimensional image on the first display and a right image component of the three-dimensional image on the second display.

15. The medium of claim 12 , further comprising instructions which, when executed by one or more processors, cause the one or more processors to perform additional operations comprising:
obtaining the calibration offset by calculating the calibration offset as a difference between an actual offset and a previously determined offset for one or more features in a known scene determined with eyewear not experiencing any flexure, where the actual offset is a number of pixels between the stereoscopic images for the one or more features in the known scene in respective stereoscopic images captured by the dual cameras.

16. The medium of claim 12 , further comprising instructions which, when executed by one or more processors, cause the one or more processors to perform additional operations comprising:
estimating the calibration offset based on a value generated by a strain gauge in the support structure.

17. The

Assignees

• Snap Inc

Inventors

• Ardisana Ii John Bernard
• Ben Ezra David
• Daube Nir
• Hintermann Mathias
• Katz Sagi
• Kligler Nati
• Ryner Michael Christian
• Shukla Ashutosh Y

Classifications

• G02B27/0172

  characterised by optical features · CPC title

• H04N13/344 Primary

  with head-mounted left-right displays · CPC title

• H04N23/54

  Mounting of pick-up tubes, electronic image sensors, deviation or focusing coils · CPC title

• G02B2027/0134

  of stereoscopic type · CPC title

• G02B2027/014

  comprising information/image processing systems · CPC title