Systems and methods for obtaining a smart panoramic image

Patent metadata

Field	Value
Publication number	US-11949976-B2
Application number	US-202117535730-A
Country	US
Kind code	B2
Filing date	Nov 26, 2021
Priority date	Dec 9, 2019
Publication date	Apr 2, 2024
Grant date	Apr 2, 2024

Abstract

Mobile handheld electronic devices such as smartphones, comprising a Wide camera for capturing Wide images with respective Wide fields of view (FOV W ), a Tele camera for capturing Tele images with respective Tele fields of view (FOV T ) smaller than FOV W , and a processor configured to stitch a plurality of Wide images into a panorama image with a field of view FOV P >FOV W and to pin a Tele image to a given location within the panorama image to obtain a smart panorama image.

First claim

What is claimed is:

1. A handheld device, comprising:
a Wide camera for capturing Wide images, each Wide image having a respective Wide field of view (FOV W );
a scanning Tele camera (STC) for capturing Tele images, each Tele image having a respective native Tele field of view (FOV T ) smaller than FOV W , wherein the STC is configured to scan with the native FOV T within FOV W ; and
a processor configured to capture the Tele images autonomously, to apply a particular strategy for the autonomous capturing of the Tele images that depends on an analysis of Wide image data, and to personalize the particular strategy for the autonomous capturing of the Tele images according to a preference of a particular user, wherein the Wide image data analysis provides a personalized saliency map based on Wide image data.

2. The handheld device of claim 1 , wherein the processor is additionally configured to capture the Wide images autonomously.

3. The handheld device of claim 1 , wherein the particular strategy for the autonomous capturing of the Tele images is defined by manual training by the particular user.

4. The handheld device of claim 1 , wherein the personalization according to a particular user preference includes defining particular objects that are of high value for the particular user.

5. The handheld device of claim 4 , wherein the particular objects are persons.

6. The handheld device of claim 4 , wherein the particular objects are animals.

7. The handheld device of claim 1 , wherein the processor is further configured to crop a Tele image.

8. The handheld device of claim 1 , wherein the processor is further configured to use a motion model that predicts a future movement of the handheld device.

9. The handheld device of claim 1 , wherein the processor is further configured to use a motion model that predicts a future movement of an object within the FOV W .

10. A handheld device, comprising:
a Wide camera for capturing Wide images, each Wide image having a respective Wide field of view (FOV W );
a scanning Tele camera (STC) for capturing Tele images, each Tele image having a respective native Tele field of view (FOV T ) smaller than FOV W , wherein the STC is configured to scan with the native FOV T within FOV W ; and
a processor configured to capture the Tele images autonomously, to apply a particular strategy for the autonomous capturing of the Tele images that depends on an analysis of Wide image data wherein the particular strategy for the autonomous capturing of the Tele images is defined automatically, and to personalize the particular strategy for the autonomous capturing of the Tele images according to a preference of a particular user, wherein the processor is further configured to define the particular strategy for the autonomous capturing of the Tele images based on a past behaviour of the particular user.

11. A handheld device, comprising:
a Wide camera for capturing Wide images, each Wide image having a respective Wide field of view (FOV W );
a scanning Tele camera (STC) for capturing Tele images, each Tele image having a respective native Tele field of view (FOV T ) smaller than FOV W wherein the STC is configured to scan with the native FOV T within FOV W ; and
a processor configured to capture the Tele images autonomously, to apply a particular strategy for the autonomous capturing of the Tele images that depends on an analysis of Wide image data, and to personalize the particular strategy for the autonomous capturing of the Tele images according to a preference of a particular user, wherein the particular strategy for the autonomous capturing of the Tele images is defined automatically, and wherein the processor is further configured to define the particular strategy for the autonomous capturing of the Tele images based on user content of the particular user stored in an image gallery.

12. A handheld device, comprising:
a Wide camera for capturing Wide images, each Wide image having a respective Wide field of view (FOV W );
a scanning Tele camera (STC) for capturing Tele images, each Tele image having a respective native Tele field of view (FOV T ) smaller than FOV W , wherein the STC is configured to scan with the native FOV T within FOV W ; and
a processor configured to capture the Tele images autonomously, to apply a particular strategy for the autonomous capturing of the Tele images that depends on an analysis of Wide image data wherein the particular strategy for the autonomous capturing of the Tele images is defined automatically, and to personalize the particular strategy for the autonomous capturing of the Tele images according to a preference of a particular user, wherein the processor is further configured to define the particular strategy for the autonomous capturing of the Tele images based on gaze tracking of a gaze of the particular user.

13. A handheld device, comprising:
a Wide camera for capturing Wide images, each Wide image having a respective Wide field of view (FOV W );
a scanning Tele camera (STC) for capturing Tele images, each Tele image having a respective native Tele field of view (FOV T ) smaller than FOV W , wherein the STC is configured to scan with the native FOV T within FOV W ; and
a processor configured to capture the Tele images autonomously, to apply a particular strategy for the autonomous capturing of the Tele images that depends on an analysis of Wide image data, and to personalize the particular strategy for the autonomous capturing of the Tele images according to a preference of a particular user, wherein the Tele and/or Wide images are included in video streams of respective Tele and/or Wide images, wherein a Wide image is selected to be output to a user from a video stream of Wide images that includes a scene similar to a scene included in a particular Tele image.

14. A handheld device, comprising:
a Wide camera for capturing Wide images, each Wide image having a respective Wide field of view (FOV W );
a scanning Tele camera (STC) for capturing Tele images, each Tele image having a respective native Tele field of view (FOV T ) smaller than FOV W wherein the STC is configured to scan with the native FOV T within FOV W ; and
a processor configured to capture the Tele images autonomously, to apply a particular strategy for the autonomous capturing of the Tele images that depends on an analysis of Wide image data, and to personalize the particular strategy for the autonomous capturing of the Tele images according to a preference of a particular user, wherein the personalized particular strategy for the autonomous capturing of the Tele images according to a preference of a particular user includes a strategy based on face and/or person recognition and/or identification in the Wide and/or STC image data.

15. A handheld device, comprising:
a Wide camera for capturing Wide images, each Wide image having a respective Wide field of view (FOV W );
a scanning Tele camera (STC) for capturing Tele images, each Tele image having a respective native Tele field of view (FOV T ) smaller than FOV W , wherein the STC is configured to scan with the native FOV T within FOV W ; and
a processor configured to capture the Tele images autonomously, to apply a particular strategy for the autonomous capturing of the Tele images that depends on an analysis of Wide image data, and to personalize the particular strategy for the autonomous capturing of the Tele images according to a preference of a particular user, wherein the processor configuration to apply a particular strategy for the autonomous capturing of the Tele images that depends on the Wide image data analysis

Assignees

• Corephotonics Ltd

Inventors

• Katz Ruthy
• Teitel Adi
• Shabtay Gal
• Mayberg Maya
• Cohen Noy

Classifications

• H04N23/611 Primary

  where the recognised objects include parts of the human body · CPC title

• G06T7/20

  Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51) · CPC title

• G06V40/18

  Eye characteristics, e.g. of the iris · CPC title

• H04N23/45

  for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images · CPC title

• H04N23/69

  Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming · CPC title