Apparatus and method for seamless video capture during flex-state transition

Patent metadata

Field	Value
Publication number	US-12556830-B2
Application number	US-202418799784-A
Country	US
Kind code	B2
Filing date	Aug 9, 2024
Priority date	Aug 11, 2023
Publication date	Feb 17, 2026
Grant date	Feb 17, 2026

Abstract

A method for seamless video capture during flex-state transition in a foldable device includes identifying, by one or more sensors of the foldable device, an initiation of a flex movement of the foldable device based on a plurality of frames of a video being captured by a source camera from among one or more cameras of the foldable device; extracting, based on the identifying of the initiation of the flex movement, a semantic scene from the plurality of frames to determine one or more regions of interest (ROIs) in the semantic scene; determining an optical flow for each of the one or more ROIs; determining a flex trajectory of the foldable device; determining a target camera from among the one or more cameras; determining a transition period to switch to the target camera; and switching capturing of the plurality of frames from the source camera to the target camera.

First claim

What is claimed is:

1 . A method for seamless video capture in a foldable device, the method comprising:
identifying, by one or more sensors of the foldable device, an initiation of a flex movement of the foldable device for flex-state transition based on a plurality of frames of a video being captured by a source camera from among one or more cameras; extracting, based on the identifying of the initiation of the flex movement, a semantic scene from the plurality of frames of the video to determine one or more regions of interest (ROIs); determining an optical flow for each of the one or more ROIs which encodes a motion of ROIs in subsequent frames of the plurality of frames; determining a flex trajectory of the foldable device by forecasting an orientation of the foldable device with respect to the flex movement; determining a target camera from among the one or more cameras, based on the optical flows and the flex trajectory of the foldable device; determining a transition period to switch to the target camera; and switching from displaying at least one frame captured by the source camera to displaying at least one frame captured by the target camera, based on the transition period.

2 . The method of claim 1 , further comprising:
receiving, from the one or more sensors, sensor data and information corresponding to the flex movement, the sensor data comprising at least one of gyroscope data and accelerometer data; and determining a type of the flex movement based on the sensor data, wherein the determining of the flex trajectory of the foldable device comprises:
forecasting, using a machine learning model, the orientation of the foldable device based on the sensor data and the type of the flex movement; and
determining the flex trajectory of the foldable device based on the orientation of the foldable device.

3 . The method of claim 1 , wherein the one or more ROIs comprises at least one of:
one or more persons that are present in the semantic scene; and one or more objects that are present in the semantic scene.

4 . The method of claim 1 , wherein the determining of the optical flow for each of the one or more ROIs comprises:
determining, using a machine learning model, the optical flow for each of the one or more ROIs in the subsequent frames of the plurality of frames, based on frames captured by the source camera prior to the identifying of the initiation of the flex movement.

5 . The method of claim 1 , wherein the determining of the target camera comprises:
receiving, from the one or more cameras, position information of the one or more cameras; determining, based on the flex trajectory, a final orientation of the foldable device corresponding to a completion of the flex-state transition; and determining based on the optical flow and the position information, the target camera, the target camera having a final line of sight equal to a predetermined line of sight at the final orientation.

6 . The method of claim 5 , wherein the determining of the transition period comprises:
detecting a first line of sight of the source camera and a second line of sight of the target camera, based on the optical flow and the flex trajectory; determining a first timestamp based on detecting a loss of the first line of sight of the source camera; determining a second timestamp based on detecting a gain of the second line of sight of the target camera; and determining the transition period based on the first timestamp and the second timestamp.

7 . The method of claim 5 , further comprising:
determining a hinge angle of the foldable device during the flex movement, wherein the determining of the transition period comprises:
detecting a first line of sight of the source camera and a second line of sight of the target camera, based on the optical flow and the hinge angle;
determining a first timestamp based on detecting a loss of the first line of sight of the source camera;
determining a second timestamp based on detecting a gain of the second line of sight of the target camera; and
determining the transition period based on the first timestamp and the second timestamp.

8 . The method of claim 7 , wherein the determining of the hinge angle of the foldable device comprises:
extracting, using a machine learning model, temporal semantic representation of sensor data of the one or more sensors at each output timestamp; and determining, using a logistic regression model, the hinge angle at each output timestamp based on the temporal semantic representation of the sensor data.

9 . The method of claim 7 , wherein the determining of the hinge angle of the foldable device comprises:
generating, using a Hall effect sensor, an output signal proportional to a magnetic flux of a magnet pair on edges of two displays of the foldable device that are attached to a hinge of the foldable device; and determining, using a linear mapping, the hinge angle based on the output signal.

10 . The method of claim 7 , wherein the determining of the hinge angle of the foldable device comprises determining the hinge angle based on sensor data of the one or more sensors and an output of a Hall effect sensor.

11 . The method of claim 7 , wherein the determining of the hinge angle of the foldable device comprises:
generating, using an ambient light sensor, an output signal proportional to an illumination intensity of light; and determining, using a linear mapping, the hinge angle based on the output signal.

12 . The method of claim 7 , wherein the determining of the hinge angle of the foldable device comprises:
determining an intermittent time interval taken by a user associated with the foldable device in performing the flex-state transition by tracking user flexing dynamics behavior; and determining the hinge angle based on a mapping of the intermittent time interval with the hinge angle.

13 . The method of claim 1 , further comprising:
integrating the subsequent frames captured by the source camera and the target camera in the transition period to form a plurality of target frames; correcting a position of each of the one or more ROIs in the plurality of target frames; eliminating, based on the corrected position of each of the one or more ROIs in the plurality of target frames, destabilization induced during the flex movement; identifying a transition effect based on a shifting of camera parameters, semantic context of the semantic scene being captured, and a user preference for transition effect type; and compositing a final frame by blending the transition effect in the transition period and consolidating each of the plurality of target frames to generate an output video.

14 . A foldable device, comprising:
one or more cameras; one or more sensors configured to identify an initiation of a flex movement of the foldable device for flex-state transition based on a plurality of frames of a video being captured by a source camera from among the one or more cameras; a memory storing instructions; and one or more processors communicatively coupled to the memory, wherein the one or more processors are configured to execute the instructions to:
extract, based on identification of the initiation of the flex movement, a semantic scene in the plurality of frames of the video to determine one or more regions of interest (ROIs);
determine an optical flow for each of the one or more ROIs, which encodes a motion of ROIs in subsequent frames of the plurality of frames;
determine a flex trajectory of the foldable device by forecasting an orientation of the foldable device with respect to the flex

Assignees

• Samsung Electronics Co Ltd

Inventors

• Kasaraneni Sai Hemanth
• Jain Nitin
• Anand Gunit
• Yadav Chhavi
• Kumar Baljeet

Classifications

• H04N23/683

  performed by a processor, e.g. controlling the readout of an image memory · CPC title

• G06T2207/20212

  Image combination · CPC title

• G06T2207/30196

  Human being; Person · CPC title

• G06T2207/30241

  Trajectory · CPC title

• H04N23/62

  Control of parameters via user interfaces · CPC title