Sensor configuration switching for adaptation of video capturing frame rate

Patent metadata

Field	Value
Publication number	US-2016155472-A1
Application number	US-201414558130-A
Country	US
Kind code	A1
Filing date	Dec 2, 2014
Priority date	Dec 2, 2014
Publication date	Jun 2, 2016
Grant date	—

Abstract

On the basis of a first configuration of one or more imaging sensors, first video data is captured at a first frame rate and a first resolution and second video data is captured at a second frame rate and a second resolution. The second frame rate is higher than the first frame rate and the second resolution is lower than the first resolution. Further, an amount of motion is detected in the captured second video data. On the basis of the detected amount of motion, the at least one imaging sensor is switched to a second configuration. On the basis of the second configuration, third video data is captured at a third frame rate and a third resolution. The third frame rate is higher than the first frame rate and the third resolution is higher than the second resolution.

First claim

1 . A method of capturing video, the method comprising:
configuring at least one imaging sensor in a first configuration; on the basis of the first configuration, capturing first video data at a first frame rate and a first resolution and second video data at a second frame rate and a second resolution, the second frame rate being higher than the first frame rate and the second resolution being lower than the first resolution; detecting an amount of motion in the captured second video data; on the basis of the detected amount of motion, switching the at least one imaging sensor to a second configuration; and on the basis of the second configuration, capturing third video data at a third frame rate and a third resolution, the third frame rate being higher than the first frame rate and the third resolution being higher than the second resolution.

2 . The method according to claim 1 ,
wherein the third resolution is equal to the first resolution.

3 . The method according to claim 1 ,
wherein the third frame rate is equal to or higher than the second frame rate.

4 . The method according to claim 1 ,
wherein the third frame rate is at least 100 frames per second.

5 . The method according to claim 1 ,
wherein the at least one imaging sensor comprises an array camera formed of a plurality of sub-cameras; wherein in the first configuration only a subset of the sub-cameras is assigned to capturing the second video data while the other subcameras are assigned to capturing the first video data; and wherein in the second configuration all the sub-cameras are assigned to capturing the third video data.

6 . The method according to claim 5 ,
wherein the subset of the sub-cameras corresponds to only one of the sub-cameras.

7 . The method according to claim 1 ,
wherein the at least one imaging sensor comprises first image sensor and a second imaging sensor having a lower performance than the first imaging sensor; wherein in the first configuration the first image sensor is assigned to capturing the first video data while the second image sensor is assigned to capturing the second video data; and wherein in the second configuration the first image sensor is assigned to capturing the third video data.

8 . The method according to claim 1 , comprising:
generating a slow motion video at least on the basis of the third video data.

9 . The method according to claim 8 , comprising:
generating the slow motion video on the basis of the first video data, the second video data, and the third video data.

10 . The method according to claim 9 , comprising:
on the basis of the first video data, calculating enhanced resolution video frames from the second video data.

11 . The method according to claim 9 , comprising:
on the basis of the second video data, calculating intermediate video frames for the first video data.

12 . The method according to claim 1 ,
wherein said amount of motion is detected on the basis of blur in the second video data.

13 . A device, comprising:
at least one image sensor; and at least one processor, the at least one processor being configured to:
configure the at least one imaging sensor in a first configuration;
on the basis of the first configuration, capture first video data at a first frame rate and a first resolution and second video data at a second frame rate and a second resolution, the second frame rate being higher than the first frame rate and the second resolution being lower than the first resolution;
detect an amount of motion in the captured second video data;
on the basis of the detected amount of motion, switch the at least one imaging sensor to a second configuration; and
on the basis of the second configuration, capture third video data at a third frame rate and a third resolution, the third frame rate being higher than the first frame rate and the third resolution being higher than the second resolution.

14 . The device according to claim 13 ,
wherein the third resolution is equal to the first resolution.

15 . The device according to claim 13 ,
wherein the third frame rate is equal to or higher than the second frame rate.

16 . The device according to claim 13 ,
wherein the third frame rate is at least 100 frames per second.

17 . The device according to claim 13 ,
wherein the at least one imaging sensor comprises an array camera formed of a plurality of sub-cameras; wherein in the first configuration only a subset of the sub-cameras is assigned to capturing the second video data while the other subcameras are assigned to capturing the first video data; and wherein in the second configuration all the sub-cameras are assigned to capturing the third video data.

18 . The device according to claim 17 ,
wherein the subset of the sub-cameras corresponds to only one of the sub-cameras.

19 . The device according to claim 13 ,
wherein the at least one imaging sensor comprises first imaging sensor and a second imaging sensor having a lower performance than the first imaging sensor; wherein in the first configuration the first imaging sensor is assigned to capturing the first video data and the second imaging sensor is assigned to capturing the second video data; and wherein in the second configuration the first imaging sensor is assigned to capturing the third video data.

20 . The device according to claim 13 ,
wherein the at least one processor is configured to generate a slow motion video at least on the basis of the third video data.

21 . The device according to claim 20 ,
wherein the at least one processor is configured to generate the slow motion video on the basis of the first video data, the second video data, and the third video data.

22 . The device according to claim 21 ,
wherein the at least one processor is configured to calculate, on the basis of the first video data, enhanced resolution video frames from the second video data.

23 . The device according to claim 21 , comprising:
wherein the at least one processor is configured to calculate, on the basis of the second video data, intermediate video frames for the first video data.

24 . The device according to claim 13 ,
wherein the at least one processor is configured to detect said amount of motion on the basis of blur in the second video data.

Assignees

• Sony Corp

Inventors

• Elg Johannes
• Mattisson Fredrik
• Linåker Daniel
• Gustavsson Jonas
• Thörn Ola
• Wikander Anders

Classifications

• H04N25/42

  by switching between different modes of operation using different resolutions or aspect ratios, e.g. switching between interlaced and non-interlaced mode · CPC title

• H04N25/443

  by reading pixels from selected two-dimensional [2D] regions of the array, e.g. for windowing or digital zooming · CPC title

• H04N23/667

  Camera operation mode switching, e.g. between still and video, sport and normal or high- and low-resolution modes · CPC title

• H04N23/45 Primary

  for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images · CPC title

• H04N7/0127

  by changing the field or frame frequency of the incoming video signal, e.g. frame rate converter · CPC title