Sensor configuration switching for adaptation of video capturing frame rate

Patent metadata

Field	Value
Publication number	US-9570106-B2
Application number	US-201414558130-A
Country	US
Kind code	B2
Filing date	Dec 2, 2014
Priority date	Dec 2, 2014
Publication date	Feb 14, 2017
Grant date	Feb 14, 2017

Abstract

On the basis of a first configuration of one or more imaging sensors, first video data is captured at a first frame rate and a first resolution and second video data is captured at a second frame rate and a second resolution. The second frame rate is higher than the first frame rate and the second resolution is lower than the first resolution. Further, an amount of motion is detected in the captured second video data. On the basis of the detected amount of motion, the at least one imaging sensor is switched to a second configuration. On the basis of the second configuration, third video data is captured at a third frame rate and a third resolution. The third frame rate is higher than the first frame rate and the third resolution is higher than the second resolution.

First claim

The invention claimed is:

1. A method of capturing video, the method comprising:
configuring at least one imaging sensor in a first configuration;
on the basis of the first configuration, capturing first video data at a first frame rate and a first resolution and second video data at a second frame rate and a second resolution, the second frame rate being higher than the first frame rate and the second resolution being lower than the first resolution, wherein the first video data is captured at a same time that the second video data is captured;
detecting an amount of motion in the captured second video data;
on the basis of the detected amount of motion, switching the at least one imaging sensor to a second configuration; and
on the basis of the second configuration, capturing third video data at a third frame rate and a third resolution, the third frame rate being higher than the first frame rate and the third resolution being higher than the second resolution,
wherein the at least one imaging sensor comprises an array camera comprising a plurality of sub-cameras;
wherein in the first configuration only a subset of the sub-cameras is assigned to capturing the second video data while the other sub-cameras are assigned to capturing the first video data; and
wherein in the second configuration all the sub-cameras are assigned to capturing the third video data.

2. The method according to claim 1 ,
wherein the third resolution is equal to the first resolution.

3. The method according to claim 1 ,
wherein the third frame rate is equal to or higher than the second frame rate.

4. The method according to claim 1 ,
wherein the third frame rate is at least 100 frames per second.

5. The method according to claim 1 ,
wherein the subset of the sub-cameras consists of one of the sub-cameras.

6. The method according to claim 1 , comprising:
generating a slow motion video at least on the basis of the third video data.

7. The method according to claim 1 ,
wherein said amount of motion is detected on the basis of blur in the second video data.

8. A method of capturing video, the method comprising:
configuring at least one imaging sensor in a first configuration;
on the basis of the first configuration, capturing first video data at a first frame rate and a first resolution and second video data at a second frame rate and a second resolution, the second frame rate being higher than the first frame rate and the second resolution being lower than the first resolution, wherein the first video data is captured at a same time that the second video data is captured;
detecting an amount of motion in the captured second video data;
on the basis of the detected amount of motion, switching the at least one imaging sensor to a second configuration; and
on the basis of the second configuration, capturing third video data at a third frame rate and a third resolution, the third frame rate being higher than the first frame rate and the third resolution being higher than the second resolution,
wherein the at least one imaging sensor comprises first image sensor and a second imaging sensor having a lower performance than the first imaging sensor;
wherein in the first configuration the first image sensor is assigned to capturing the first video data while the second image sensor is assigned to capturing the second video data; and
wherein in the second configuration the first image sensor is assigned to capturing the third video data.

9. A method of capturing video, the method comprising:
configuring at least one imaging sensor in a first configuration;
on the basis of the first configuration, capturing first video data at a first frame rate and a first resolution and second video data at a second frame rate and a second resolution, the second frame rate being higher than the first frame rate and the second resolution being lower than the first resolution, wherein the first video data is captured at a same time that the second video data is captured;
detecting an amount of motion in the captured second video data;
on the basis of the detected amount of motion, switching the at least one imaging sensor to a second configuration;
on the basis of the second configuration, capturing third video data at a third frame rate and a third resolution, the third frame rate being higher than the first frame rate and the third resolution being higher than the second resolution; and
generating the slow motion video on the basis of the first video data, the second video data, and the third video data.

10. The method according to claim 9 , comprising:
on the basis of the first video data, calculating enhanced resolution video frames from the second video data.

11. The method according to claim 9 , comprising:
on the basis of the second video data, calculating intermediate video frames for the first video data.

12. A device, comprising:
at least one image sensor; and
at least one processor, the at least one processor being configured to perform operations comprising:
configuring the at least one imaging sensor in a first configuration;
on the basis of the first configuration, capturing first video data at a first frame rate and a first resolution and second video data at a second frame rate and a second resolution, the second frame rate being higher than the first frame rate and the second resolution being lower than the first resolution, wherein the first video data is captured at a same time that the second video data is captured;
detecting an amount of motion in the captured second video data;
on the basis of the detected amount of motion, switching the at least one imaging sensor to a second configuration; and
on the basis of the second configuration, capturing third video data at a third frame rate and a third resolution, the third frame rate being higher than the first frame rate and the third resolution being higher than the second resolution,
wherein the at least one imaging sensor comprises an array camera comprising a plurality of sub-cameras,
wherein in the first configuration only a subset of the sub-cameras is assigned to capturing the second video data while the other sub-cameras are assigned to capturing the first video data, and
wherein in the second configuration all the sub-cameras are assigned to capturing the third video data.

13. The device according to claim 12 ,
wherein the third resolution is equal to the first resolution.

14. The device according to claim 12 ,
wherein the third frame rate is equal to or higher than the second frame rate.

15. The device according to claim 12 ,
wherein the third frame rate is at least 100 frames per second.

16. The device according to claim 12 ,
wherein the subset of the sub-cameras consists of one of the sub-cameras.

17. The device according to claim 12 ,
wherein the at least one processor is configured to generate a slow motion video at least on the basis of the third video data.

18. A device, comprising:
at least one image sensor; and
at least one processor, the at least one processor being configured to perform operations comprising:
configuring the at least one imaging sensor in a first configuration;
on the basis of the first configuration, capturing first video data at a first frame rate and a first resolution and second video data at a second frame rate and a second resolution, the second frame rate being higher than the first frame rate and the second resolution being lower than the first resolution, wherein the first video data is captured at a same time that the s

Assignees

• Sony Corp
• Sony Mobile Communications Inc

Inventors

• Elg Johannes
• Mattisson Fredrik
• Linåker Daniel
• Gustavsson Jonas
• Thörn Ola
• Wikander Anders

Classifications

• H04N25/42

  by switching between different modes of operation using different resolutions or aspect ratios, e.g. switching between interlaced and non-interlaced mode · CPC title

• H04N23/667

  Camera operation mode switching, e.g. between still and video, sport and normal or high- and low-resolution modes · CPC title

• H04N25/443

  by reading pixels from selected two-dimensional [2D] regions of the array, e.g. for windowing or digital zooming · CPC title

• H04N23/45 Primary

  for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images · CPC title

• H04N5/343

  Electricity · mapped topic