Using an image sensor for always-on application within a mobile device

Patent metadata

Field	Value
Publication number	US-11962899-B2
Application number	US-202318112298-A
Country	US
Kind code	B2
Filing date	Feb 21, 2023
Priority date	Aug 18, 2020
Publication date	Apr 16, 2024
Grant date	Apr 16, 2024

Abstract

A mobile device includes an application processor and an image sensor. The application processor includes an imaging subsystem configured to process high resolution image data through a first interface and a sensor hub configured to process sensor data through a second interface. The image sensor operates in one of first and second modes. The image sensor is configured to capture the high resolution image data in response to a request from the imaging subsystem and the imaging subsystem is configured to access the high resolution image data using the first interface for performing a first operation, during the first mode. The image sensor is configured to capture low resolution image data and the sensor hub is configured to access the low resolution image data using the second bus for performing a second operation, during the second mode.

First claim

What is claimed is:

1. A mobile device comprising:
a CMOS image sensor configured to generate high resolution image data in a normal power state and low resolution image data in a low power state;
an application processor including an imaging subsystem configured to receive the high resolution image data in the normal power state, and a sensor hub configured to receive the low resolution image data in the low power state; and
a non-image sensor configured to generate sensor data and transmit the sensor data to the sensor hub,
wherein the imaging subsystem is further configured to process the high resolution image data,
wherein the sensor hub is further configured to receive both the low resolution image data and the sensor data of the non-image sensor from a same sensor bus and process the low resolution image data to change from the low power state to the normal power state, and
wherein the sensor hub is an always-on sensor-hub.

2. The mobile device of claim 1 , wherein the CMOS image sensor includes a register configured to indicate whether the mobile device is in the normal power state or the low power state, the register being set by the application processor.

3. The mobile device of claim 1 , wherein the non-image sensor includes at least one of an ambient light sensor, a motion sensor, a temperature sensor, an infrared sensor, a barometer, and an inertial movement unit (IMU) including a combination of accelerometers, gyroscopes, or magnetometers.

4. The mobile device of claim 1 , wherein the imaging subsystem includes an image data interface configured to receive the high resolution image data from the CMOS image sensor, and the sensor hub includes a sensor interface configured to receive the low resolution image data from the CMOS image sensor.

5. The mobile device of claim 4 , wherein the sensor interface is further configured to receive the sensor data from the non-image sensor.

6. The mobile device of claim 5 , wherein the sensor hub is configured to inform the CMOS image sensor of a start time and an end time during which it is capable of processing the low resolution image data, through the sensor interface, and the CMOS image sensor is configured to send an interrupt signal to the sensor hub at a time between the start time and the end time, through the sensor interface.

7. The mobile device of claim 6 , wherein the CMOS image sensor is configured to send the low resolution image data along with the interrupt signal to the sensor hub.

8. The mobile device of claim 6 , wherein the sensor hub is configured to retrieve the low resolution image data from the CMOS image sensor upon receiving the interrupt signal.

9. The mobile device of claim 5 , wherein the sensor hub is configured to send a control signal to the CMOS image sensor to inform the CMOS image sensor of a valid time and a store time, through the sensor interface, and the CMOS image sensor is configured to generate the low resolution image data before the valid time and retain the low resolution image data until the store time.

10. The mobile device of claim 9 , wherein when the sensor hub is activated, the sensor hub is configured to read the low resolution image data from the CMOS image sensor, and the CMOS image sensor is further configured to generate a next low resolution image data after the store time.

11. The mobile device of claim 10 , wherein a frame rate at which the CMOS image sensor generates the low resolution image data is synchronized with an activation frequency of the sensor hub.

12. The mobile device of claim 1 , wherein the CMOS image sensor is configured to periodically generate the low resolution image data.

13. The mobile device of claim 1 , wherein when the low resolution image data depicts a scene change, the sensor hub is configured to change state of the mobile device changes from the low power state to the normal power state.

14. A mobile device comprising:
a first CMOS image sensor configured to generate a first high resolution image data in a normal power state and a first low resolution image data in a low power state;
a second CMOS image sensor configured to generate a second high resolution image data in the normal power state and not generate any image data in the low power state;
an application processor including an imaging subsystem configured to receive the first high resolution image data and the second high resolution image data from the first CMOS image sensor and the second CMOS image sensor in the normal power state, and a sensor hub configured to receive the first low resolution image data from the first CMOS image sensor in the low power state; and
a non-image sensor configured to generate sensor data and transmit the sensor data to the sensor hub, and
wherein the imaging subsystem is further configured to process the first high resolution image data and the second high resolution image data, the sensor hub is further configured to process the first low resolution image data to change from the low power state to the normal power state, and the sensor hub is an always-on sensor-hub.

15. The mobile device of claim 14 , wherein the first CMOS image sensor is a front camera of the mobile device.

16. The mobile device of claim 14 , wherein the first CMOS image sensor includes a register configured to indicate whether the mobile device is in the normal power state or the low power state, the register being set by the application processor.

17. The mobile device of claim 14 , wherein the sensor hub is further configured to receive the sensor data from the non-image sensor, and process the first low resolution image data and the sensor data to change from the low power state to the normal power state.

18. The mobile device of claim 14 , further comprising:
a third CMOS image sensor configured to generate a third high resolution image data in the normal power state and a second low resolution image data in the low power state.

19. The mobile device of claim 18 , wherein the imaging subsystem is further configured to receive the third high resolution image data in the normal power state and process the third high resolution image data in the normal power state, and the sensor hub is further configured to receive the second low resolution image data in the low power state and process the first low resolution image data and the second low resolution image data to change from the low power state to the normal power state.

Assignees

• Samsung Electronics Co Ltd

Inventors

• Elizov Roee
• Yaffe Yoel

Classifications

• H04N23/667 Primary

  Camera operation mode switching, e.g. between still and video, sport and normal or high- and low-resolution modes · CPC title

• G06F3/017

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title

• G06T7/0012

  Biomedical image inspection · CPC title

• H04N23/651

  for reducing power consumption by affecting camera operations, e.g. sleep mode, hibernation mode or power off of selective parts of the camera · CPC title

• G06T2207/30201

  Face · CPC title