Driver mechanism for rolling shutter sensor to acquire structured light pattern

Patent metadata

Field	Value
Publication number	US-11284022-B2
Application number	US-202016996114-A
Country	US
Kind code	B2
Filing date	Aug 18, 2020
Priority date	Aug 7, 2020
Publication date	Mar 22, 2022
Grant date	Mar 22, 2022

Abstract

An apparatus includes an RGB-IR rolling shutter image sensor, a structured light projector, and a control circuit. The control circuit may be configured to control an exposure time of the RGB-IR rolling shutter image sensor and a turn on time of the structured light projector to obtain a structured light pattern in images captured by the RGB-IR rolling shutter image sensor.

First claim

The invention claimed is:

1. An apparatus comprising:
an RGB-IR rolling shutter image sensor;
a structured light projector; and
a control circuit configured to control an exposure time of said RGB-IR rolling shutter image sensor and a turn on time of said structured light projector to obtain a structured light pattern in images captured by said RGB-IR rolling shutter image sensor, wherein said control circuit is configured to (i) determine when a last line of said RGB-IR rolling shutter image sensor begins a respective exposure time and (ii) control the structured light projector to turn on the structured light pattern for a predefined period of time when the last line of said RGB-IR rolling shutter image sensor begins the respective exposure time.

2. The apparatus according to claim 1 , wherein said control circuit is further configured to:
receive a first signal from the RGB-IR rolling shutter image sensor, the first signal is configured to provide at least one of an indication of when said last line of said RGB-IR rolling shutter image sensor begins the respective exposure time and information from which a start of the respective exposure time can be calculated using a predefined formula; and
upon receiving the first signal, send a second signal to the structured light projector to turn on the structured light pattern for a predefined period of time.

3. The apparatus according to claim 2 , wherein said control circuit comprises a timer and is configured to program said time to control an on-time of said structured light projector in response to said second signal.

4. The apparatus according to claim 1 , further comprising an image signal processing circuit configured to process images captured by said RGB-IR rolling shutter image sensor, wherein said image signal processing circuit splits image data received from said RGB-IR rolling shutter image sensor into color image data and infrared (IR) image data.

5. The apparatus according to claim 4 , wherein the color image data comprises at least one of RGB or YUV color space data, and the IR image data comprises one or more of monochrome image data and the structured light pattern.

6. The apparatus according to claim 5 , wherein the control circuit is further configured to:
analyze the IR image data to obtain 3D information for a field of view of said RGB-IR rolling shutter image sensor; and
analyze the IR image data and the color image data to obtain one or more features in the field of view of said RGB-IR rolling shutter image sensor.

7. The apparatus according to claim 4 , wherein the control circuit is further configured to:
generate one or more control signals in response to analyzing the images captured by said RGB-IR rolling shutter image sensor; and
control one or more features of a security system in response to the one or more control signals.

8. The apparatus according to claim 7 , wherein the one or more features of said security system comprise one or more of a door lock, an alarm system, a controlled access system, and a payment system.

9. The apparatus according to claim 1 , wherein said apparatus comprises a low power camera.

10. The apparatus according to claim 1 , wherein the control circuit is configured to perform facial recognition and liveness determination.

11. The apparatus according to claim 1 , wherein said structured light projector comprises:
an array of vertical-cavity surface-emitting lasers configured to generate a laser light pattern; and
a lens configured to decompose the laser light pattern to a dense dot pattern array.

12. The apparatus according to claim 11 , wherein said array of vertical-cavity surface-emitting lasers emits light having a wavelength in an 800 to 1000 nanometers range.

13. A method of implementing a low-cost structured-light based 3D sensing system comprising:
setting an exposure time of an RGB-IR rolling shutter image sensor based on a predetermined value;
determining when a last line of said RGB-IR rolling shutter image sensor begins a respective exposure time; and
when the last line of said RGB-IR rolling shutter image sensor begins the respective exposure time, sending a control signal to turn on a structured-light projector to project a structured light pattern for a predetermined period of time.

14. The method according to claim 13 , further comprising:
programming a timer to said predetermined period of time for controlling an on-time of said structured-light projector; and
in response to receiving a signal from said RGB-IR rolling shutter image sensor indicating that said last line of said RGB-IR rolling shutter image sensor is starting the respective exposure time, triggering the timer to turn on said structured-light projector.

15. The method according to claim 13 , further comprising:
programming a timer to said predetermined period of time for controlling an on-time of said structured-light projector;
determining a trigger time for triggering the timer to turn on said structured-light projector in response a signal received from said RGB-IR rolling shutter image sensor providing information from which a start of the respective exposure time of said last line of said RGB-IR rolling shutter image sensor can be calculated using a predefined formula; and
triggering the timer to turn on said structured-light projector at said trigger time.

16. The method according to claim 13 , further comprising:
analyzing one or more images captured by said RGB-IR rolling shutter image sensor;
generate one or more control signals in response to a result of analyzing said one or more images; and
controlling one or more features of a security system in response to the one or more control signals.

17. The method according to claim 16 , wherein analyzing said one or more images captured by said RGB-IR rolling shutter image sensor comprises applying an artificial neural network for at least one of face detection and facial recognition.

18. The method according to claim 17 , further comprising:
controlling said one or more features of said security system in response to said facial recognition.

19. The method according to claim 18 , wherein the one or more features of said security system comprise one or more of a door lock, an alarm system, a controlled access system, and a payment system.

20. An apparatus comprising:
an interface configured to communicate with an RGB-IR rolling shutter image sensor and a structured light projector; and
a control circuit configured to control an exposure time of said RGB-IR rolling shutter image sensor and a turn on time of said structured light projector to obtain a structured light pattern in images captured by said RGB-IR rolling shutter image sensor, wherein said control circuit is configured to receive a first signal from said RGB-IR rolling shutter image sensor via said interface, the first signal is configured to provide at least one of an indication of when a last line of said RGB-IR rolling shutter image sensor begins a respective exposure time and information from which a start of the respective exposure time can be calculated using a predefined formula, and upon receiving the first signal, send a second signal via said interface to the structured light projector to turn on the structured light pattern for a predefined period of time.

Assignees

• Ambarella Int Lp

Inventors

• Zhang Zhe
• Zhao Long
• Yang Zhikan
• Xu Ruian

Classifications

• H04N25/531 Primary

  by controlling rolling shutters in CMOS SSIS · CPC title

• H04N25/77

  Pixel circuitry, e.g. memories, A/D converters, pixel amplifiers, shared circuits or shared components · CPC title

• H04N23/957

  Light-field or plenoptic cameras or camera modules · CPC title

• H04N23/73

  by influencing the exposure time · CPC title

• H04N23/56

  provided with illuminating means · CPC title