Pixel circuit and method of operating the same in an always-on mode

Patent metadata

Field	Value
Publication number	US-11626446-B2
Application number	US-202017036357-A
Country	US
Kind code	B2
Filing date	Sep 29, 2020
Priority date	Aug 22, 2018
Publication date	Apr 11, 2023
Grant date	Apr 11, 2023

Abstract

An embodiment method of operating an imaging device including a sensor array including a plurality of pixels, includes: capturing a first low-spatial resolution frame using a subset of the plurality of pixels of the sensor array; generating, using a processor coupled to the sensor array, a first depth map using raw pixel values of the first low-spatial resolution frame; capturing a second low-spatial resolution frame using the subset of the plurality of pixels of the sensor array; generating, using the processor, a second depth map using raw pixel values of the second low-spatial resolution frame; and determining whether an object has moved in a field of view of the imaging device based on a comparison of the first depth map to the second depth map.

First claim

What is claimed is:

1. An imaging device, comprising:
a sensor array comprising an array of pixels, the sensor array being configured to:
at a first time, capture a first low-spatial resolution frame using a subset of pixels of the sensor array; and
at a second time after the first time, capture a second low-spatial resolution frame using the same subset of pixels of the sensor array; and

a processor coupled to the sensor array, the processor being configured to:
generate a first depth map using raw pixel values of the first low-spatial resolution frame;
generate a second depth map using raw pixel values of the second low-spatial resolution frame; and
compare the first depth map to the second depth map to determine whether an object has moved in a field of view of the imaging device.

2. The device of claim 1 , wherein, upon determining that the object has not moved in the field of view of the imaging device,
the sensor array is further configured to, at a third time after the second time, capture a third low-spatial resolution frame using the same subset of pixels of the sensor array;
the processor is further configured to:
generate a third depth map using raw pixel values of the third low-spatial resolution frame; and
determine whether an object has moved in the field of view of the imaging device based on a comparison of the second depth map to the third depth map.

3. The device of claim 1 , wherein the processor, upon determining that the object has moved in the field of view of the imaging device, is further configured to:
receive a high-spatial resolution frame from the sensor array; and
generate a third depth map using raw pixel values of the high-spatial resolution frame.

4. The device of claim 3 , wherein the processor is further configured to determine, based on the third depth map, at least one of a gesture performed by the object, a distance between the object and the imaging device, a feature of the object, or a motion characteristic of the object.

5. The device of claim 1 , wherein the raw pixel values of the first low-spatial resolution frame comprise a first time-of-flight measurement, a first brightness, and a first signal-to-noise ratio, and wherein the raw pixel values of the second low-spatial resolution frame comprise a second time-of-flight measurement, a second brightness, and a second signal-to-noise ratio.

6. The device of claim 1 , wherein the raw pixel values of the first low-spatial resolution frame comprise a first signal-to-noise ratio and the raw pixel values of the second low-spatial resolution frame comprise a second signal-to-noise ratio and wherein the processor is configured to determine whether the object has moved in the field of view of the imaging device by determining that the object has not moved in the field of view of the imaging device in response to the first signal-to-noise ratio or the second signal-to-noise ratio being less than a pixel signal-to-noise ratio limit.

7. The device of claim 1 , wherein the raw pixel values of the first low-spatial resolution frame comprise a first time-of-flight measurement and the raw pixel values of the second low-spatial resolution frame comprise a second time-of-flight measurement and wherein the processor is configured to determine whether the object has moved in the field of view of the imaging device by determining that the object has moved in the field of view of the imaging device in response to a change between the first time-of-flight measurement and the second time-of-flight measurement being greater than a threshold.

8. The device of claim 7 , wherein the threshold is a configurable predetermined limit dependent on an application of the imaging device.

9. The device of claim 1 , wherein the raw pixel values of the first low-spatial resolution frame comprise a first brightness and the raw pixel values of the second low-spatial resolution frame comprise a second brightness and wherein the processor is configured to determine whether the object has moved in the field of view of the imaging device by determining that the object has moved in the field of view of the imaging device in response to a change between the first brightness and the second brightness being greater than a threshold.

10. The device of claim 9 , wherein the threshold is a configurable predetermined limit dependent on an application of the imaging device.

11. The device of claim 1 , wherein the sensor array comprises an array of single-photon avalanche diodes (SPADs).

12. The device of claim 11 , further comprising a reference SPAD imaging array coupled to the processor.

13. The device of claim 12 , further comprising an optical barrier adjacent the reference SPAD imaging array.

14. The device of claim 1 , further comprising a time-to-digital converter (TDC) coupled to the processor, wherein the TDC is configured to send a digital signal to the processor each time an optical pulse is emitted by the device and a corresponding return portion of the optical pulse is detected at a given pixel, the digital signal being indicative of a time duration between a time of the emission the optical pulse and a time of reception of the return portion.

15. The device of claim 14 , wherein the device comprises a plurality of TDCs such that there is one TDC for each pixel.

16. The device of claim 14 , wherein the device comprises a plurality of TDCs, each TDC coupled to a plurality of pixels by an associated OR gate.

17. An imaging device, comprising:
a sensor array comprising an array of pixels;
a processor coupled to control the sensor array, the processor being configured to:
at a first time, receive a first low-spatial resolution frame based on a subset of the pixels of the sensor array;
at a second time after the first time, receive a second low-spatial resolution frame based on the same subset of the pixels of the sensor array;
perform a comparison based on the subset of pixels of the first low-spatial resolution frame and the subset of pixels of the second low-spatial resolution frame to determine whether an object has moved in a field of view of the imaging device;
when it has been determined that the object has moved in the field of view, capture a high-spatial resolution frame using pixels of the sensor array; and
when it has been determined that the object has not moved in the field of view, designate the second low-spatial resolution frame as the first low-spatial resolution frame and, repeat the steps of receiving the second low-spatial resolution frame and performing the comparison to again determine whether the object has moved in the field of view of the imaging device.

18. The imaging device of claim 17 , wherein the processor is further configured to:
generate a first depth map using raw pixel values of the first low-spatial resolution frame;
generate a second depth map using raw pixel values of the second low-spatial resolution frame; and
perform the comparison by comparing the first depth map and the second depth map.

19. The imaging device of claim 17 , wherein raw pixel values of the first low-spatial resolution frame comprise a first time-of-flight measurement, a first brightness, and a first signal-to-noise ratio, and wherein raw pixel values of the second low-spatial resolution frame comprise a second time-of-flight measurement, a second brightness, and a second signal-to-noise ratio.

20. A device, comprising:
a processor; and
a non-transitory computer-readable storage medium storing a program to be executed by the processor

Assignees

• St Microelectronics Res & Dev Ltd

Inventors

• Dutton Neale

Classifications

• H10W90/00

  Package configurations · CPC title

• H10F77/959

  for devices working in avalanche mode · CPC title

• H10F39/18 Primary

  Complementary metal-oxide-semiconductor [CMOS] image sensors; Photodiode array image sensors · CPC title

• H10F39/803 Primary

  Pixels having integrated switching, control, storage or amplification elements · CPC title

• H04N25/77

  Pixel circuitry, e.g. memories, A/D converters, pixel amplifiers, shared circuits or shared components · CPC title