Computational imaging of the electric grid

Patent metadata

Field	Value
Publication number	US-11601602-B2
Application number	US-201917254384-A
Country	US
Kind code	B2
Filing date	Jun 20, 2019
Priority date	Jun 20, 2018
Publication date	Mar 7, 2023
Grant date	Mar 7, 2023

Abstract

A method comprising receiving data representing light intensity values corresponding to a flicker pattern of a reference light source powered by an alternating current (AC); operating a controllable illumination source, based, at least in part, on said data; capturing, using an imaging device, a sequence of images of a scene illuminated, at least in part, by said controllable illumination source; estimating an intensity value for at least one pixel in said array, correspondingly in each of said images in said sequence of images; and determining a temporal point in said flicker pattern of said reference light source, based, at least in part, on said estimating.

First claim

The invention claimed is:

1. A system comprising an imaging device comprising a light sensing element comprising an array of pixels;
a controllable illumination source;
at least one hardware processor; and
a non-transitory computer-readable storage medium having stored thereon program instructions, the program instructions executable by the at least one hardware processor to:
receive data representing light intensity values corresponding to a flicker pattern of a reference light source powered by an alternating current (AC) of an AC grid;
operate said controllable illumination source, based on said data, to replicate said flicker pattern of said reference light source;
capture, using said imaging device, a sequence of images of a scene illuminated by one or more light sources attached to the AC grid, and a diffuse surface, illuminated at least in part, by said controllable illumination sourced;
calculate an intensity value for at least one specified pixel in said array in each of said images in said sequence of images, to estimate a temporal response of a specific light source of the one or more light sources to the AC in relation to the flicker pattern; and
determine one or more characteristics of said AC grid, based, at least in part, on said estimating.

2. The system of claim 1 , wherein said one or more characteristics are selected from the group consisting of:
a temporal point in said flicker pattern of said reference light source, a phase of said AC grid at the specific light source of the one or more light sources, phase changes of said AC grid at the specific light source, frequency fluctuations of said AC grid at the specific light source, amplitude of said AC grid at the specific light source, amplitude fluctuations of said AC grid at the specific light source, and nonlinearities in said AC grid at the specific light source.

3. The system of claim 1 , wherein said estimating further comprises calculating relative temporal differences in said estimated intensity values of said at least one specified pixel in said sequence of images.

4. The system of claim 1 , wherein said scene is located outside a direct illumination area of said reference light source.

5. The system of claim 1 , wherein said light sensor further comprises an optical module located in the optical path between said light sensor and said reference light source, wherein said optical module is configured to perform at least one of: amplify said light flux, focus said light flux, and direct said light flux.

6. The system of claim 1 , wherein said at least one specified pixel corresponds to a location within the diffuse surface.

7. The system of claim 1 , wherein said imaging device further comprises a mask configured to expose said scene to said light sensing element in a spatially varying manner, and wherein said mask is a rolling shutter configured to expose rows of said light sensing element in a sequential manner, and said at least one specified pixel comprises a vertical or horizontal sequence of pixels.

8. The system of claim 7 , wherein said mask is configured to expose regions of said light sensing element in a random manner.

9. The system of claim 1 , wherein said determining comprises detecting a phase of said AC current powering said reference light source.

10. The system of claim 1 , wherein the at least one hardware processor is further configured to extract different components of the image, related to different light sources of the one or more light sources, based on said estimation.

11. The system of claim 1 , wherein the at least one hardware processor is further configured to:
determine, based on said estimation, (a) a contribution of each specific light source to the total illumination in the image, thus decomposing the lighting of the scene, and (b) a chromaticity value of at least one specific light source; and
correct a color in at least one image of the sequence of images of the scene, based on said decomposition and determined chromaticity.

12. The system of claim 1 , wherein the at least one hardware processor is further configured to determine, based on said estimation, at least one of: a response function of the specific light source, a chromaticity value of the specific light source, a contribution of each specific light source to the total illumination in the image, and an electric phase of the specific light source.

13. The system of claim 1 , wherein the at least one hardware processor is further configured to:
determine, based on said estimation, a contribution of each specific light source to the total illumination in the image, thus decomposing the lighting of the scene; and
create at least one new image of the scene, each image simulating an appearance of the scene under illumination by not all of the at least one of the one or more light sources, based on said decomposition.

14. A method comprising:
operating at least one hardware processor for:
receiving data representing light intensity values corresponding to a flicker pattern of a reference light source powered by an alternating current (AC) of an AC grid;
operating a controllable illumination source, based on said data, to replicate said flicker pattern of said reference light source;
operating an imaging device to capture a sequence of images of a scene illuminated, by one or more light sources attached to the AC grid, and a diffuse surface, illuminated at least in part, by said controllable illumination sourced;
calculating an intensity value for at least one specified pixel in said array in each of said images in said sequence of images, to estimate a temporal response of a specific light source of the one or more light sources to the AC in relation to the flicker pattern; and
determining one or more characteristics of said AC grid, based, at least in part, on said estimating.

15. The method of claim 14 , wherein said one or more characteristics are selected from the group consisting of:
a temporal point in said flicker pattern of said reference light source, a phase of said AC grid at the specific light source of the one or more light sources, phase changes of said AC grid at the specific light source, frequency fluctuations of said AC grid at the specific light source, amplitude of said AC grid at the specific light source, amplitude fluctuations of said AC grid at the specific light source, and nonlinearities in said AC grid at the specific light source.

16. The method of claim 14 , wherein said estimating further comprises calculating relative temporal differences in said estimated intensity values of said at least one specified pixel in said sequence of images.

17. The method of claim 14 , wherein said scene is located outside a direct illumination area of said reference light source.

18. The method of claim 14 , wherein said at least one specified pixel corresponds to a location within the diffuse surface.

19. A computer program product comprising a non-transitory computer-readable storage medium having program instructions embodied therewith, the program instructions executable by at least one hardware processor to:
receive data representing light intensity values corresponding to a flicker pattern of a reference light source powered by an alternating current (AC);
operate a controllable illumination source, based on said data to replicate said flicker pattern of said reference light source;
operate an imaging device to capture a sequence of images of a scene illuminated, by one or more light sources attached to the AC grid, and a diffuse surfa

Assignees

• Technion Res & Dev Foundation
• Governing Council Univ Toronto

Inventors

• Sheinin Mark
• Schechner Yoav
• Levron Yoash
• Kutulakos Kiriakos Neoklis

Classifications

• G03B7/093 Primary

  for control of exposure time · CPC title

• H04N23/745 Primary

  Detection of flicker frequency or suppression of flicker wherein the flicker is caused by illumination, e.g. due to fluorescent tube illumination or pulsed LED illumination · CPC title

• H04N23/56

  provided with illuminating means · CPC title

• H04N5/2256

  Electricity · mapped topic

• H04N5/2357 Primary

  Electricity · mapped topic