Monitoring ambient light for object detection

Patent metadata

Field	Value
Publication number	US-11594129-B2
Application number	US-202017067585-A
Country	US
Kind code	B2
Filing date	Oct 9, 2020
Priority date	May 2, 2018
Publication date	Feb 28, 2023
Grant date	Feb 28, 2023

Abstract

In one embodiment, a method includes receiving an image of an object captured in a geographic location. The method includes determining the geographic location associated with the image. The geographic location is represented in a map that includes one or more ambient light measurements corresponding to one or more geographic locations. The method includes using the one or more ambient light measurements corresponding to the geographic location in the map associated with the image to generate a color corrected image. The method includes determining a classification of the object using the color corrected image.

First claim

What is claimed is:

1. A method comprising, by one or more computing system:
receiving an image of an object captured in a geographic location;
determining the geographic location associated with the image, wherein the geographic location is represented in a map that includes ambient light measurements that were previously acquired corresponding to geographic locations;
using the one or more ambient light measurements corresponding to the geographic location in the map associated with the image to generate a color corrected image;
determining a classification of the object using the color corrected image;
receiving additional ambient light measurements associated with the geographic location from one or more ambient light sensors; and
updating the map based on the additional ambient light measurements associated with the geographic location.

2. The method of claim 1 ,
wherein using the one or more additional ambient light measurements corresponding to the geographic location in the map to generate the color corrected image comprises comparing the received additional ambient light measurements to the one or more ambient light measurements corresponding to the geographic location in the map.

3. The method of claim 1 , further comprising:
determining current ambient light conditions for the geographic location; and
sending a notification in response to a comparison of the current ambient light conditions to the ambient light measurements corresponding to geographic location in the map.

4. The method of claim 1 , wherein the image of the object is received from an optical sensor of a vehicle.

5. The method of claim 4 , further comprising:
using the one or more ambient light measurements corresponding to the geographic location in the map associated with the image to supplement or replace values from the optical sensor of the vehicle.

6. The method of claim 4 , wherein one or more computing systems are associated with the vehicle, and the method further comprises:
providing instructions to cause the vehicle to perform a vehicle action in response to the classification of the object.

7. The method of claim 6 , wherein the vehicle action comprises performing at least one of:
activating a safety feature of the vehicle;
adjusting a display setting of a display device within the vehicle; or
generating an alert associated with the classification of the object.

8. The method of claim 4 , wherein the map is at least one of a spatial map or a temporal map.

9. The method of claim 1 , wherein the map further comprises contextual parameters associated with the one or more ambient light measurements corresponding to the geographic locations associated with the image, the contextual parameters including at least one of a time of day of the ambient light measurements or a temperature associated with the ambient light measurements.

10. The method of claim 1 , wherein using the one or more ambient light measurements corresponding to the geographic location in the map associated with the image to generate the color corrected image and determining the classification of the object using the color corrected image are performed by a machine-learning model.

11. The method of claim 10 , wherein the machine-learning model is trained based on training samples, each of the training samples comprising:
a raw image of a sample visual scene including a sample object, wherein perceived color information of the sample object is a result of a true color of the sample object and ambient light reflected or bounced off from the sample object;
ambient light measurements for the sample visual scene; and
a corrected image of the sample visual scene, the corrected image representing the true color of the sample object.

12. The method of claim 1 , further comprising:
determining perceived color information associated with the object based on at least the image of the object; and
wherein generating the color corrected image comprises modifying the perceived color information based on the one or more ambient light measurements corresponding to the geographic location in the map.

13. A system comprising: one or more processors and one or more computer-readable non-transitory storage media coupled in communication with the one or more of the processors, the one or more computer-readable non-transitory storage media comprising instructions that, when executed by the one or more of the processors, are configured to cause the system to perform operations comprising:
receiving an image of an object captured in a geographic location;
determining the geographic location associated with the image, wherein the geographic location is represented in a map that includes ambient light measurements that were previously acquired corresponding to geographic locations;
using the one or more ambient light measurements corresponding to the geographic location in the map associated with the image to generate a color corrected image;
determining a classification of the object using the color corrected image;
receiving additional ambient light measurements associated with the geographic location from one or more ambient light sensors; and
updating the map based on the additional ambient light measurements associated with the geographic location.

14. The system of claim 13 ,
wherein using the one or more additional ambient light measurements corresponding to the geographic location in the map to generate the color corrected image comprises comparing the received additional ambient light measurements to the one or more ambient light measurements corresponding to the geographic location in the map.

15. The system of claim 13 , wherein the instructions, when executed by the one or more of the processors, are configured to cause the system to perform operations further comprising:
determining current ambient light conditions for the geographic location; and
sending a notification in response to a comparison of the current ambient light conditions to the ambient light measurements corresponding to geographic location in the map.

16. One or more computer-readable non-transitory storage media including instructions that, when executed by one or more processors, are configured to cause the one or more processors to perform operations comprising:
receiving an image of an object captured in a geographic location;
determining the geographic location associated with the image, wherein the geographic location is represented in a map that includes ambient light measurements that were previously acquired corresponding to geographic locations;
using the one or more ambient light measurements corresponding to the geographic location in the map associated with the image to generate a color corrected image;
determining a classification of the object using the color corrected image;
receiving additional ambient light measurements associated with the geographic location from one or more ambient light sensors; and
updating the map based on the additional ambient light measurements associated with the geographic location.

17. The one or more computer-readable non-transitory storage media of claim 16 ,
wherein using the one or more additional ambient light measurements corresponding to the geographic location in the map to generate the color corrected image comprises comparing the received additional ambient light measurements to the one or more ambient light measurements corresponding to the geographic location in the map.

Assignees

• Woven Planet North America Inc
• Woven Planet North Amrrica Inc

Inventors

• Abari Farzad Cyrus Foroughi
• Bangs Corey Frederick

Classifications

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06V20/582

  of traffic signs · CPC title

• G06N5/025

  Extracting rules from data · CPC title

• G08G1/166 Primary

  for active traffic, e.g. moving vehicles, pedestrians, bikes · CPC title