Scene illumination detection for autonomous systems and applications

Patent metadata

Field	Value
Publication number	US-12576776-B2
Application number	US-202418929469-A
Country	US
Kind code	B2
Filing date	Oct 28, 2024
Priority date	Aug 16, 2023
Publication date	Mar 17, 2026
Grant date	Mar 17, 2026

Abstract

The present disclosure relates to determining a first illumination level corresponding to an area based at least on a first illumination detection obtained using a first illumination detector corresponding to a machine. A second illumination level corresponding to the area may be determined based at least on a second illumination detection obtained using a second illumination detector corresponding to the machine. Based at least on the first illumination level and the second illumination level, a scene illumination state of the area may be determined. Based at least on the scene illumination state, one or more lights of the machine may be controlled.

First claim

What is claimed is:

1 . An autonomous or semi-autonomous machine comprising:
one or more central processing units (CPUs); one or more graphics processing units (GPUs); one or more hardware accelerators; one or more lighting components; one or more external sensors including one or more fields of view or sensory fields external to the autonomous or semi-autonomous machine; and one or more internal sensors including one or more fields of view or sensory fields internal to the autonomous or semi-autonomous machine; wherein the autonomous or semi-autonomous machine performs operations comprising:
causing control of the one or more lighting components of the autonomous or semi-autonomous machine based at least on a scene-illumination state of an environment determined using a plurality of illumination states, the plurality of illumination states being determined based at least on sensor data obtained using at least a subset of the one or more external sensors.

2 . The autonomous or semi-autonomous machine of claim 1 , wherein at least a first external sensor of the one or more external sensors is of a different type than at least a second external sensor of the one or more external sensors.

3 . The autonomous or semi-autonomous machine of claim 2 , wherein:
the first external sensor includes a wide-view camera; and the second external sensor includes a telescopic camera.

4 . The autonomous or semi-autonomous machine of claim 1 , wherein the plurality of illumination states include:
a first illumination state that is based at least on a first level of illumination, as detected using a first external sensor of the one or more external sensors, corresponding to a first position along a first illumination scale; and a second illumination state that is based at least on a second level of illumination, as detected using a second external sensor of the one or more external sensors, corresponding to a second position along a second illumination scale.

5 . The autonomous or semi-autonomous machine of claim 4 , wherein a specific value of the first illumination state is based at least on the first position corresponding to a particular threshold range of a set of threshold ranges along the first illumination scale.

6 . The autonomous or semi-autonomous machine of claim 5 , wherein the set of threshold ranges includes one or more of:
a first threshold range corresponding to a first illumination value of “illuminated”; a second threshold range corresponding to a second illumination value of “dark”; or a third threshold range between the first threshold range and the second threshold range corresponding to a third illumination value of “transitory”.

7 . The autonomous or semi-autonomous machine of claim 5 , wherein one or more threshold ranges of the set of threshold ranges varies based at least on a previous scene illumination state determination.

8 . The autonomous or semi-autonomous machine of claim 4 , wherein a determination as to a specific value of the second illumination state with respect to the second position varies depending on where the first position lies along the first illumination scale.

9 . The autonomous or semi-autonomous machine of claim 4 , wherein the second illumination state is used as part of determining the scene-illumination state in response to a determined illumination state value of the first illumination state being determined as being “transitory”.

10 . The autonomous or semi-autonomous machine of claim 1 , further comprising one or more systems on a chip (SoCs).

11 . The autonomous or semi-autonomous machine of claim 1 , wherein the one or more hardware accelerators include at least one of a vision accelerator, a deep learning accelerator (DLA), or a ray-tracing hardware accelerator.

12 . A method comprising:
determining a first illumination state corresponding to an area based at least on a first level of illumination corresponding to a first position along a first illumination scale; determining a second illumination state corresponding to the area based at least on a second level of illumination corresponding to a second position along a second illumination scale; determining a scene-illumination state of the area based at least on the first illumination state and the second illumination state; and controlling one or more lighting components of a machine based at least on the scene-illumination state.

13 . The method of claim 12 , wherein a specific value of the first illumination state is based at least on the first position corresponding to a particular threshold range of a set of threshold ranges along the first illumination scale.

14 . The method of claim 13 , wherein the set of threshold ranges includes one or more of:
a first threshold range corresponding to a first illumination value of “illuminated”; a second threshold range corresponding to a second illumination value of “dark”; or a third threshold range between the first threshold range and the second threshold range corresponding to a third illumination value of “transitory”.

15 . The method of claim 13 , wherein one or more threshold ranges of the set of threshold ranges are based at least on a previous scene illumination state determination.

16 . The method of claim 12 , wherein a determination as to a specific value of the second illumination state with respect to the second position varies depending on where the first position lies along the first illumination scale.

17 . The method of claim 12 , wherein:
the first level of illumination is determined based at least on first sensor data obtained using a first illumination sensor; and the second level of illumination is determined based at least on second sensor data obtained using a second illumination sensor.

18 . A system of a machine comprising:
one or more central processing units (CPUs); one or more graphics processing units (GPUs); one or more hardware accelerators; one or more lighting components; external sensors including fields of view or sensory fields external to the machine; and internal sensors including fields of view or sensory fields internal to the machine; wherein the system causes performance of operations comprising:
causing control of the one or more lighting components of the machine based at least on a combined scene-illumination state of an environment determined based at least on sensor data obtained using two or more external sensors of the external sensors.

19 . The system of claim 18 , wherein:
at least a first external sensor of the external sensors is of a different type than at least a second external sensor of the external sensors.

20 . The system of claim 18 , wherein the external sensors include two or more image sensors.

Assignees

• Nvidia Corp

Inventors

• Nagaraja Keerthi Raj
• Gupta Arjun
• Bajpayee Abhishek
• Temel Kivanc
• Doblar Dylan
• Chandrasekar Sai Krishnan
• Wang Yu

Classifications

• G06V10/60

  relating to illumination properties, e.g. using a reflectance or lighting model · CPC title

• G06V10/82

  using neural networks · CPC title

• G06V20/56 Primary

  exterior to a vehicle by using sensors mounted on the vehicle · CPC title

• B60Q1/143 Primary

  combined with another condition, e.g. using vehicle recognition from camera images or activation of wipers · CPC title

• G06N5/04

  Inference or reasoning models · CPC title