Inward/outward vehicle monitoring for remote reporting and in-cab warning enhancements

Patent metadata

Field	Value
Publication number	US-11993277-B2
Application number	US-202318124351-A
Country	US
Kind code	B2
Filing date	Mar 21, 2023
Priority date	Sep 11, 2018
Publication date	May 28, 2024
Grant date	May 28, 2024

Abstract

Systems and methods are provided for intelligent driving monitoring systems, advanced driver assistance systems and autonomous driving systems, and providing alerts to the driver of a vehicle, based on anomalies detected between driver behavior and environment captured by the outward facing camera. Various aspects of the driver, which may include his direction of sight, point of focus, posture, gaze, is determined by image processing of the upper visible body of the driver, by a driver facing camera in the vehicle. Other aspects of environment around the vehicle captured by the multitude of cameras in the vehicle are used to correlate driver behavior and actions with what is happening outside to detect and warn on anomalies, prevent accidents, provide feedback to the driver, and in general provide a safer driver experience.

First claim

What is claimed is:

1. A method comprising:
capturing, by at least one processor of a computing device with an outward facing camera on or in a vehicle, visual data outside of the vehicle;
detecting, by the at least one processor based on the visual data, a red light turned to a green light for the vehicle;
responsive to detecting, by the at least one processor, that the vehicle has not moved for at least a threshold period of time since detecting the red light turned to the green light, generating, by the at least one processor, an alert for a driver of the vehicle; and
responsive to detecting, by the at least one processor, a sound of a horn after detecting the red light turned to the green light, extending, by the at least one processor, the threshold period of time.

2. The method of claim 1 , further comprising responsive to detecting, by the at least one processor, a sound of a siren after detecting the red light turned to the green light, suppressing, by the at least one processor, the alert to the driver of the vehicle.

3. A method comprising:
capturing, by at least one processor of a computing device with an outward facing camera on or in a vehicle, visual data outside of the vehicle;
detecting, by the at least one processor based on the visual data, a red light turned to a green light for the vehicle, wherein detecting that a driver of the vehicle is not looking at the green light comprises detecting, using a microphone, sound inside of the vehicle; and
responsive to detecting, by the at least one processor, that the driver of the vehicle is not looking at the green light, generating, by the at least one processor, an alert for a driver of the vehicle.

4. The method of claim 3 , wherein detecting that the driver of the vehicle is not looking at the green light comprises detecting that the driver is looking into the cab of the vehicle, shoulder checking, eating, interacting with a dashboard of the vehicle, using a phone, gazing at a phone, texting, talking to a passenger of the vehicle, sleeping, appearing drowsy, or handling children.

5. The method of claim 3 , wherein the microphone is inside of the vehicle.

6. The method of claim 3 , wherein the microphone is outside of the vehicle.

7. The method of claim 6 , wherein sound outside of the vehicle comprises a siren, a car screeching, or a horn honking.

8. The method of claim 7 , further comprising responsive to detecting, by the at least one processor, the horn honking after detecting the red light turned to the green light, transmitting, by the at least one processor, the alert after a period of delay.

9. The method of claim 3 , further comprising responsive to detecting, by the at least one processor, the siren after detecting the red light turned to the green light, suppressing, by the at least one processor, the alert to the driver of the vehicle.

10. A method comprising:
capturing, by at least one processor of a computing device with an outward facing camera on or in a vehicle, visual data outside of the vehicle;
detecting, by the at least one processor based on the visual data, a traffic light corresponding to a lane of the vehicle;
detecting, by the at least one processor based on the visual data, a red light turned to a green light for the traffic light corresponding to the lane of the vehicle;
generating, by the at least one processor, an alert for a driver of the vehicle when the traffic light corresponding to the lane of the vehicle turned to the green light;
detecting, by the at least one processor based on the visual data, a second traffic light corresponding to an adjacent lane of the vehicle;
detecting, by the at least one processor based on the visual data, a second red light turned to a second green light for the second traffic light corresponding to the adjacent lane; and
suppressing, by the at least one processor, the alert for the driver of the vehicle when the second traffic light corresponding to the adjacent lane of the vehicle turned to the green light.

11. The method of claim 10 , further comprising responsive to detecting, by the at least one processor, that the vehicle has not moved for at least a threshold period of time since detecting the red light turned to the green light, transmitting, by the at least one processor, an alert to a driver of the vehicle.

12. The method of claim 11 , further comprising responsive to detecting, by the at least one processor, a sound of a horn after detecting the red light turned to the green light, extending, by the at least one processor, the threshold period of time.

13. A method comprising:
capturing, by at least one processor of a computing device with an outward facing camera on or in a vehicle, visual data outside of the vehicle;
detecting, by the at least one processor based on the visual data, a traffic light corresponding to a lane of the vehicle;
detecting, by the at least one processor based on the visual data, a red light turned to a green light for the traffic light corresponding to the lane of the vehicle;
generating, by the at least one processor, an alert for a driver of the vehicle when the traffic light corresponding to the lane of the vehicle turned to the green light; and
responsive to detecting, by the at least one processor, a sound of a siren after detecting the red light turned to the green light, suppressing, by the at least one processor, the alert to the driver of the vehicle.

14. The method of claim 10 , further comprising detecting, by the at least one processor, that the driver of the vehicle is not looking at the green light.

15. The method of claim 14 , wherein detecting that the driver of the vehicle is not looking at the green light comprises detecting that the driver is looking into the cab of the vehicle, shoulder checking, eating, interacting with a dashboard of the vehicle, using a phone, gazing at a phone, texting, talking to a passenger of the vehicle, sleeping, appearing drowsy, or handling children.

16. The method of claim 14 , wherein detecting that the driver of the vehicle is not looking at the green light comprises detecting, using a microphone inside of the vehicle, sound inside of the vehicle.

17. The method of claim 14 , wherein detecting that the driver of the vehicle is not looking at the green light comprises detecting, using a microphone outside of the vehicle, sound outside of the vehicle.

18. The method of claim 17 , wherein sound outside of the vehicle comprises a siren, a car screeching, or a horn honking.

Assignees

• Netradyne Inc

Inventors

• Julian David Jonathan
• Annapureddy Venkata Sreekanta Reddy
• Pandya Sandeep
• Yedla Arvind
• Willis Dale Alan
• Venkatachalam Jayaraman Venkata Ramanan
• Yanamala Suresh Babu
• Agrawal Avneesh
• Taranalli Veeresh
• Kim Jaeyoon

Classifications

• G08B21/06

  indicating a condition of sleep, e.g. anti-dozing alarms · CPC title

• B60W50/14 Primary

  Means for informing the driver, warning the driver or prompting a driver intervention · CPC title

• B60W40/02

  related to ambient conditions · CPC title

• B60W40/08

  related to drivers or passengers · CPC title

• G06V20/58 Primary

  Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads · CPC title