Primary preview region and gaze based driver distraction detection

Patent metadata

Field	Value
Publication number	US-11977675-B2
Application number	US-202117226853-A
Country	US
Kind code	B2
Filing date	Apr 9, 2021
Priority date	Jan 29, 2018
Publication date	May 7, 2024
Grant date	May 7, 2024

Abstract

A computer-implemented method of detecting distracted driving comprises: determining, by one or more processors, a primary preview region (PPR) in a representation of an environment; determining, by the one or more processors, a gaze point for a driver based on a sequence of images of the driver; determining, by the one or more processors, that the gaze point is outside of the PPR; based on the determined gaze point being outside of the PPR, decreasing, by the one or more processors, an attention level for the PPR; based on the attention level for the PPR, generating, by the one or more processors, an alert.

First claim

What is claimed is:

1. A method of transferring control of an automated vehicle to a human driver comprising:
receiving, by one or more processors, a request for transferring control of an automated vehicle to a human driver;
identifying, by the one or more processors, a primary preview point (PPP) for an object depicted in a representation of an environment, a location of the PPP within a bounding box of the object depiction being based on a velocity of the object, such that a PPP for a stationary object is located closer to the center of a bounding box of the object depiction and a PPP for a fast-moving object is located closer to an edge of the bounding box;
determining, by the one or more processors, based on the PPP, a primary preview region (PPR) for the object;
determining, by the one or more processors, that the human driver is attentive to the PPR; and
transferring, by the one or more processors, control of the automated vehicle to the human driver based on the human driver being attentive to the PPR.

2. The method of claim 1 , further comprising:
determining a gaze point for the human driver based on a sequence of images of the human driver;
wherein the determining that the human driver is attentive to the PPR is based on the gaze point being inside of the PPR.

3. The method of claim 1 , further comprising:
determining a gaze point for the human driver based on a sequence of images of the human
determining that the gaze point is outside of the PPR;
based on the gaze point being outside of the PPR, decreasing an attention level for the PPR; and
based on the attention level for the PPR, generating an alert.

4. The method of claim 1 , wherein:
the determining that the human driver is attentive to the PPR is part of determining that the human driver is attentive to each of a plurality of PPRs; and
the transferring of the control of the automated vehicle to the human driver is further based on the human driver being attentive to each other PPR of the plurality of PPRs.

5. The method of claim 4 , further comprising:
determining a priority score for each PPR of the plurality of PPRs;
wherein an attention level for each PPR of the plurality of PPRs is based on the priority score for said PPR.

6. The method of claim 1 , wherein the transferring of the control of the automated vehicle to the human driver comprises disengaging automatic control of the automated vehicle.

7. The method of claim 1 , wherein the transferring of the control of the automated vehicle to the human driver comprises providing an alert to the human driver indicating that manual control has been engaged.

8. The method of claim 1 , wherein the receiving of the request to transfer control comprises detecting a button press on a steering wheel of the automated vehicle.

9. The method of claim 1 , wherein the receiving of the request to transfer control comprises detecting a tap on a brake pedal of the automated vehicle.

10. The method of claim 1 , wherein the receiving of the request to transfer control comprises detecting a voice command.

11. A system for transferring control of an automated vehicle to a human driver comprising:
one or more processors; and
memory that stores instructions that, when executed by the one or more processors, cause the one or more processors to perform:
receiving a request for transferring control of an automated vehicle to a human driver;
identifying a primary preview point (PPP) for an object depicted in a representation of an environment, a location of the PPP within a bounding box of the object depiction being based on a velocity of the object, such that a PPP for a stationary object is located closer to the center of a bounding box of the object depiction and a PPP for a fast-moving object is located closer to an edge of the bounding box;
determining, based on the PPP, a primary preview region (PPR) for the object;
determining that the human driver is attentive to the PPR; and
transferring control of the automated vehicle to the human driver based on the human driver being attentive to the PPR.

12. The system of claim 11 , wherein the instructions further cause the one or more processors to perform:
determining a gaze point for the human driver based on a sequence of images of the human driver;
wherein the determining that the human driver is attentive to the PPR is based on the gaze point being inside of the PPR.

13. The system of claim 11 , wherein the instructions further cause the one or more processors to perform:
determining a gaze point for the human driver based on a sequence of images of the human
determining that the gaze point is outside of the PPR;
based on the gaze point being outside of the PPR, decreasing an attention level for the PPR; and
based on the attention level for the PPR, generating an alert.

14. The system of claim 11 , wherein:
the determining that the human driver is attentive to the PPR is part of determining that the human driver is attentive to each of a plurality of PPRs; and
the transferring of the control of the automated vehicle to the human driver is further based on the human driver being attentive to each other PPR of the plurality of PPRs.

15. The system of claim 14 , wherein the instructions further cause the one or more processors to perform:
determining a priority score for each PPR of the plurality of PPRs;
wherein an attention level for each PPR of the plurality of PPRs is based on the priority score for said PPR.

16. A non-transitory computer-readable medium storing computer instructions for detecting distracted driving, that when executed by one or more processors, cause the one or more processors to perform steps of:
receiving a request for transferring control of an automated vehicle to a human driver;
identifying a primary preview point (PPP) for an object depicted in a representation of an environment, a location of the PPP within a bounding box of the object depiction being based on a velocity of the object, such that a PPP for a stationary object is located closer to the center of a bounding box of the object depiction and a PPP for a fast-moving object is located closer to an edge of the bounding box;
determining, based on the PPP, a primary preview region (PPR) for the object;
determining that the human driver is attentive to the PPR; and
transferring control of the automated vehicle to the human driver based on the human driver being attentive to the PPR.

17. The non-transitory computer-readable medium of claim 16 , wherein the steps further comprise:
determining a gaze point for the human driver based on a sequence of images of the human driver;
wherein the determining that the human driver is attentive to the PPR is based on the gaze point being inside of the PPR.

18. The non-transitory computer-readable medium of claim 16 , wherein the steps further comprise:
determining a gaze point for the human driver based on a sequence of images of the human driver;
determining that the gaze point is outside of the PPR;
based on the gaze point being outside of the PPR, decreasing an attention level for the PPR; and
based on the attention level for the PPR, generating an alert.

19. The non-transitory computer-readable medium of claim 16 , wherein:
the determining that the human driver is attentive to the PPR is part of determining that the human driver is attentive to each of a plurality of PPRs; and
the transferring of the control of the automated vehicle to the human driver is further based on the human driver being at

Assignees

• Futurewei Technologies Inc

Inventors

• Yu Hai
• Porikli Fatih
• Wu Yuzhu

Classifications

• H04N23/20

  for generating image signals from infrared radiation only · CPC title

• H04N5/33

  Transforming infrared radiation (cameras or camera modules for generating image signals from infrared radiation H04N23/20; circuitry of SSIS for transforming infrared radiation into image signals H04N25/20) · CPC title

• G06V10/70

  using pattern recognition or machine learning (optical pattern recognition or electronic computations therefor G06V10/88) · CPC title

• B60W60/0015

  specially adapted for safety · CPC title

• G06F3/013 Primary

  Eye tracking input arrangements (G06F3/015 takes precedence) · CPC title