System and method of depth sensor activation

Patent metadata

Field	Value
Publication number	US-10474145-B2
Application number	US-201615346000-A
Country	US
Kind code	B2
Filing date	Nov 8, 2016
Priority date	Nov 8, 2016
Publication date	Nov 12, 2019
Grant date	Nov 12, 2019

Abstract

An apparatus includes a first sensor configured to generate first sensor data. The first sensor data is related to an occupant of a vehicle. The apparatus further includes a depth sensor and a processor. The depth sensor is configured to generate data corresponding to a volume associated with at least a portion of the occupant. The processor is configured to receive the first sensor data and to activate the depth sensor based on the first sensor data.

First claim

What is claimed is:

1. An apparatus comprising:
a first sensor configured to generate sensor data related to an occupant of a vehicle;
a depth sensor configured to generate depth data corresponding to a volume associated with at least a portion of the occupant; and
a processor configured to:
receive the sensor data;
activate the depth sensor, conditioned on identifying a state of the occupant based on the sensor data, to generate the depth data; and
initiate a remedial action conditioned on verifying the identified state.

2. The apparatus of claim 1 , wherein the processor is configured to reduce exposure of the occupant to output from the depth sensor as compared to a device in which an active depth sensor is constantly activated.

3. The apparatus of claim 1 , wherein the first sensor comprises an image sensor.

4. The apparatus of claim 1 , wherein the first sensor includes an audio sensor.

5. The apparatus of claim 1 , wherein the remedial action includes an alert responsive to a determination, based on the sensor data and the depth data, that the state corresponds to one or more defined states.

6. The apparatus of claim 5 , wherein the depth sensor includes a laser device, and wherein the alert is provided via a display device or an audio device.

7. The apparatus of claim 1 , wherein the depth sensor is configured to:
receive an activation signal from the processor, the activation signal configured to activate the depth sensor to project energy and generate the depth data; and
upon activation of the depth sensor, target a seat of the vehicle or a head of the occupant with the energy.

8. The apparatus of claim 5 , wherein the alert is provided via haptic feedback.

9. The apparatus of claim 1 , wherein the sensor data comprises image data corresponding to an image of the occupant, and wherein the processor is configured to classify the state by determining, based on the image data, a head pose of the occupant, features of a face of the occupant, or a combination thereof.

10. The apparatus of claim 1 , wherein the sensor data comprises light level data, and wherein the processor is configured to activate the depth sensor in response to the light level data satisfying a threshold.

11. The apparatus of claim 1 , wherein the first sensor, the depth sensor, and the processor are integrated into the vehicle.

12. A method of activating an active depth sensor, the method comprising:
receiving, at a processor, sensor data from a first sensor, the sensor data related to an occupant of a vehicle;
conditioned on detecting a compromised or hazardous state of the occupant based on the sensor data, sending an activation signal from the processor to the active depth sensor to cause the active depth sensor to generate depth data; and
receiving, at the processor, the depth data from the active depth sensor, the depth data corresponding to a volume associated with at least a portion of the occupant.

13. The method of claim 12 , wherein the processor is configured to reduce exposure of the occupant to output from the active depth sensor as compared to a device in which a depth sensor is constantly activated, and further comprising determining, at the processor, a classification of the state based on the depth data and the sensor data.

14. The method of claim 12 , further comprising, in response to determining that the state corresponds to one or more defined states, outputting an audio alert via a speaker, outputting a visual alert via a display, activating an automated driving system, deactivating an engine, deactivating the vehicle, or a combination thereof.

15. The method of claim 12 , further comprising:
determining, at the processor, whether the state corresponds to one or more defined states based on the sensor data; and
determining, at the processor, to activate the active depth sensor in response to determining that the state corresponds to the one or more defined states for a period of time that satisfies a threshold.

16. The method of claim 15 , wherein the one or more defined states are associated with distraction of the occupant, sleeping of the occupant, impairment of the occupant, or a combination thereof.

17. The method of claim 15 , wherein the sensor data comprises image data corresponding to an image of the occupant, and wherein determining the state includes determining a head pose of the occupant based on the image data.

18. The method of claim 12 , further comprising:
determining, based on the sensor data, whether a light level associated with the volume satisfies a threshold; and
determining to activate the active depth sensor in response to the light level satisfying the threshold.

19. A computer-readable storage device storing instructions that, when executed by a processor, cause the processor to:
receive sensor data from a first sensor, the sensor data related to an occupant of a vehicle;
activate a depth sensor, conditioned on identifying a state of the occupant based on the sensor data, to generate depth data; and
receive the depth data from the depth sensor, the depth data corresponding to a volume associated with at least a portion of the occupant.

20. The computer-readable storage device of claim 19 , wherein:
the instructions are configured to reduce exposure of the occupant to output from the depth sensor as compared to an active depth sensor that is constantly activated, and
the instructions, when executed by the processor, further cause the processor to determine whether to classify the state as one or more defined states based on the depth data and the sensor data.

21. The computer-readable storage device of claim 20 , wherein determining whether to classify the state as the one or more defined states includes combining two dimensional image data from the sensor data with the depth data to determine three dimensional data corresponding to the occupant.

22. The computer-readable storage device of claim 21 , wherein combining the two dimensional image data with the depth data includes generating a three dimensional head pose of the occupant based on the two dimensional image data and the depth data.

23. The computer-readable storage device of claim 22 , wherein the instructions, when executed by the processor, further cause the processor to classify the state as the one or more defined states in response to detecting that the three dimensional head pose indicates the one or more defined states for a duration of time that satisfies a threshold.

24. The computer-readable storage device of claim 23 , wherein the instructions, when executed by the processor, further cause the processor to detect that the three dimensional head pose indicates the one or more defined states in response to the three dimensional head pose indicating that the occupant is focused away from a direction of operation of the vehicle.

25. The computer-readable storage device of claim 20 , wherein the instructions, when executed by the processor, further cause the processor to:
identify passing of a period of time since activation of the depth sensor; and
deactivate the depth sensor in response to the period of time satisfying a threshold.

26. The computer-readable storage device of claim 20 , wherein the instructions, when executed by the processor, further cause the processor to, in response to determining to classify the state as the one or more defined states, initiate a remedial act

Assignees

• Qualcomm Inc

Inventors

• Guo Feng
• Qi Yingyong
• Bi Ning
• Jiang Bolan
• Chang Chienchung

Classifications

• B60W2900/00

  Indexing codes relating to the purpose of, or problem solved of road vehicle drive control systems not otherwise provided for in groups B60W30/00 · CPC title

• B60W10/06

  including control of combustion engines · CPC title

• B60W2540/26

  Incapacity · CPC title

• B60Q9/00

  Arrangement or adaptation of signal devices not provided for in one of main groups B60Q1/00 - B60Q7/00 {, e.g. haptic signalling} · CPC title

• B60W2040/0818

  Inactivity or incapacity of driver · CPC title