Human presence detection on a mobile machine

Patent metadata

Field	Value
Publication number	US-10183667-B2
Application number	US-201514854491-A
Country	US
Kind code	B2
Filing date	Sep 15, 2015
Priority date	Sep 15, 2015
Publication date	Jan 22, 2019
Grant date	Jan 22, 2019

Abstract

A mobile machine includes a set of human presence sensors that comprise one or more optical sensors, one or more thermal sensors, and an additional sensor that senses a characteristic of a human. The contribution of the various sensor values generated by the sensors is determined based on environmental conditions. A human presence metric, indicative of human presence, is generated from the contributions of the various sensor signals.

First claim

What is claimed is:

1. A mobile machine, comprising:
an optical sensor that generates an optical sensor signal indicative of an optical image of a field of regard in a forward direction, relative to a direction of travel of the mobile machine;
a thermal sensor that generates a thermal sensor signal indicative of a thermal characteristic of the field of regard in the forward direction;
a capacitive sensor that generates capacitance sensor signal indicative of sensed capacitance in the forward direction;
standing water sensor that generates a standing water sensor signal indicative of a presence of standing water in the field of regard;
an obscurant sensor that generates an obscurant sensor signal indicative of a presence of an optical obscurant in the field of regard;
an ambient temperature sensor that generates an ambient temperature sensor signal indicative of an ambient temperature proximate the field of regard;
a sensor weighting component that generates a visibility factor based on the obscurant sensor signal and applies the visibility factor to the optical sensor signal to obtain a net optical contribution of received values of the optical sensor signal, the sensor weighting component generating a net thermal contribution of received values of the thermal sensor signal based on the ambient temperature sensor signal, and the sensor weighting component generating a net capacitance contribution, to the human presence metric, for the capacitance sensor signal based on the standing water sensor signal; and
a human presence detection system that receives the net optical contribution, the net thermal contribution and the net capacitance contribution and generates a human presence metric, based on the net optical contribution, the net thermal contribution and the net capacitance contribution, the human presence metric having a metric value indicative of human presence in the field of regard, and the human presence detection system generating an action signal based on the human presence metric.

2. The mobile machine of claim 1 wherein the human presence detection system comprises:
a metric generator component that generates the human presence metric.

3. The mobile machine of claim 1 wherein the standing water sensor comprises:
a precipitation sensor that senses precipitation in the field of regard.

4. The mobile machine of claim 1 wherein the standing water sensor comprises:
a position sensor that senses a position of the mobile machine, and wherein the sensor weighting component receives weather information and map information and generates an estimate of whether standing water is present in the field of regard based on the weather information, the map information and the position of the mobile machine.

5. The mobile machine of claim 1 and further comprising:
an image processing system that performs color and structural segmentation processing on the optical sensor signal.

6. The mobile machine of claim 1 wherein the additional sensor comprises:
a firmness sensor that senses a firmness of objects proximate the mobile machine in the forward direction of the mobile machine.

7. A method, comprising:
sensing an optical image of a field of regard in a forward direction, relative to a direction of travel of a mobile machine;
sensing a thermal characteristic of the field of regard in the forward direction;
sensing a capacitive characteristic of the field of regard in the forward direction;
sensing an environmental characteristic proximate the mobile machine;
sensing a location of the mobile machine;
receiving topographical information of the field of regard;
receiving weather history information for the field of regard;
generating a standing water factor based on the topographical and weather history information;
generating a net optical contribution based on the sensed optical image and the environmental characteristics;
generating a net capacitive contribution based on the sensed capacitive characteristic, the location of the mobile machine and the standing water factor;
generating a human presence metric, based on the net optical contribution, the net capacitive contribution and the thermal characteristic the human presence metric having a metric value indicative of human presence in the field of regard; and
generating an action signal based on the human presence metric.

8. The method of claim 7 wherein generating an action signal comprises:
controlling a user interface system to surface an indication of the human presence metric for user interaction.

9. The method of claim 7 wherein the mobile machine includes a set of controlled systems and a control system that controls the set of controlled systems, and wherein generating an action signal comprises:
generating a control signal to control the controlled systems based on the human presence metric.

10. The method of claim 9 wherein generating a control signal comprises:
generating a stop signal to stop travel of the mobile machine based on the human presence metric.

11. The method of claim 9 wherein generating a control signal comprises:
generating a steering signal to change a course of the mobile machine based on the human presence metric.

12. The method of claim 9 wherein generating the control signal comprises:
generating a position change signal to change a position of a ground engaging portion of the mobile machine based on the human presence metric.

13. The method of claim 7 wherein at least one of the steps of sensing an optical image, sensing a thermal characteristic, sensing a capacitive characteristic and sensing an environmental characteristic are performed on a remote machine that is separate from the mobile machine.

14. A mobile machine, comprising:
a first set of sensors indicative of a human presence, the first set of sensors comprising:
an optical sensor that generates an optical sensor signal indicative of an optical image of a field of regard in a forward direction, relative to a direction of travel of the mobile machine;
a thermal sensor that generates a thermal sensor signal indicative of a thermal characteristic of the field of regard in the forward direction; and
a capacitive sensor that generates a capacitance sensor signal indicative of sensed capacitance in the field of regard;

a second set of sensors indicative of environmental characteristics proximate the mobile machine, the second set of sensors comprising:
an obscurant sensor that generates an obscurant sensor signal indicative of a presence of a visual obscurant proximate the mobile machine;
an ambient temperature sensor that generates an ambient temperature sensor signal indicative of an ambient temperature proximate the mobile machine; and
a standing water sensor that generates a standing water factor indicative of standing water proximate the mobile machine based on a sensed location of the mobile machine;

a sensor weighting component that determines a first weighted value corresponding to the optical sensor signal based on the obscurant sensor signal, a second weighted value corresponding to the thermal sensor signal based on the ambient temperature sensor signal, and a third weighted value corresponding to the capacitive sensor signal based on the standing water factor;
a human presence metric generator component that generates a human presence metric based on the first weighted value, the second weighted value, and the third weighted value, the human presence metric having a metric value indicative of human presence in the field of regard; and
an action signal generator component that generates an action signal based on the human pre

Assignees

• Deere & Co

Inventors

• Anderson Noel W
• Turner Christopher D
• Le Duong M
• Vollmar Uwe
• Davis Travis J

Classifications

• B60W30/09 Primary

  Taking automatic action to avoid collision, e.g. braking and steering · CPC title

• B60Y2200/22

  Agricultural vehicles · CPC title

• B60W30/0956

  the prediction being responsive to traffic or environmental parameters · CPC title

• B60W2420/40

  Photo, light or radio wave sensitive means, e.g. infrared sensors · CPC title

• B60W2554/00

  Input parameters relating to objects · CPC title