Machine Onboard Activity and Behavior Classification

Patent metadata

Field	Value
Publication number	US-2017138017-A1
Application number	US-201514942338-A
Country	US
Kind code	A1
Filing date	Nov 16, 2015
Priority date	Nov 16, 2015
Publication date	May 18, 2017
Grant date	—

Abstract

A machine behavior classification system may include an electronic data collection system and an electronic data analysis system. The electronic data collection system may have an image capture apparatus that is configured to monitor a machine and the image capture apparatus may produce an image data file of the machine. Moreover, the electronic data analysis system may include a data analytics module that is configured to receive the image data file and the data analytics module may perform a data analysis of the image data file.

First claim

What is claimed is:

1 . A machine behavior classification system, the classification system comprising:
an electronic data collection system; an electronic data analysis system; the electronic data collection system includes an image capture apparatus configured to monitor a machine and produce an image data file of the machine; and the electronic data analysis system has a data analytics module configured to receive the image data file and to perform a data analysis of the image data file.

2 . The classification system of claim 1 , wherein the image capture apparatus is a digital camera mounted on the machine, the image data file includes a plurality of images and the data analytics module uses the plurality of images to perform the data analysis and determine a machine behavior, a machine operating environment and a set of machine actions.

3 . The classification system of claim 2 , wherein the image data file is a video stream captured by the image capture apparatus, the electronic data collection system is coupled to a computing device configured with a storage device and the video stream is saved to the storage device.

4 . The classification system of claim 3 , wherein the electronic data collection system is communicably coupled to a first communication apparatus, the electronic data analysis system is communicably coupled to a second communication apparatus, the first communication apparatus sends the image data file to the second communication apparatus, the electronic data analysis system receives the image data file, the data analytics module performs the data analysis on the image data file using an object detection and an optical flow analysis to determine various states of the machine.

5 . The classification system of claim 4 , wherein the first communication apparatus and the second communication apparatus to create a live video stream between the electronic data collection system and the electronic data analysis system, the live video stream being viewable by a user of the electronic data collection system and the electronic data analysis system.

6 . The classification system of claim 4 , wherein the electronic data collection system is connected to a plurality of sensors located on the machine, the plurality of sensors create a sensor data file, the first communication apparatus is configured to send the sensor data file to the second communication apparatus and the data analytics module uses the sensor data file and the image data file to perform the data analysis.

7 . The classification system of claim 1 , wherein the image capture apparatus has an adjustable field of view and the field of view can be adjusted remotely by a user of the machine behavior classification system.

8 . The classification system of claim 1 , wherein the machine behavior classification system is used to optimize an operation of the machine or a fleet of machines working at a job site.

9 . A machine, comprising:
an engine; a tool movably attached to the machine; a machine behavior classification system operatively associated with the machine and includes an electronic data collection system and an electronic data analysis system; the electronic data collection system has an image capture apparatus adjustably attached to the machine, the image capture apparatus configured to monitor a tool activity and an area surrounding the machine; the image capture apparatus produces an image data file of the tool activity and the area surrounding the machine; and the electronic data analysis system having a data analytics module configured to receive the image data file and to perform a data analysis of the image data file.

10 . The machine of claim 9 , wherein the image capture apparatus is a digital camera, the image data file includes a plurality of images and the data analytics module uses the plurality of images to perform the data analysis.

11 . The machine of claim 10 , wherein the image data file is a video stream captured by the image capture apparatus, the electronic data collection system is coupled to a computing device located on the machine, the computing device configured with a storage device and the video stream is saved to the storage device.

12 . The machine of claim 11 , wherein a video monitor and a control module are located in an operator compartment of the machine, the image capture apparatus and the computing device are coupled to the video monitor and configured to display the image data file on the video monitor and the electronic data collection system is operatively coupled to the control module and the control module is used to operate the electronic data collection system.

13 . The machine of claim 11 , wherein the electronic data analysis system is located at a distance away from the machine, the electronic data collection system being communicably coupled to a first communication apparatus, the electronic data analysis system being communicably coupled to a second communication apparatus, the first communication apparatus sending the image data file to the second communication apparatus and the data analytics module performing the data analysis on the image data file, the data analysis using an object detection and optical flow analysis to determine various states of the machine.

14 . The machine of claim 13 , wherein the electronic data collection system is connected to a plurality of sensors located on the machine, the plurality of sensors create a sensor data file, the first communication apparatus is configured to send the sensor data file to the second communication apparatus and the data analytics module uses the sensor data file and the image data file to perform the data analysis.

15 . The machine of claim 12 , wherein the image capture apparatus has an adjustable field of view configured to increase or decrease the surrounding area of the machine viewed by the image capture apparatus and the adjustable field of view is controlled remotely by a user of the machine behavior classification system.

16 . The machine of claim 9 , wherein the machine behavior classification system is used to optimize an operation of the machine or a fleet of machines working at a job site.

17 . The machine of claim 16 , wherein the job site has a communications network, the fleet of machines working at the job site each having the electronic data collection system, the fleet of machines using the communications network to share a sensor data file and the image data file collected by the electronic data collection system of each machine in the fleet of machines.

18 . A method for monitoring a machine activity and behavior classification, the method comprising:
creating a machine behavior classification system, the machine behavior classification system having an electronic data collection system and an electronic data analysis system; installing the electronic data collection system on a machine located at a job site; collecting an image data file using the electronic data collection system, the electronic data collection system having an image capture apparatus configured to monitor the machine; analyzing the image data file using the electronic data analysis system, the electronic data analysis system having a data analytics module configured to receive and perform a data analysis of the image data file; and determining a machine behavior, a machine operating environment and a set of actions performed by the machine through the data analysis of the image data file.

19 . The method of claim 18 , furth

Assignees

• Caterpillar Inc

Inventors

• Halepatali Praveen
• Kim Sang Kyum
• Han Meng
• Finch Nolan S
• Agarwal Vineet
• Moloney Neil

Classifications

• E02F3/32

  working downwardly and towards the machine, e.g. with backhoes · CPC title

• E02F9/261

  Surveying the work-site to be treated · CPC title

• B60R1/00

  Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles · CPC title

• G06K9/00718

  Physics · mapped topic

• E02F9/26 Primary

  Indicating devices {(E02F5/145 takes precedence)} · CPC title