Tracking system and method employing an artificial neural network for tracking tools in and nearby a transportation vehicle

Patent metadata

Field	Value
Publication number	US-11935280-B2
Application number	US-202217649290-A
Country	US
Kind code	B2
Filing date	Jan 28, 2022
Priority date	Jan 29, 2021
Publication date	Mar 19, 2024
Grant date	Mar 19, 2024

Abstract

A tracking system and method for tracking tools in and nearby a transportation vehicle is provided. The tracking system comprises a vehicle-based detection unit for optically acquiring tool sets in the vehicle loading space from different angles and providing digital image data as well as corresponding range information. An electronic main controller unit is operatively coupled to a communication receiving unit and communicates with the vehicle-based detection unit and a cloud-based computer system of the tracking system. A mobile computing unit includes an optical camera and a LIDAR sensor device and wirelessly communicates with the electronic main controller unit and the cloud-based computer system. The tool detection and tracking is accomplished by combining an image-based detection employing an artificial neural network in the cloud-based computer system and a signal-based detection employing the short-range wireless network communication means.

First claim

The invention claimed is:

1. A tracking system for tracking tools in and nearby a transportation vehicle, the system including:
a plurality of tool sets that is arrangeable in a vehicle loading space of the transportation vehicle at least during transportation, wherein each tool set includes a tool and at least one fixedly attached short-range wireless network communication member;
a short-range wireless network communication receiving unit that is arrangeable in the vehicle loading space and that is configured to receive signals transmitted by and unique to each short-range wireless network communication member;
a vehicle-based detection unit having one or more optical cameras and a LIDAR sensor device that are each separate from the tool sets and arrangeable in the vehicle loading space for optically acquiring at least a portion of the vehicle loading space and at least a portion of the plurality of tool sets at least in the vehicle loading space from different angles, wherein the one or more optical cameras and the LIDAR sensor device are configured for providing digital image data as well as corresponding range information for the at least a portion of the plurality of tool sets;
an electronic main controller unit, which is operatively coupled at least to the wireless network communication receiving unit, which is configured to communicate with the vehicle-based detection unit, and which is configured to communicate with a cloud-based computer system;
a mobile computing unit including an optical camera, a LIDAR sensor device, and configured to wirelessly communicate with the electronic main controller unit and a cloud-based computer system; and
the cloud-based computer system comprising at least one trained artificial neural network that at least serves as a basis for providing a prediction output regarding identity and location of tool sets of the plurality of tool sets, based on digital image data and corresponding range information provided by the vehicle-based detection unit.

2. The tracking system as claimed in claim 1 , wherein each of the optical cameras and the LIDAR sensor device of the vehicle-based detection unit are arrangeable in the vehicle loading space such that a respective field of view comprises at least a portion of the vehicle loading space.

3. The tracking system as claimed in claim 1 , further comprising an image processing unit that is operatively coupled to the vehicle-based detection unit and to the mobile computing unit, and that is configured for calibrating and rectifying sets of digital image data and their corresponding range information, and for creating merged data sets, each merged data set comprising calibrated and rectified sets of digital image data and their corresponding range information.

4. The tracking system as claimed in claim 1 , wherein at least one of the one or more optical cameras and a LIDAR sensor device of the vehicle-based detection unit and the optical camera of the mobile computing unit is configured for operating at visible light wavelengths as well as at near-infrared wavelengths.

5. The tracking system as claimed in claim 1 , wherein the short-range wireless network communication members are designed as Bluetooth beacons, and the signal transmitted by each short-range wireless network communication member comprises a unique identifier.

6. The tracking system as claimed in claim 1 , wherein the mobile computing unit includes a display unit that is configured for visualizing tracked tool sets and their respective location.

7. The tracking system as claimed in claim 1 , wherein the electronic main controller unit includes a data interface that is connectable to a vehicle-internal data communications interface, and wherein the electronic main controller unit is configured to receive vehicle-related data via the data communications interface.

8. The tracking system as claimed in claim 1 , wherein the at least one trained artificial neural network comprises at least one deep neural network.

Assignees

• Ford Global Tech Llc

Inventors

• Aslandere Turgay Isik
• Grein Marcel

Classifications

• G06V10/809 Primary

  of classification results, e.g. where the classifiers operate on the same input data · CPC title

• G06T7/70

  Determining position or orientation of objects or cameras (camera calibration G06T7/80) · CPC title

• G06V20/59

  inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions · CPC title

• H04W4/025

  using location based information parameters · CPC title

• H04W4/38

  for collecting sensor information · CPC title