Autonomous Vision System for Monitoring Cargo

Patent metadata

Field	Value
Publication number	US-2024242514-A1
Application number	US-202318484063-A
Country	US
Kind code	A1
Filing date	Oct 10, 2023
Priority date	Jan 18, 2023
Publication date	Jul 18, 2024
Grant date	—

Abstract

Devices and methods for monitoring cargo that is loaded onto a vehicle. The computing device includes memory circuitry and processing circuitry configured to operate according to programming instructions stored in the memory circuitry to: receive images of the cargo; identify the cargo within the images; based on the identification, determine one or more aspects the cargo including the volume of the cargo; and determine a position on the vehicle where the cargo is loaded based on the one or more aspects.

First claim

What is claimed is:

1 . A computing device configured to monitor cargo that is being loaded onto a vehicle, the computing device comprising:
memory circuitry; processing circuitry configured to operate according to programming instructions stored in the memory circuitry to:
receive images of the cargo from at least one electro-optical sensor affixed to the vehicle;
identify the cargo within the images;
compute one or more aspects of the cargo including a volume of the cargo;
determine a position on the vehicle where the cargo is loaded based on the images of the cargo; and
based on the loaded position on the vehicle, associate one or more aspects with the cargo including the volume of the cargo.

2 . The computing device of claim 1 , wherein the processing circuitry is further configured to identify the cargo based on at least one of the images.

3 . The computing device of claim 1 , wherein the processing circuitry is configured to:
determine a temporary position of the cargo within an alignment area within the vehicle; determine a lane within the vehicle that the cargo enters after being positioned at the alignment area; and determine a location along the lane where the cargo is finally positioned.

4 . The computing device of claim 1 , further comprising communication circuitry configured to upload through a communication network the position of the cargo on the vehicle while onboard the vehicle.

5 . The computing device of claim 1 , wherein the processing circuitry is configured to:
use semantic segmentation to identify a first set of pixels in the images as the cargo and identify a different second set of pixels as another object through semantic segmentation; and determine a confidence value of an identification of the cargo.

6 . The computing device of claim 1 , wherein the processing circuitry is further configured to:
identify the cargo within the images with a bounding box that encapsulates each instance of the cargo; and determine a confidence value of an identification of the cargo.

7 . The computing device of claim 1 , wherein the processing circuitry is configured to determine a point on the cargo based on the images and track the position of the cargo within the vehicle based on the point.

8 . The computing device of claim 1 , wherein the processing circuitry is further configured to:
receive the images from a primary camera and one or more subordinate cameras; select the images from the primary camera that capture the cargo; select the images from the one or more subordinate cameras that were taken at the same time as the images that are selected from the primary camera; and identifying the cargo based on the selected images.

9 . The computing device of claim 8 , wherein the processing circuitry is further configured to discard the images that were not selected from the primary camera and the one or more subordinate cameras.

10 . The computing device of claim 1 , wherein the processing circuitry is further configured to calculate one of a 3D mesh or a 3D point cloud of the cargo based on the images and determine the volume of the cargo using the 3D mesh or the 3D point cloud.

11 . A computing device configured to monitor cargo that is being loaded onto a vehicle 100 , the computing device comprises:
memory circuitry; processing circuitry configured to operate according to programing instructions stored in the memory circuitry to:
receive images of the cargo from a plurality of electro-optical sensors;
analyze the images and identify the cargo;
determine a point on the cargo based on the images; and
determine a position on the vehicle where the cargo is loaded during transport by the vehicle.

12 . The computing device of claim 11 , wherein the processing circuitry is further configured to determine a shape of the cargo, and dimensions of the cargo.

13 . The computing device of claim 11 , wherein the processing circuitry is further configured to select a limited number of the images received from the plurality of electro-optical sensors and identify the cargo based on the limited number of images.

14 . The computing device of claim 11 , wherein the processing circuitry is further configured to:
receive the images from a primary camera and one or more subordinate cameras; select the images from the primary camera that capture the cargo; select the images from the one or more subordinate cameras that were taken at the same time as the images that are selected from the primary camera; and analyze and identify the cargo based on just the selected images.

15 . The computing device of claim 14 , wherein the processing circuitry is further configured to discard the images that were not selected from the primary camera and the one or more subordinate cameras.

16 . A method of monitoring cargo that is being loaded onto a vehicle, the method comprising:
receiving images of the cargo from a plurality of electro-optical sensors; determining a working set of the images comprising a limited number of the images; identifying the cargo within the working set of the images; determining a point on the cargo based on the working set of the images; determining a volume of the cargo based on the working set of the images; and determining a position on the vehicle where the cargo is loaded during transport by the vehicle.

17 . The method of claim 16 , further comprising discarding the images that do not capture the cargo.

18 . The method of claim 16 , further comprising:
selecting the images from a primary camera that capture the cargo; selecting the images from one or more subordinate cameras that were taken at the same time as the images that are selected from the primary camera; and creating the working set from the selected images.

19 . The method of claim 16 , further comprising transmitting to a remote node the position of the cargo on the vehicle with the transmitting occurring while the cargo is loaded on the vehicle.

20 . The method of claim 16 , further comprising determining the one or more aspects of the cargo based on a limited number of the images that are received.

Assignees

• Boeing Co

Inventors

• Evans Nick Shadbeh
• Callahan Kevin S
• Ashok Veda
• Ayyagari Arun
• Beajow Gabriel Fayez

Classifications

• G06T7/62 Primary

  of area, perimeter, diameter or volume · CPC title

• G06Q10/083

  Shipping · CPC title

• G06V2201/12

  Acquisition of 3D measurements of objects · CPC title

• G06V20/64

  Three-dimensional [3D] objects · CPC title

• G06V20/52 Primary

  Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69) · CPC title