Tracking vehicles in a warehouse environment

Patent metadata

Field	Value
Publication number	US-10242273-B1
Application number	US-201815993343-A
Country	US
Kind code	B1
Filing date	May 30, 2018
Priority date	May 30, 2018
Publication date	Mar 26, 2019
Grant date	Mar 26, 2019

Abstract

This specification generally discloses technology for tracking vehicle positions in a warehouse environment. A system receives stereoscopic image data from a camera on a forklift, in some implementations. The system recognizes an object that is represented in the stereoscopic image data, identifies a representation of the recognized object in a spatial model that identifies, for each of a plurality of objects in an environment, a corresponding location of the object in the environment, determines the location of the recognized object in the environment, determines a relative position between the forklift and the recognized object, based on a portion of the received stereoscopic image data that represents the recognized object, and determines a location of the forklift in the environment, based on the determined location of the recognized object in the environment, and the determined relative position between the forklift and the recognized object.

First claim

What is claimed is:

1. A computer-implemented method comprising:
recognizing a first object that is represented in first stereoscopic image data based on one or more stereoscopic images received from a first stereoscopic camera that has been affixed to a vehicle;
recognizing a second object that is represented in second stereoscopic image data based on one or more stereoscopic images received from a second stereoscopic camera that has been affixed to the vehicle;
identifying respective representations of the first object and the second object in a spatial model that tracks, for each of a plurality of objects, a corresponding location in an environment and a level of confidence that the corresponding location is the object's actual location;
determining a first location of the first object in the environment and a first level of confidence that the first location is the first object's actual location, according to the spatial model;
determining a second location of the second object in the environment and a second level of confidence that the second location is the second object's actual location, according to the spatial model; and
in response to determining that the first level of confidence for the first object's location is greater than the second level of confidence for the second object's location:
determining a relative position between the vehicle and the first object, based on a portion of the first stereoscopic image data that represents the first object; and
determining a location of the vehicle in the environment, based on the determined first location of the first object in the environment, and the determined relative position between the vehicle and the first object.

2. The computer-implemented method of claim 1 , wherein a level of confidence that a corresponding location is an object's actual location is proportional to an amount of time that the object has been at the corresponding location, according to the spatial model.

3. The computer-implemented method of claim 1 , wherein a level of confidence that a corresponding location is an object's actual location is a highest level of confidence when the object has been designated as a fixed location object.

4. The computer-implemented method of claim 1 , further comprising:
receiving, from at least one environmental sensor other than the stereoscopic camera, an environmental signal, wherein determining the location of the vehicle within the environment is based at least in part on the environmental signal.

5. The computer-implemented method of claim 3 , wherein the object is designated as a fixed location object when the object's location has not changed for a predetermined length of time.

6. The computer-implemented method of claim 3 , further comprising:
while the vehicle is within a predefined area associated with the fixed location object, determining locations of further recognized objects in the environment, including processing only a portion of the spatial model that corresponds to the predefined area.

7. The computer-implemented method of claim 1 , wherein the first and second stereoscopic image data are each based on a respective series of stereoscopic images received in real time as the images are captured by the first and second stereoscopic cameras, each stereoscopic image being captured at a fixed time interval.

8. A system, comprising:
a vehicle;
a first stereoscopic camera affixed to the vehicle;
a second stereoscopic camera affixed to the vehicle; and
a computing device communicatively coupled to each of the first and second stereoscopic cameras, the computing device configured to perform operations comprising:
recognizing a first object that is represented in first stereoscopic image data based on one or more stereoscopic images received from a first stereoscopic camera that has been affixed to a vehicle;
recognizing a second object that is represented in second stereoscopic image data based on one or more stereoscopic images received from a second stereoscopic camera that has been affixed to the vehicle;
identifying respective representations of the first object and the second object in a spatial model that tracks, for each of a plurality of objects, a corresponding location in an environment and a level of confidence that the corresponding location is the object's actual location;
determining a first location of the first object in the environment and a first level of confidence that the first location is the first object's actual location, according to the spatial model;
determining a second location of the second object in the environment and a second level of confidence that the second location is the second object's actual location, according to the spatial model; and
in response to determining that the first level of confidence for the first object's location is greater than the second level of confidence for the second object's location:
determining a relative position between the vehicle and the first object, based on a portion of the first stereoscopic image data that represents the first object; and
determining a location of the vehicle in the environment, based on the determined first location of the first object in the environment, and the determined relative position between the vehicle and the first object.

9. The system of claim 8 , wherein the vehicle is a forklift.

10. The system of claim 9 , wherein the first and second stereoscopic cameras are each affixed to an overhead guard of the forklift such that the first stereoscopic camera points away from the forklift and to a side of the forklift and the second stereoscopic camera points away from the forklift and to an opposite side of the forklift.

11. The system of claim 10 , wherein the first and second stereoscopic cameras each include a lens heater.

12. The system of claim 8 , the operations further comprising:
receiving, from at least one environmental sensor other than the stereoscopic camera, an environmental signal, wherein determining the location of the vehicle within the environment is based at least in part on the environmental signal.

13. The system of claim 8 , wherein the first and second stereoscopic image data are each based on a respective series of stereoscopic images received in real time as the images are captured by the first and second stereoscopic cameras, each stereoscopic image being captured at a fixed time interval.

14. A non-transitory computer-readable storage medium having instructions stored thereon which, when executed by one or more processors, cause the one or more processors to perform operations comprising:
recognizing a first object that is represented in first stereoscopic image data based on one or more stereoscopic images received from a first stereoscopic camera that has been affixed to a vehicle;
recognizing a second object that is represented in second stereoscopic image data based on one or more stereoscopic images received from a second stereoscopic camera that has been affixed to the vehicle;
identifying respective representations of the first object and the second object in a spatial model that tracks, for each of a plurality of objects, a corresponding location in an environment and a level of confidence that the corresponding location is the object's actual location;
determining a first location of the first object in the environment and a first level of confidence that the first location is the first object's actual location, according to the spatial model;
determining a second location of the second object in the environment and a second level of confidence that the second location is the second object's actua

Assignees

• Lineage Logistics Llc

Inventors

• Eckman Christopher Frank

Classifications

• G01S3/00

  Direction-finders for determining the direction from which infrasonic, sonic, ultrasonic or electromagnetic waves, or particle emission, not having a directional significance, are being received (position-fixing by co-ordinating a plurality of determinations of direction or position lines G01S5/00) · CPC title

• G06T7/74

  involving reference images or patches · CPC title

• G06T2207/30236

  Traffic on road, railway or crossing · CPC title

• G06Q10/0833 Primary

  Tracking · CPC title

• G01S5/16

  using electromagnetic waves other than radio waves · CPC title