System and method for robotic object detection using a convolutional neural network

Patent metadata

Field	Value
Publication number	US-11748903-B2
Application number	US-202016732322-A
Country	US
Kind code	B2
Filing date	Jan 1, 2020
Priority date	Jan 2, 2019
Publication date	Sep 5, 2023
Grant date	Sep 5, 2023

Abstract

A system includes a mobile robot, the robot comprising a sensor; and a server operably connected to the robot over a network, the robot being configured to detect an object by processing sensor data using a convolutional neural network. A pipeline for robotic object detection using a convolutional neural network includes: a system comprising a mobile robot, the robot comprising a sensor, the system further comprising a server operably connected to the robot over a network, the robot being configured to detect an object by processing sensor data using a pipeline, the pipeline comprising a convolutional neural network, the pipeline configured to perform a data collection step, the pipeline further configured to perform a data transformation step, the pipeline further configured to perform a convolutional neural network step, the pipeline further configured to perform a network output transformation step, the pipeline further configured to perform a results output step.

First claim

What is claimed is:

1. A system comprising:
a mobile robot, the robot comprising a two-dimensional (2D) sensor, the 2D sensor configured to provide spatial data about a frontward facing surface of one or more nearby objects, the 2D sensor further configured to provide infrared data related to one or more of a shape, a size, a type, a reflectivity, and a location of the object,
the robot further comprising a three-dimensional (3D) sensor, the 3D sensor configured to provide spatial data about the frontward facing surface of one or more nearby objects, the 3D sensor further configured to provide infrared data related to one or more of the shape, the size, the type, the reflectivity, and the location of the object; and
a server operably connected to the robot over a network, the server configured to manage the robot, the server further configured to provide the robot with location data regarding one or more of a location of the robot, a destination of the robot, and the location of the object;
the robot being configured to detect the object by processing sensor data using a convolutional neural network, the convolutional neural network being configured to determine one or more of information about the location of the object and information about the type of the object;
the server being configured to train the convolutional neural network while the server is not managing the robot by being in an offline status,
wherein the training comprises using a plurality of examples of an input to the convolutional neural network and a corresponding desired output from the convolutional neural network.

2. The system of claim 1 , wherein the 2D sensor comprises a light detection and ranging (LIDAR) sensor.

3. The system of claim 1 , wherein the object is selected from the group consisting of another robot, a forklift, a golf cart, an autonomous guided vehicles (AGV), a vehicle, and a shelf.

4. The system of claim 1 , wherein the type of the object is selected from the group consisting of a vehicle, a robot, a cart, a landmark, a stationary object, a warehouse, and an inventory shelf.

5. The system of claim 1 , wherein one or more of the 2D sensor and the 3D sensor is further configured to detect an object that is onboard the robot.

Assignees

• Fetch Robotics Inc
• Zebra Tech Corp

Inventors

• Cairl Brian

Classifications

• G06N3/0475

  Generative networks · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G05D1/223

  Command input arrangements on the remote controller, e.g. joysticks or touch screens · CPC title

• G05D1/617

  Safety or protection, e.g. defining protection zones around obstacles or avoiding hazards (arrangements for controlling the position or course of two or more vehicles for avoiding collisions therebetween G05D1/693; arrangements for reacting to or preventing system or operator failure G05D1/80) · CPC title