Method and a system for hierarchical network based diverse trajectory proposal

Patent metadata

Field	Value
Publication number	US-11526174-B2
Application number	US-202016894411-A
Country	US
Kind code	B2
Filing date	Jun 5, 2020
Priority date	Jun 7, 2019
Publication date	Dec 13, 2022
Grant date	Dec 13, 2022

Abstract

The disclosure herein generally relates to the field of autonomous navigation, and, more particularly, to a diverse trajectory proposal for autonomous navigation. The embodiment discloses a hierarchical network based diverse trajectory proposal for autonomous navigation. The hierarchical 2-stage neural network architecture maps the perceived surroundings to diverse trajectories in the form of trajectory waypoints, that an autonomous navigation system can choose to navigate/traverse. The first stage of the disclosed hierarchical 2-stage Neural Network architecture is a Trajectory Proposal Network which generates a set of diverse traversable regions in an environment which can be occupied by the autonomous navigation system in the future. The second stage is a Trajectory Sampling network which predicts a fine-grained trajectory/trajectory waypoint over the diverse traversable regions proposed by Trajectory Proposal Network.

First claim

We claim:

1. A processor-implemented method for a hierarchical network based diverse trajectory proposal for a plurality of autonomous navigation systems, comprising:
receiving input sensor data and a trajectory mask, by one or more hardware processors, wherein
the input sensor data is received from a plurality of sensors, and
the trajectory mask is a set of grid cells occupied during a navigation track history of the plurality of autonomous navigation systems;

pre-processing the input sensor data, by one or more hardware processors, to obtain an occupancy map based on a mapping technique, wherein
the mapping technique is based on a discrete 2D occupancy grid map technique,
the occupancy map is an intermediate representation of the input sensor data, and
the pre-processing further includes:
transforming a history of an autonomous navigation system of the plurality of autonomous navigation system to an autonomous navigation system coordinate frame; and
discretizing the autonomous navigation system coordinate frame to form a grid;

processing the occupancy map to obtain a plurality of binary masks, wherein
the obtained plurality of binary masks is suitable for training a Convolution Neural Network (CNN),
the processed occupancy map includes at least three mutually exclusive binary masks of the plurality of binary masks;

generating, by the one or more hardware processors, a ground truth trajectory using the processed occupancy map and the received trajectory mask, wherein the generation of the ground truth trajectory is based on a rapidly exploring random tree (RRT) star technique;
training a trajectory proposal network (TPNet), using a multiple choice learning technique, based on the generated ground truth trajectories, wherein
the multiple choice learning technique includes a set of loss functions,
the TPNet is further trained, using deep supervision, by computing a specific Trajectory Diversity Loss at two different levels of the TPNet,
the two different levels include a first level and a second level,
the first level corresponds to an outermost layer of the TPNet, and
the second level corresponds to a level immediately after an outermost decoder layer of the TPNet;

predicting, by the one or more hardware processors, a set of diverse traversable regions for the autonomous navigation system of the plurality of autonomous navigation systems using the TPNet, wherein
the set of loss functions includes a trajectory diversity loss (Ltd), and
the trajectory diversity loss (Ltd) is computed based on a weighted entropy loss between the ground truth trajectory and associated trajectory output;

identifying, by the one or more hardware processors, a goal point for each of the set of diverse traversable regions based on a goal identification technique, wherein
the goal point is a final co-ordinate position among a plurality of co-ordinate positions to be reached by the autonomous navigation system in a current map,
the current map is a real time map received by the autonomous navigation system,
the autonomous navigation system is in a specific co-ordinate position of the plurality of co-ordinate positions in the current map;

identifying, using the goal identification technique, a specific probability co-ordinate position in a diverse traversable region of the set of diverse traversable regions, wherein
the specific probability co-ordinate position is greater than a pre-defined threshold, and
the specific probability co-ordinate position is farthest from the specific co-ordinate position of the autonomous navigation system in the current map;

generating, by the one or more hardware processors, a set of ground truth waypoints for each of the set of diverse traversable regions, wherein
the set of ground truth waypoints is between the goal point and the specific co-ordinate position of the autonomous navigation system in the current map, and
the set of ground truth waypoints is generated using a rapidly exploring random tree (RRT star) technique;

predicting, by the one or more hardware processors, a set of trajectory waypoints for each of the set of diverse traversable regions to obtain a set of diverse trajectories using a trajectory sampler network (TSNet), wherein
the TSNet is a specific Convolutional Neural Network (CNN)—a Long Short Term Memory (LSTM) network trained using the set of diverse traversable regions and the set of ground truth waypoints, and
the set of trajectory waypoints is a set of co-ordinate positions along the set of diverse trajectories; and

displaying the set of diverse trajectories along with associated set of trajectory waypoints.

2. The method of claim 1 , wherein the occupancy map includes a set of occupied space (O 1 ), a set of free space (O 2 ), and a specific set of unknown space (O 3 ).

3. The method of claim 1 , wherein
the plurality of sensors includes a laser scanner, a sonar, and a multi-camera system.

4. The method of claim 1 , wherein a discrete occupancy grid map is based on a discretization bayesian map, a dempster-shafer map and a fuzzy map.

5. The method of claim 1 , wherein the set of loss functions further includes an obstacle avoidance loss (Lobs).

6. The method of claim 5 , wherein
the obstacle avoidance loss (Lobs) penalizes the TPNet for every traversable region prediction that intersects with an obstacle, and
the obstacle avoidance loss (Lobs) is predicted by minimizing a negative log likelihood of a trajectories at an obstacle location.

7. The method of claim 1 , wherein
the trajectory diversity loss (Ltd) enables the TPNet to predict the set of diverse traversable regions.

8. The method of claim 1 , wherein the autonomous navigation system includes a robot, a self-driven car, and a drone.

9. A system for a hierarchical network based diverse trajectory proposal for a plurality of autonomous navigation systems, comprising:
a memory storing instructions;
one or more Input/Output (I/O) interfaces; and
one or more hardware processors coupled to the memory via the one or more I/O interfaces, wherein the one or more hardware processors are configured by the instructions to:
receive input sensor data wherein
the input sensor data is received from a plurality of sensors, and
the trajectory mask is a set of grid cells occupied during a navigation track history of the plurality of autonomous navigation systems;

pre-process the input sensor data to obtain an occupancy map based on a mapping technique, wherein
the mapping technique is based on a discrete 2D occupancy grid map technique,
the occupancy map is an intermediate representation of the input sensor data, and
the pre-processing further includes:
transforming a history of an autonomous navigation system of the plurality of autonomous navigation system to an autonomous navigation system coordinate frame; and
discretizing the autonomous navigation system coordinate frame to form a grid;

process the occupancy map to obtain a plurality of binary masks, wherein
the obtained plurality of binary masks is suitable for training a Convolution Neural Network (CNN),
the processed occupancy map includes at least three mutually exclusive binary masks of the plurality of binary masks;

generate a ground truth trajectory using the processed occupancy map and the received trajectory mask, wherein the generation of the ground truth trajectory is based on a rapidly exploring random tree (RRT) technique;
train a trajectory proposal network (TPNet), using a multiple choice learning technique, based on the generated ground truth trajectories, wherein
the multiple choice learning technique includes a set of loss functions,
the TPNet is further trained, using deep supervision, by comput

Assignees

• Tata Consultancy Services Ltd

Inventors

• Bhowmick Brojeshwar
• Krishna Krishnam Madhava
• Narayanan Sriram Nochur
• KUMAR Gourav
• Singh Abhay
• Mustikovela Siva Karthik
• Saurav Saket

Classifications

• G06N7/01

  Probabilistic graphical models, e.g. probabilistic networks · CPC title

• G06N3/045

  Combinations of networks · CPC title

• G06N3/044

  Recurrent networks, e.g. Hopfield networks · CPC title

• G05B13/027

  using neural networks only · CPC title

• G06N3/08

  Learning methods · CPC title