Efficient situational awareness by event generation and episodic memory recall for autonomous driving systems

Patent metadata

Field	Value
Publication number	US-10409279-B2
Application number	US-201715420946-A
Country	US
Kind code	B2
Filing date	Jan 31, 2017
Priority date	Jan 31, 2017
Publication date	Sep 10, 2019
Grant date	Sep 10, 2019

Abstract

A system and method is taught for data processing in an autonomous vehicle control system. Using information is acquired from the vehicle, network interface, and sensors mounted on the vehicle, the system can perceive situations around it with much less complexity in computation without losing crucial details, and then make navigation and control decisions. The system and method are operative to generate situation aware events, store them, and recall to predict situations for autonomous driving.

First claim

What is claimed is:

1. A method comprising:
receiving a first plurality of information via a vehicle sensor;
receiving a second plurality of information via a network interface;
generating a combined coordinate system in response to a first coordinate system centered on a moving object and a second coordinate system centered on a fixed location;
generating an episode in response to a temporal ordering of the first plurality of information and the second plurality of information and the combined coordinate system;
incrementing a counter corresponding to the episode;
generating a header in response to the combined coordinate system and the temporal ordering of the first plurality of information and the second plurality of information wherein the header is used to characterize the episode and wherein the header is used as part of a tree structure based classification system for storing the episode;
storing the episode in a memory in response to the counter exceeding a threshold value;
receiving a third plurality of information via the vehicle sensor;
receiving a fourth plurality of information via the network interface;
generating an event in response to the third plurality of information and the fourth plurality of information;
comparing the episode to the event to generate a predicted outcome of the event wherein the event corresponds to a first temporal portion of the episode and the predicted outcome of the event corresponds to a second temporal portion of the episode;
generating a control signal in response to the predicted outcome; and
controlling a vehicle in response to the control signal.

2. The method of claim 1 further comprising generating a header in response to the temporal ordering of the first plurality of information and the second plurality of information wherein the header includes environment type.

3. The method of claim 1 wherein the episode is generated in comparison to an encoding threshold for encoding the episodes into a memory.

4. A vehicle control system for controlling a vehicle comprising:
a vehicle sensor for receiving a first plurality of information and a third plurality of information;
a network interface for receiving a second plurality of information and a fourth plurality of information;
a counter corresponding to an episode;
a processor for generating the episode in response to a temporal ordering of the first plurality of information and the second plurality of information, generating a combined coordinate system in response to a first coordinate system centered on a moving object and a second coordinate system centered on a fixed location wherein the episode is generated in response to the combined coordinate system, storing the episode in a memory in response to the counter exceeding a threshold value, for generating an event in response to the third plurality of information and the fourth plurality of information, for comparing the episode to the event to generate a predicted outcome of the event, wherein the event corresponds to a first temporal portion of the episode and the predicted outcome of the event corresponds to a second temporal portion of the episode and for generating a control signal in response to the predicted outcome, and wherein the processor is further operative to generate a header in response to the combined coordinate system and the temporal ordering of the first plurality of information and the second plurality of information wherein the header is used to characterize the episode and wherein the header is used as part of a tree structure based classification system for storing the episode; and
a controller for controlling the vehicle in response to the control signal.

5. The apparatus of claim 4 wherein the processor is further operative to generate a header in response to the temporal ordering of the first plurality of information and the second plurality of information wherein the header includes environment type.

6. The apparatus of claim 4 wherein the episode is generated after comparison to an encoding threshold for encoding the episodes into a memory.

Assignees

• Gm Global Tech Operations Llc

Inventors

• Kwon Hyukseong
• Cho Youngkwan
• Bhattacharyya Rajan
• Daily Michael J

Classifications

• G06N5/02 Primary

  Knowledge representation; Symbolic representation · CPC title

• G06N7/01

  Probabilistic graphical models, e.g. probabilistic networks · CPC title

• H04L67/12

  specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks · CPC title

• B60W50/00

  Details of control systems for road vehicle drive control not related to the control of a particular sub-unit {, e.g. process diagnostic or vehicle driver interfaces} · CPC title

• B60W2050/0002

  Automatic control, details of type of controller or control system architecture · CPC title