System and methods for object filtering and uniform representation for autonomous systems

Patent metadata

Field	Value
Publication number	US-2018373992-A1
Application number	US-201715633470-A
Country	US
Kind code	A1
Filing date	Jun 26, 2017
Priority date	Jun 26, 2017
Publication date	Dec 27, 2018
Grant date	—

Abstract

A computer-implemented method of controlling an autonomous system comprises: accessing, by one or more processors, sensor data that includes information regarding an area; disregarding, by the one or more processors, a portion of the sensor data that corresponds to objects outside of a region of interest; identifying, by the one or more processors, a plurality of objects from the sensor data; assigning, by the one or more processors, a priority to each of the plurality of objects; based on the priorities of the objects, selecting, by the one or more processors, a subset of the plurality of objects; generating, by the one or more processors, a representation of the selected objects; providing, by the one or more processors, the representation to a machine learning system as an input; and based on an output from the machine learning system resulting from the input, controlling the autonomous system.

First claim

What is claimed is:

1 . A computer-implemented method of controlling an autonomous system, comprising:
accessing, by one or more processors, sensor data that includes information regarding an area; disregarding, by the one or more processors, a portion of the sensor data that corresponds to objects outside of a region of interest included in the area; identifying, by the one or more processors, a plurality of objects from the sensor data; assigning, by the one or more processors, a priority to each of the plurality of objects; based on the priorities of the objects, selecting, by the one or more processors, a subset of the plurality of objects; generating, by the one or more processors, a representation of the selected objects; providing, by the one or more processors, the representation to a machine learning system as an input; and controlling the autonomous system based on an output from the machine learning system resulting from the input.

2 . The computer-implemented method of claim 1 , wherein:
the region of interest is defined by a sector map comprising a plurality of sectors, each sector of the sector map being defined by an angle range and a distance from the autonomous vehicle.

3 . The computer-implemented method of claim 2 , wherein, at least two sectors of the plurality of sectors are defined by different distances from the autonomous system.

4 . The computer-implemented method of claim 1 , wherein:
the region of interest includes a segment for each of one or more lanes.

5 . The computer-implemented method of claim 4 , wherein:
the disregarding of the sensor data generated by the objects outside of the region of interest comprises:
identifying a plurality of objects from the sensor data;
for each of the plurality of objects:
identifying a lane based on sensor data generated from the object; and
associating the identified lane with the object; and

disregarding sensor data generated by objects associated with a predetermined lane.

6 . The computer-implemented method of claim 1 , further comprising:
based on the sensor data and a set of criteria, switching the region of interest from a first region of interest to a second region of interest in the area, the first region of interest being defined by a sector map comprising a plurality of sectors, each sector of the sector map being defined by an angle range and a distance from the autonomous system, the second region of interest including a segment for each of one or more lanes.

7 . The computer-implemented method of claim 1 , further comprising:
based on the sensor data and a set of criteria, switching the region of interest from a first region of interest to a second region of interest, the first region of interest including a segment for each of one or more lanes, the second region of interest being defined by a sector map comprising a plurality of sectors, each sector of the sector map being defined by an angle range and a distance from the autonomous system.

8 . The computer-implemented method of claim 1 , wherein:
the region of interest includes a height.

9 . The computer-implemented method of claim 1 , wherein the selecting of the subset of the plurality of objects comprises selecting a predetermined number of the plurality of objects.

10 . The computer-implemented method of claim 9 , wherein the selecting of the subset of the plurality of objects comprises selecting the subset of the plurality of objects having priorities above a predetermined threshold.

11 . The computer-implemented method of claim 1 , wherein:
the generated representation is a uniform representation that matches a representation used to train the machine learning system; and the uniform representation is a two-dimensional image

12 . The computer-implemented method of claim 11 , wherein:
the generating of the two-dimensional image comprises encoding a plurality of attributes of each selected object into each of a plurality of channels of the two-dimensional image.

13 . The computer-implemented method of claim 11 , wherein the generating of the two-dimensional image comprises:
generating a first two-dimensional image; and generating the two-dimensional image from the first two-dimensional image using a topology-preserving downsampling.

14 . The computer-implemented method of claim 1 , wherein:
the representation is a uniform representation that matches a representation used to train the machine learning system; and the uniform representation is a vector of fixed length.

15 . The computer-implemented method of claim 14 , wherein:
the generating of the vector of fixed length comprises adding one or more phantom objects to the vector, each phantom object being semantically meaningful.

16 . The computer-implemented method of claim 15 , wherein each phantom object has a speed attribute that matches a speed of the autonomous system.

17 . An autonomous system controller comprising:
a memory storage comprising instructions; and one or more processors in communication with the memory storage, wherein the one or more processors execute the instructions to perform:
accessing sensor data that includes information regarding an area;
disregarding a portion of the sensor data that corresponds to objects outside of a region of interest included in the area;
identifying a plurality of objects from the sensor data;
assigning a priority to each of the plurality of objects;
based on the priorities of the objects, selecting a subset of the plurality of objects;
generating a representation of the selected objects;
providing the representation to a machine learning system as an input; and
controlling the autonomous system based on an output from the machine learning system resulting from the input.

18 . The autonomous system controller of claim 14 , wherein:
the region of interest is defined by a sector map comprising a plurality of sectors, each sector of the sector map being defined by an angle range and a distance from the autonomous system.

19 . The autonomous system controller of claim 18 , wherein at least two sectors of the plurality of sectors are defined by different distances from the autonomous system.

20 . A non-transitory computer-readable medium storing computer instructions for controlling an autonomous system, that when executed by one or more processors, cause the one or more processors to perform steps of:
accessing sensor data that includes information regarding an area; disregarding a portion of the sensor data that corresponds to objects outside of a region of interest included in the area; identifying a plurality of objects from the sensor data; assigning a priority to each of the plurality of objects; based on the priorities of the objects, selecting a subset of the plurality of objects; generating a representation of the selected objects; providing the representation to a machine learning system as an input; and based on an output from the machine learning system resulting from the input, controlling the autonomous system.

Assignees

• Futurewei Technologies Inc

Inventors

• Yin Xiaotian
• Liu Lifeng
• Zhu Yingxuan
• Zhang Jun
• Li Jian

Classifications

• G06V10/70

  using pattern recognition or machine learning (optical pattern recognition or electronic computations therefor G06V10/88) · CPC title

• G06V10/82

  using neural networks · CPC title

• G06V10/25

  Determination of region of interest [ROI] or a volume of interest [VOI] · CPC title

• G06N3/008 Primary

  based on physical entities controlled by simulated intelligence so as to replicate intelligent life forms, e.g. based on robots replicating pets or humans in their appearance or behaviour · CPC title

• G06N5/04 Primary

  Inference or reasoning models · CPC title