Parallel scene primitive detection using a surround camera system

Patent metadata

Field	Value
Publication number	US-10402670-B2
Application number	US-201715487753-A
Country	US
Kind code	B2
Filing date	Apr 14, 2017
Priority date	Apr 19, 2016
Publication date	Sep 3, 2019
Grant date	Sep 3, 2019

Abstract

Techniques for road scene primitive detection using a vehicle camera system are disclosed. In one example implementation, a computer-implemented method includes receiving, by a processing device having at least two parallel processing cores, at least one image from a camera associated with a vehicle on a road. The processing device generates a plurality of views from the at least one image that include a feature primitive. The feature primitive is indicative of a vehicle or other road scene entities of interest. Using each of the parallel processing cores, a set of primitives are identified from one or more of the plurality of views. The feature primitives are identified using one or more of machine learning and classic computer vision techniques. The processing device outputs, based on the plurality of views, result primitives based on the plurality of identified primitives from multiple views based on the plurality of identified entities.

First claim

What is claimed is:

1. A computer-implemented parallelization method comprising:
receiving, by a processing device having at least two parallel processing cores, at least one image from a camera associated with a vehicle on a road;
generating, by the processing device, a plurality of views from the at least one image that include a feature primitive indicative of one or more road scene entities of interest;
identifying a road scene entity from one or more of the plurality of views with each of the at least two parallel processing cores simultaneously using one or more of machine learning and a computer vision technique, wherein the at least two parallel processing cores each identify the road scene entity in a separate portion of the one or more of the plurality of views; and
outputting, based on the plurality of views, a set of result road scene primitives in parallel based on the identified road scene entity.

2. The computer-implemented method of claim 1 , wherein the set of result road scene primitives is indicative of a road scene entity comprising one of a pedestrian, a traffic sign, a traffic signal, and a road feature.

3. The computer-implemented method of claim 1 , wherein the machine learning utilizes a convolutional neural network.

4. The computer-implemented method of claim 1 , wherein identifying the road scene entity at each parallel processing core further comprises:
performing, at each parallel processing core, a feature extraction to extract the road scene entity from one view using a neural network; and
performing a classification of the road scene entity using the neural network.

5. The computer-implemented method of claim 1 , wherein one parallel processing core identifies the road scene entity from a different view than another of the parallel processing cores.

6. The computer-implemented method of claim 5 , further comprising identifying at least two different road scene entities and outputting a result indicative of the two different road scene entities.

7. A system for parallelization, the system comprising:
a plurality of cameras associated with a vehicle;
a memory comprising computer readable instructions; and
a processing device having at least two parallel processing cores configured to:
receive at least one image from a camera associated with a vehicle on a road;
generate a plurality of views from the at least one image that include a feature primitive indicative of one or more road scene entities of interest;
identify a road scene entity from one or more of the plurality of views with each of the at least two parallel processing cores simultaneously using one or more of machine learning and a computer vision technique, wherein the at least two parallel processing cores each identify the road scene entity in a separate portion of the one or more of the plurality of views; and
output, based on the plurality of views, a set of result road scene primitives based on the identified road scene entity.

8. The system of claim 7 , wherein the set of result road scene primitives is indicative of a road scene entity comprising one of a pedestrian, a traffic sign, a traffic signal, and a road feature.

9. The system of claim 7 , wherein the machine learning utilizes a convolutional neural network.

10. The system of claim 7 , wherein identifying the road scene entity at each parallel processing core further comprises:
performing, at each parallel processing core, a feature extraction to extract the road scene entity from one view using a neural network; and
performing a classification of the road scene entity using the neural network.

11. The system of claim 7 , wherein one parallel processing core analyzes a different view than another of the parallel processing cores.

12. The system of claim 11 , further comprising identifying at least two different feature primitives and outputting a result indicative of the two different feature primitives.

13. The system of claim 7 , wherein the camera comprises a fisheye lens.

14. A computer program product for parallel scene primitive detection, the computer program product comprising:
a computer readable storage medium having program instructions embodied therewith, wherein the computer readable storage medium is not a transitory signal per se, the program instructions executable by a processing device having at least two parallel processing cores to cause the processing device to perform a method comprising:
receiving, by a processing device having at least two parallel processing cores, at least one image from a camera associated with a vehicle on a road;
generating, by the processing device, a plurality of views from the at least one image that include a feature primitive indicative of one or more road scene entity of interest;
identifying a road scene entity from one or more of the plurality of views with each of the at least two parallel processing cores simultaneously using one or more of machine learning and a computer vision technique, wherein the at least two parallel processing cores each identify the road scene entity in a separate portion of the one or more of the plurality of views; and
outputting, based on the plurality of views, a set of result road scene primitives in parallel based on the identified road scene entity.

15. The computer program product of claim 14 , wherein the set of result road scene primitives is indicative of a road scene entity comprising one of a pedestrian, a traffic sign, a traffic signal, and a road feature.

16. The computer program product of claim 14 , wherein the machine learning utilizes a convolutional neural network.

17. The computer program product of claim 14 , wherein identifying the feature primitive at each parallel processing core further comprises:
performing, at each parallel processing core, a feature extraction to extract the road scene entity from one view using a neural network; and
performing a classification of the road scene entity using the neural network.

18. The computer program product of claim 14 , wherein one parallel processing core identifies the road scene entity from a different view than another of the parallel processing cores.

19. The computer program product of claim 18 , further comprising identifying at least two different road scene entities and outputting a result indicative of the two different road scene entities.

20. The computer-implemented method of claim 1 further comprising:
identifying and classifying the feature primitive;
determining a road feature type in response to the classification of the feature primitive being a road feature;
determining a vehicle feature type in response to the classification of the feature primitive being a vehicle; and
determining a sign or signal in response to the classification of the feature primitive being a road marking.

Assignees

• Gm Global Tech Operations Llc

Inventors

• Zou Guangyu J
• Mudalige Upali P
• Wang Jinsong

Classifications

• G06V10/955 Primary

  using specific electronic processors · CPC title

• H04N23/90

  Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums · CPC title

• G06F18/2414

  Smoothing the distance, e.g. radial basis function networks [RBFN] · CPC title

• H04N23/63

  by using electronic viewfinders · CPC title

• G06F18/2411

  based on the proximity to a decision surface, e.g. support vector machines · CPC title